Is the Refi 'Apocalypse' Really Upon Us?

Mortgage rates today are very low, but U.S. borrowers have a very short memory. They forget that the rate on the 30-year fixed, which sits around 3.6 percent today, was a full percentage point higher a year ago, and above 5 percent in January of 2010. The purchasing power gained through today's low rates have arguably helped fuel the recovery in home sales. Low rates have also sparked a boom in mortgage refinancing, which in turn has put more spending money in consumers' pockets.

Still, the slightest move higher has dramatic effects.

(Read More: US Mortgage Applications Fall as Refi Demand Drops )

Witness the 10 percent drop in refinance applications from a week ago, on the Mortgage Bankers Association's weekly report. The rate on the 30-year fixed moved from 3.62 percent to 3.67 percent.

"We're busy because we're pushing," says Julian Hebron at California-based RPM Mortgage. "Mostly people have been lulled into complacency by long-term rate lows, and it's common for them to maintain their 'It'll come back down' stance when rates rise. But rates are now up .375 percent, and it may hold given MBS/Treasury market technicals and moderately improving economic fundamentals. The complacency has a lot to do with rates having been low enough to make no-cost refis easy. But when rates rise this much, the no-cost options go away and people tend to wake up."

In Bethesda, Maryland, Apex Home Loans CEO, Craig Strent, says a rise in rates could actually bring in more business in the short term.

"There is a huge population that have benefitted from adjustable rate mortgages. When the rates adjusted, they adjusted down. Those homeowners have been riding those low, one-year arms. If they start to hear about rates going up, they may come out of the woodwork to lock into fixed rates," says Strent.

That may be, but 88 percent of loans outstanding today are fixed, according to the Mortgage Bankers Association. Just 12 percent are adjustable rate. Even if rates do not rise any higher than they are today, which they may not, they would have to fall below last year's lows to see the high refinance volume of 2012 continue in 2013.

(Read More: Link Between Credit and Mortgages: Not What You Think)

"The refi apocalypse is upon us," says Mark Hanson, a mortgage analyst in Northern California. "The thought is that there are a bunch of homeowners on the fence who haven't refi'd who will all jump in thinking they will miss out. The theory is 100 percent nonsense. The series will simply plunge. That's because after 16 months of sub 4 percent rates -- and every bank loan officer and mortgage broker doing everything they can after a long mortgage banking income drought that ended with Twist -- there is nobody left to refi. In fact, the only reason refi applications stayed flat in Q3 and Q4 was because they passed a new law allowing refinances regardless of the LTV [loan to value]...the HARP unlimited LTV refi."

While the Federal Reserve does not set mortgage rates, a signal that the economic recovery is improving and even the slightest hint that the Fed could end its purchases of mortgage-backed securities, could push rates slightly higher.

"The Fed likely won't use its statement to markets to finger a specific date on which QE3 will end, but that won't stop investors from guessing. If the herd believes that QE3 will terminate within the next 6 months, mortgage rates will likely rise. If QE3 is believed to extend into 2014 and beyond, mortgage rates will likely fall," writes Dan Green of Waterstone Mortgage in his blog.

(Read more: What to Expect from Interest Rates This Year)

While refinances may suffer under even slightly higher rates, more important to the housing recovery is new mortgages to purchase homes. Purchase applications are still running at half the rate they were in 2007, when last the Dow hit a new high. Small moves in mortgage rates do affect purchasing power, but lending standards are a far bigger driver today. New regulations for lenders and a consolidation of lending overall to the mega-banks are certainly slowing, and in some cases stalling, the process for some would-be buyers.

(Read More: Cities That Are Most Prepared for Retirement)

—By CNBC's Diana Olick; Follow her on Twitter @Diana_Olick or on Facebook at facebook.com/DianaOlickCNBC

Questions? Comments? RealtyCheck@cnbc.com