Economic Voting in Democracies. The theory of economic voting behind this claim is that economic conditions powerfully shape electoral outcomes in democracies everywhere. As Michael Lewis-Beck puts it in a great review article, "good times keep parties in office, bad times cast them out."

I should note that the evidence for this effect and its size varies a lot across countries, and the sometimes puzzling variation in the size of economic effects remains an open area of inquiry in political science. 

For instance, when the government is supported by a coalition of several parties, it's harder for voters to figure out which members deserve the blame for bad performance. The lack of a credible alternative to the incumbent--an opposition party or candidate who appears likely to do better--also leads to a weaker effect. (The opposition to the LDP in Japan has long struggled with a credibility problem, for instance.) And sometimes it's clear to voters that governments don't have much influence at all over bad economic outcomes because of global factors beyond their control, so they are less likely to punish incumbents at the ballot box.

In addition, voters turn out to have really short memories (i.e. they're "myopic," in the jargon of the discipline): the performance of the economy over the last six months matters a great deal more than the performance over a government's whole term in office. This is probably why the Conservatives in Britain, for instance, recently won re-election after presiding over an austerity-induced downturn during much of their first term.

Nevertheless, the basic claim, that economic downturns motivate voters to vote out incumbent governments when they can, is quite robust. In the United States, in fact, the state of the economy in the few months before a presidential election appears to be the single most important factor in who wins, more than the candidates themselves, their parties' policy platforms, or their campaigns.