May 18, 2011
Dean Baker
TPMCafé, May 17, 2011
See article on original website.
Yes, they are back again. We have another paper claiming that the stimulus did not create jobs. Timothy Conley and Bill Dupor, professors at Western Ontario University and Ohio State respectively, have a new study of state level employment that purports to show that the stimulus cost more jobs in the private sector than it created in the public sector. I’ll just quickly note a few problems with the paper.
With an exercise like this, you always have to worry about the problem of cherry picking. It is very easy to run 1000 regressions in an hour. Inevitably, you find 4 or 5 of these 1000 that show you almost anything. (Our standard of significance is a result that you would not get by random chance more than 10 times in a hundred. This means that if you ran 1000 regressions of things that had nothing to do with each other, you would expect 100 of them to have statistically significant results.)
For this reason, you usually want to run your regressions a variety of different ways to show that the results do not depend on some arbitrary specification. It doesn’t look like they have done this, or at least they did not show much evidence of such robustness tests in their paper.
Their results depend on pulling out four private sector industry groups (lumped together) and measuring the stimulus against trend job growth in these industries. Even for these four industry groups, most of the results are only marginally significant. It is clear from their tables that if they took all private sector jobs, their results would be insignificant. So, how did they decide on lumping these four industry groups together? It certainly is not a standard break out. It does raise a suspicion that they ran many different regressions and then discovered that they got the results they wanted with these four industries lumped together.
There are many other peculiar items here. Their instrumental variable for stimulus spending is very strange. While it makes some sense, it would be interesting to see how the results are affected by using other equally plausible instruments. They do some sensitivity analysis here, but not nearly as much as I would like to see.
What about the length of the employment trends used in the analysis? It would be interesting to see if the results are sensitive to this, especially when we had such an extraordinary period. Did they test for different trend lengths? If they did, they didn’t show it.
They also have the peculiar result that in one specification they find no significant effect of stimulus on public sector job creation, yet do find a significant loss of jobs in the private sector. Both sides of this are troubling. It really is hard to believe that the stimulus did not even create jobs (or prevent job loss) in the public sector. What exactly did those boneheads do with the money, eat it? In you didn’t find that the stimulus created jobs in the public sector, then it seems likely that your instrumental variable is not capturing the effect of the stimulus very well.
The other problem is that their story of private sector job loss depends on the stimulus actually creating jobs in the public sector. Their story is that the stimulus employed people in the public sector who otherwise would have been employed in the private sector. If the stimulus didn’t actually employ anyone in the public sector, then how do we explain the job loss in the private sector?
It also would have been nice to see a variable for the drop in house prices by state. The economics profession as a whole was too thick to notice the $8 trillion housing bubble on the way up, or to realize that its collapse would have any impact on the economy. Now that the collapse of this bubble has led to the worst downturn since the Great Depression, one might think that economists would finally start paying attention to it.
Helene Jorgensen and I ran a few regressions on employment that had the decline in house prices as an independent variable. The results were highly significant in every specification. A few are shown here. (We controlled for reverse causation by taking the price decline in the period prior to the big plunge in employment.) At this point, it should be economic malpractice to run state employment regressions without including a housing price variable.
One last point that is very peculiar, they divided the stimulus by state spending rather than state population or GDP. This implies that $1 billion in stimulus spending should create more jobs in a state with a small budget than a large budget. I can’t see any reason why this would be the case.
In short, there are many unusual aspects to this analysis and very little effort to determine whether these quirks are driving the results. For my money the analysis by Feyrer and Sacerdot is a far more serious effort to measure the state by state effects of the stimulus. I have a short discussion of the paper here.
