February 06, 2016
The Washington Post had an article on a vote in West Virginia to make it a “right-to-work” state. This means that workers cannot sign enforceable contracts with employers, which require that workers covered by the contract pay the union their share of its operating costs.
The piece refers to a study that purported to find that states with “right-to-work” (RTW) laws had more job growth than other states. This study was fundamentally flawed in its design, since it was treated as a time series study, using years as individual observations, when it really was a cross section study.
Only one state actually switched from being non-RTW to being RTW in the period analyzed. (The study wrongly has both Texas and Utah making this switch, when in fact they just adopted stronger RTW laws.) This means that they really only have data on the 50 states, not separate observations for each year for each state. Treating the analysis as a time-series in this way gives an illusion of having much more data than is actually available. This makes it possible to get statistical significant results when it almost certainly would not be possible if the analysis were done correctly.
To see this point, imagine the study had used monthly data instead of annual data. This would give them twelve times as many data points, even though there was no additional information on the relevant variable. If a test finds statistically significant results in this context, which would not be present in a simple cross section analysis, then these results are clearly being driven by other factors, not a state’s RTW status.
Comments