The Americas Blog seeks to present a more accurate perspective on economic and political developments in the Western Hemisphere than is often presented in the United States. It will provide information that is often ignored, buried, and sometimes misreported in the major U.S. media.
Spanish description lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc in arcu neque. Nulla at est euismod, tempor ligula vitae, luctus justo. Ut auctor mi at orci porta pellentesque. Nunc imperdiet sapien sed orci semper, finibus auctor tellus placerat. Nulla scelerisque feugiat turpis quis venenatis. Curabitur mollis diam eu urna efficitur lobortis.
• BoliviaBoliviaElectionsEleccionesLatin America and the CaribbeanAmérica Latina y el CaribeWorldEl Mundo
• IMFFondo Monetario InternacionalLatin America and the CaribbeanAmérica Latina y el CaribeUS Foreign PolicyPolítica exterior de EE. UU.VenezuelaVenezuela
• BoliviaBoliviaLatin America and the CaribbeanAmérica Latina y el Caribe
• IMFFondo Monetario InternacionalLatin America and the CaribbeanAmérica Latina y el CaribeSanctionsSpotlightUS Foreign PolicyPolítica exterior de EE. UU.VenezuelaVenezuelaWorldEl Mundo
On December 5, 2022, the International Monetary Fund’s (IMF) Western Hemisphere Department published a report titled “Regional Spillovers from the Venezuelan Crisis,” which assesses the causes of Venezuela’s economic crisis, the drivers of the country’s record emigration, and the impact that this influx of Venezuelan migrants has had on neighboring countries. While these are worthy topics of research, and there is much of value in the report, authors Alvarez et al. curiously omit a critical piece of the puzzle, and one of the single most important factors contributing to Venezuela’s current economic and humanitarian plight: US economic sanctions.
In August 2017, the Trump administration issued Executive Order 13808, barring the government of Venezuela, including the state-owned oil company Petróleos de Venezuela, S.A. (PDVSA) and its joint ventures, from accessing US financial markets. Though the United States had imposed sanctions on certain Venezuelan individuals and entities before this, including under the Obama administration’s E.O. 13692, which declared a US national emergency with respect to Venezuela, the August 2017 sanctions marked the beginning of a series of sweeping sanctions that would define the Trump administration’s approach to US-Venezuelan relations. Sanctions were escalated even further alongside the recognition of a parallel government beginning in 2019, most notably with the January 28 designation of PDVSA as a sanctioned entity, and the 2020 imposition of secondary sanctions against shipping companies involved in the transportation of Venezuelan oil. The vast majority of these sanctions remain in place today.
The impact of these sanctions has been swift and disastrous, particularly on Venezuela’s oil output, which is the country’s primary source of foreign revenue. While there are no doubt multiple factors that have contributed to Venezuela’s precipitous drop in oil production — from 2.4 million barrels per day (bpd) prior to the crisis, to a low of 0.4 million bpd in mid-2020 — a preponderance of evidence points to US sanctions as a significant driver of the decline. A 2022 analysis by one of the authors of this post, Francisco Rodríguez, attributes the loss of 797,000 bpd to the 2017 financial sanctions. Other studies using different methodologies have set this figure at 698,000 bpd (by Equipo Anova[1]), and from 616,000 to 1,023,000 bpd (by Luis Oliveros). And a recent paper by Rodríguez in the Latin American Economic Review uses variation in production across the country’s Orinoco basin to estimate the impact of sanctions at 255,000 to 637,000 bpd.
Despite these well-documented impacts, US sanctions on Venezuela are mentioned in the body of the 61-page IMF report only twice. Once, the report does appear to imply some adverse effects of sanctions, albeit while suggesting that they have been mitigated: “Venezuela has been able to place its heavy crude oil in the Asian market at a substantial price discount, alleviating in part the impact of sanctions.” However, the other mention of sanctions specifically downplays their impact: “The sharp decline [in oil output], which preceded the introduction of oil sanctions by the United States in January 2019, reflected both internal and external factors.”[2]
This latter assertion is an oft-repeated claim, one that is misleading at best. While it is true that the start of the decline in oil production, and indeed the beginning of the economic crisis itself, preceded the 2017 financial sanctions, this is hardly evidence that the multiple waves of sanctions have not had a significant causal effect. In fact, while the magnitude of the impact differs across the available data series, all show an accelerated decline in oil output following the imposition of sanctions (see Figure 1).
Venezuela’s Oil Production, 2008–2020
Source: OPEC. Republished from Francisco Rodríguez, “How Sanctions Contributed to Venezuela’s Economic Collapse,” Global Americans, January 9, 2023.
The challenge of assessing the sanctions’ impact is in comparing this accelerated decline with a counterfactual in which sanctions were not imposed. Though such an estimate can never be exact, this is precisely what the above-cited studies set out to do through varying econometric methodologies, each finding a significant part of the decline in oil output to be attributable to sanctions, as would be expected by their very nature and intent.
This omission in the IMF report is particularly glaring given that it does cite a long list of purported causes for the oil collapse, including some for which there is little evidence:
the oil production drop is explained by the global oil price collapse of 2015, the severe mismanagement of the oil sector domestically, declining sectoral investment (reflected by a drop in the rig count to zero in June 2020), and a loss of human capital … Moreover, power outages were another factor, impacting oil production and economic activity in general.
While it is possible that some of the initial output drop was in response to falling prices, these began to recover in 2017, at which point other countries that had experienced a similar decline saw their output bounce back. Venezuela did not. “Declining sectoral investment,” meanwhile, is itself driven in part by sanctions, and the oil rig count remained within historical bounds until sanctions were imposed. It would be one thing to contend that sanctions are not the primary cause of the collapse; it is another entirely to exclude them from a long list of causes for which there are varying degrees of evidence.
Notably, the one study cited in the report that pertains directly to the impact of sanctions is Bahar et al. (2019). Setting aside the irregular practice of exclusively citing a non-peer-reviewed study, while omitting peer-reviewed evidence, it is important to put the Bahar et al. study in context. The paper was published in May of 2019, just four months after the imposition of oil sanctions. It makes no claim regarding the oil sanctions’ effect, but focuses purely on the effect of the 2017 financial sanctions. Even so, its claims are quite limited, as the authors argue that there are “no plausible counterfactuals or enough publicly available data to rigorously estimate a causal effect at this time.”
More than three years of oil production have been released since the publication of the Bahar et al. paper, as well as that of Hausmann and Muci (2019), an article published at roughly the same time and which made similar claims. The more recent data, which has been used in the papers cited above and is shown in Figure 1, is strongly consistent with the thesis that sanctions had a significant impact on oil production. Some of the authors of these articles have even significantly revised their views. For example, Frank Muci recently wrote, “Oil sanctions hit Venezuela hard in 2019, even if the exact size of the effect is unclear.” In other words, the single piece of research cited by the IMF on the effect of sanctions is so outdated at this time that it is largely irrelevant.
Following the discussion of Venezuela’s decline in oil output, the IMF report goes on to explore the broader economic and humanitarian impacts of the country’s crisis, including a weakening of social services, increased frequency of blackouts, increases in poverty rates, disease, and malnourishment, and a lack of access to COVID-19 vaccines. But at no point does the report mention the fact that each of these variables is impacted by sanctions. In a country that depends heavily on oil — prior to the 2017 sanctions, oil accounted for 95 percent of Venezuela’s exports — the collapse in oil output has been mirrored by a 72 percent drop in GDP per capita, which is itself tightly linked to a variety of health outcomes. At the peak of the crisis, Venezuelan poverty rates reached 93 percent. Mark Weisbrot and Jeffrey Sachs showed that roughly 40,000 Venezuelans died in 2018 alone as a result of an unusual increase in mortality and argued, “it is virtually certain that US economic sanctions made a substantial contribution to these deaths.”
It is for these reasons that US Congressman Jim McGovern (D-MA), then chair of the powerful US House Rules Committee, wrote to President Biden in May of 2021, asking him to “lift all secondary and sectoral sanctions imposed on Venezuela by the Trump Administration.” He noted that “the impact of sectoral and secondary sanctions is indiscriminate, and purposely so…. the whole point of the ‘maximum pressure’ campaign is to increase the economic cost to Venezuela of failing to comply with conditions the U.S. imposes. Economic pain is the means by which the sanctions are supposed to work.”
Beyond the general effects of economic contraction and the loss of foreign exchange with which to purchase food and medicine, US sanctions are also responsible for vaccine shipments being held up by banks hesitant to process Venezuelan transactions; degradation of the energy grid driven in part by difficulty accessing new parts and resulting in frequent electricity shortages; deterioration of public health, education, water, and other public services; and more. Ultimately, according to the UN special rapporteur on unilateral coercive measures, sanctions on Venezuela have “prevented the earning of revenues and use of resources to maintain and develop infrastructure and for social support programs, which has a devastating effect on the entire population of Venezuela, especially – but not only – those living in extreme poverty, women, children, medical workers, people with disabilities or life-threatening or chronic diseases, and the indigenous population.”
The IMF’s omission of evidence of the impact of US sanctions on Venezuela is not a minor oversight. Sanctions are an integral part of the story of Venezuela’s economic collapse and the resulting migration crisis that has seen millions of Venezuelans leave their country. Without adequate accounting for this fact, the report preemptively closes the door to considering the most impactful potential policy response: turning away from the Trump-era “maximum pressure” policy and lifting the sanctions that have contributed mightily to Venezuela’s economic challenges. Recent steps by the Biden administration in this regard have been positive, but entirely insufficient.
This lesson can be extrapolated to other heavily sanctioned countries, such as Cuba, which is currently experiencing severe economic difficulties and its largest migration crisis in decades. While the Biden administration has made “addressing the root causes” a key slogan of its policy toward migration, it has seemingly excluded sanctions from its definition of “root causes.”
The omissions in this report are so glaring as to raise serious concerns regarding the extent to which the integrity of the IMF’s technical work may have been compromised by the outsized influence of the United States on the multilateral organism. The fact that an IMF report would avoid serious engagement with the harms caused by US policy should add yet more weight to long-standing calls for the democratic reform of IMF governance.
Ultimately, “Regional Spillovers from the Venezuelan Crisis” is one relatively minor report, and much of its analysis remains valuable despite this omission. But it is emblematic of a wider, systematic, and pernicious avoidance of critical engagement with the profound human costs of US sanctions policies, both in intergovernmental organizations like the IMF, as well as in much of the media (the BBC, for example, failed to mention sanctions once in its coverage of the IMF report). In excluding this piece of the puzzle, the authors not only fail to capture the full picture of Venezuela’s economic crisis, but do a disservice to the many Venezuelans whose well-being rests on whether the US government will confront the profound suffering it has caused and do what is necessary to reverse it: putting an end to these misguided and harmful economic coercive measures.
[1] Equipo Anova’s methodology has since been critiqued by Rodríguez, but this critique does not extend to their final estimate of oil output decline.
[2] This instance is accompanied by a chart comparing Venezuela’s oil output to the introduction of sanctions in 2017 and 2019. This chart is the only time that the 2017 financial sanctions are referenced in the report and, counter to the text’s narrative, appears to be evidence for the adverse impact of financial sanctions on oil output.
On December 5, 2022, the International Monetary Fund’s (IMF) Western Hemisphere Department published a report titled “Regional Spillovers from the Venezuelan Crisis,” which assesses the causes of Venezuela’s economic crisis, the drivers of the country’s record emigration, and the impact that this influx of Venezuelan migrants has had on neighboring countries. While these are worthy topics of research, and there is much of value in the report, authors Alvarez et al. curiously omit a critical piece of the puzzle, and one of the single most important factors contributing to Venezuela’s current economic and humanitarian plight: US economic sanctions.
In August 2017, the Trump administration issued Executive Order 13808, barring the government of Venezuela, including the state-owned oil company Petróleos de Venezuela, S.A. (PDVSA) and its joint ventures, from accessing US financial markets. Though the United States had imposed sanctions on certain Venezuelan individuals and entities before this, including under the Obama administration’s E.O. 13692, which declared a US national emergency with respect to Venezuela, the August 2017 sanctions marked the beginning of a series of sweeping sanctions that would define the Trump administration’s approach to US-Venezuelan relations. Sanctions were escalated even further alongside the recognition of a parallel government beginning in 2019, most notably with the January 28 designation of PDVSA as a sanctioned entity, and the 2020 imposition of secondary sanctions against shipping companies involved in the transportation of Venezuelan oil. The vast majority of these sanctions remain in place today.
The impact of these sanctions has been swift and disastrous, particularly on Venezuela’s oil output, which is the country’s primary source of foreign revenue. While there are no doubt multiple factors that have contributed to Venezuela’s precipitous drop in oil production — from 2.4 million barrels per day (bpd) prior to the crisis, to a low of 0.4 million bpd in mid-2020 — a preponderance of evidence points to US sanctions as a significant driver of the decline. A 2022 analysis by one of the authors of this post, Francisco Rodríguez, attributes the loss of 797,000 bpd to the 2017 financial sanctions. Other studies using different methodologies have set this figure at 698,000 bpd (by Equipo Anova[1]), and from 616,000 to 1,023,000 bpd (by Luis Oliveros). And a recent paper by Rodríguez in the Latin American Economic Review uses variation in production across the country’s Orinoco basin to estimate the impact of sanctions at 255,000 to 637,000 bpd.
Despite these well-documented impacts, US sanctions on Venezuela are mentioned in the body of the 61-page IMF report only twice. Once, the report does appear to imply some adverse effects of sanctions, albeit while suggesting that they have been mitigated: “Venezuela has been able to place its heavy crude oil in the Asian market at a substantial price discount, alleviating in part the impact of sanctions.” However, the other mention of sanctions specifically downplays their impact: “The sharp decline [in oil output], which preceded the introduction of oil sanctions by the United States in January 2019, reflected both internal and external factors.”[2]
This latter assertion is an oft-repeated claim, one that is misleading at best. While it is true that the start of the decline in oil production, and indeed the beginning of the economic crisis itself, preceded the 2017 financial sanctions, this is hardly evidence that the multiple waves of sanctions have not had a significant causal effect. In fact, while the magnitude of the impact differs across the available data series, all show an accelerated decline in oil output following the imposition of sanctions (see Figure 1).
Venezuela’s Oil Production, 2008–2020
Source: OPEC. Republished from Francisco Rodríguez, “How Sanctions Contributed to Venezuela’s Economic Collapse,” Global Americans, January 9, 2023.
The challenge of assessing the sanctions’ impact is in comparing this accelerated decline with a counterfactual in which sanctions were not imposed. Though such an estimate can never be exact, this is precisely what the above-cited studies set out to do through varying econometric methodologies, each finding a significant part of the decline in oil output to be attributable to sanctions, as would be expected by their very nature and intent.
This omission in the IMF report is particularly glaring given that it does cite a long list of purported causes for the oil collapse, including some for which there is little evidence:
the oil production drop is explained by the global oil price collapse of 2015, the severe mismanagement of the oil sector domestically, declining sectoral investment (reflected by a drop in the rig count to zero in June 2020), and a loss of human capital … Moreover, power outages were another factor, impacting oil production and economic activity in general.
While it is possible that some of the initial output drop was in response to falling prices, these began to recover in 2017, at which point other countries that had experienced a similar decline saw their output bounce back. Venezuela did not. “Declining sectoral investment,” meanwhile, is itself driven in part by sanctions, and the oil rig count remained within historical bounds until sanctions were imposed. It would be one thing to contend that sanctions are not the primary cause of the collapse; it is another entirely to exclude them from a long list of causes for which there are varying degrees of evidence.
Notably, the one study cited in the report that pertains directly to the impact of sanctions is Bahar et al. (2019). Setting aside the irregular practice of exclusively citing a non-peer-reviewed study, while omitting peer-reviewed evidence, it is important to put the Bahar et al. study in context. The paper was published in May of 2019, just four months after the imposition of oil sanctions. It makes no claim regarding the oil sanctions’ effect, but focuses purely on the effect of the 2017 financial sanctions. Even so, its claims are quite limited, as the authors argue that there are “no plausible counterfactuals or enough publicly available data to rigorously estimate a causal effect at this time.”
More than three years of oil production have been released since the publication of the Bahar et al. paper, as well as that of Hausmann and Muci (2019), an article published at roughly the same time and which made similar claims. The more recent data, which has been used in the papers cited above and is shown in Figure 1, is strongly consistent with the thesis that sanctions had a significant impact on oil production. Some of the authors of these articles have even significantly revised their views. For example, Frank Muci recently wrote, “Oil sanctions hit Venezuela hard in 2019, even if the exact size of the effect is unclear.” In other words, the single piece of research cited by the IMF on the effect of sanctions is so outdated at this time that it is largely irrelevant.
Following the discussion of Venezuela’s decline in oil output, the IMF report goes on to explore the broader economic and humanitarian impacts of the country’s crisis, including a weakening of social services, increased frequency of blackouts, increases in poverty rates, disease, and malnourishment, and a lack of access to COVID-19 vaccines. But at no point does the report mention the fact that each of these variables is impacted by sanctions. In a country that depends heavily on oil — prior to the 2017 sanctions, oil accounted for 95 percent of Venezuela’s exports — the collapse in oil output has been mirrored by a 72 percent drop in GDP per capita, which is itself tightly linked to a variety of health outcomes. At the peak of the crisis, Venezuelan poverty rates reached 93 percent. Mark Weisbrot and Jeffrey Sachs showed that roughly 40,000 Venezuelans died in 2018 alone as a result of an unusual increase in mortality and argued, “it is virtually certain that US economic sanctions made a substantial contribution to these deaths.”
It is for these reasons that US Congressman Jim McGovern (D-MA), then chair of the powerful US House Rules Committee, wrote to President Biden in May of 2021, asking him to “lift all secondary and sectoral sanctions imposed on Venezuela by the Trump Administration.” He noted that “the impact of sectoral and secondary sanctions is indiscriminate, and purposely so…. the whole point of the ‘maximum pressure’ campaign is to increase the economic cost to Venezuela of failing to comply with conditions the U.S. imposes. Economic pain is the means by which the sanctions are supposed to work.”
Beyond the general effects of economic contraction and the loss of foreign exchange with which to purchase food and medicine, US sanctions are also responsible for vaccine shipments being held up by banks hesitant to process Venezuelan transactions; degradation of the energy grid driven in part by difficulty accessing new parts and resulting in frequent electricity shortages; deterioration of public health, education, water, and other public services; and more. Ultimately, according to the UN special rapporteur on unilateral coercive measures, sanctions on Venezuela have “prevented the earning of revenues and use of resources to maintain and develop infrastructure and for social support programs, which has a devastating effect on the entire population of Venezuela, especially – but not only – those living in extreme poverty, women, children, medical workers, people with disabilities or life-threatening or chronic diseases, and the indigenous population.”
The IMF’s omission of evidence of the impact of US sanctions on Venezuela is not a minor oversight. Sanctions are an integral part of the story of Venezuela’s economic collapse and the resulting migration crisis that has seen millions of Venezuelans leave their country. Without adequate accounting for this fact, the report preemptively closes the door to considering the most impactful potential policy response: turning away from the Trump-era “maximum pressure” policy and lifting the sanctions that have contributed mightily to Venezuela’s economic challenges. Recent steps by the Biden administration in this regard have been positive, but entirely insufficient.
This lesson can be extrapolated to other heavily sanctioned countries, such as Cuba, which is currently experiencing severe economic difficulties and its largest migration crisis in decades. While the Biden administration has made “addressing the root causes” a key slogan of its policy toward migration, it has seemingly excluded sanctions from its definition of “root causes.”
The omissions in this report are so glaring as to raise serious concerns regarding the extent to which the integrity of the IMF’s technical work may have been compromised by the outsized influence of the United States on the multilateral organism. The fact that an IMF report would avoid serious engagement with the harms caused by US policy should add yet more weight to long-standing calls for the democratic reform of IMF governance.
Ultimately, “Regional Spillovers from the Venezuelan Crisis” is one relatively minor report, and much of its analysis remains valuable despite this omission. But it is emblematic of a wider, systematic, and pernicious avoidance of critical engagement with the profound human costs of US sanctions policies, both in intergovernmental organizations like the IMF, as well as in much of the media (the BBC, for example, failed to mention sanctions once in its coverage of the IMF report). In excluding this piece of the puzzle, the authors not only fail to capture the full picture of Venezuela’s economic crisis, but do a disservice to the many Venezuelans whose well-being rests on whether the US government will confront the profound suffering it has caused and do what is necessary to reverse it: putting an end to these misguided and harmful economic coercive measures.
[1] Equipo Anova’s methodology has since been critiqued by Rodríguez, but this critique does not extend to their final estimate of oil output decline.
[2] This instance is accompanied by a chart comparing Venezuela’s oil output to the introduction of sanctions in 2017 and 2019. This chart is the only time that the 2017 financial sanctions are referenced in the report and, counter to the text’s narrative, appears to be evidence for the adverse impact of financial sanctions on oil output.
• BoliviaBoliviaLatin America and the CaribbeanAmérica Latina y el Caribe
This is the twelfth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, part eight, part nine,part ten, and part eleven.
In the previous post , we looked at the difference-in-difference models of Escobari and Hoover. We noted that they all show the same thing, suffering from the same problem. To identify fraud, they require that however much more the results at the late-transmitting polling stations favored Morales when contrasted with the early ones, they ought to do so by no more than the corresponding increase in the Sí vote in 2016. This is the “parallel trends” assumption. Unfortunately, even among the polling stations included in the TSE announcement, the average gap between 2016 and 2019 grows with the lateness of the transmission of results. Because the late polling stations (those excluded from the announcement) were very disproportionately late in their transmissions, we expect that the problem lies in the baseline assumption of parallel trends. Neither the addition of geographic controls nor the inclusion of a common trend improves the model in any important way. Rather, the consequence is that what Escobari and Hoover interpret as fraud in the election is almost entirely a measure of the degree to which the parallel trends assumption is wrong.
Escobari and Hoover do allow for this possibility. In their next models, they account for the linear difference in trends among the polling stations included in the announcement. Incorporating this into the model, we get Figure 1 .
Figure 1
Difference-in-difference Model Allowing Different Slopes in Each Year
Sources: TSE, OEP, and author’s calculations.
For convenience, we estimate the difference-in-difference by turning the computation inside-out. In the previous post, we contrasted the difference across elections for the late-counted votes with the difference across elections for the early-counted votes. However, these differences vary with ARRIVAL. In Figure 1, we contrast the difference across late- and early-counted votes in 2019 with the difference across late- and early-counted votes in 2016. Arithmetically, this double-difference is exactly the same, but the individual differences are constant within each year.
As always, inclusion of geographic effects has no relevant impact on the estimation. We can see that in our replication, with correct valid votes and weights, that we find a much smaller double-difference: about 0.4 percentage points, compared to Escobari and Hoover’s 1 percentage point. Our estimate is not statistically significant. More importantly, none of these estimates are of political significance. Even Escobari and Hoover’s reported result leaves only 0.17 percentage points of Morales’s final margin unaccounted for.
Table 1
| As Published | Replication | Correct Voters | Weighted | |||
|---|---|---|---|---|---|---|
| (1) | (2) | (3) | (4) | (5) | (6) | |
| Variable | ||||||
| SHUTDOWN x Y2019 | 1.064 (0.390) | 0.6970 (0.409) | 0.6691 (0.408) | 0.4348 (0.277) | 0.4160 (0.288) | 0.4151 (0.391) |
| ARRIVAL x Y2019 | 5.722 (1.067) | 5.865 (0.518) | 5.876 (0.518) | 7.246 (0.345) | 7.373 (0.357) | 7.376 (0.486) |
| ARRIVAL | 21.63 (0.914) | -2.781 (0.176) | ||||
| Y2019 | * | 8.786 (0.273) | 8.792 (0.273) | 7.721 (0.180) | 7.636 (0.186) | 7.635 (0.252) |
| SHUTDOWN | 6.137 (0.691) | -0.2592 (0.128) | ||||
| Constant | * | 0.6149 (0.062) | 0.6205 (0.062) | -13.14 (0.483) | 0.5200 (0.090) | -0.9599 (0.060) |
| Fixed Effects | ||||||
| Precinct | Yes | |||||
| Station | Yes | Yes | Yes | Yes | ||
| Observations | 65,811 | 69,064 | 69,086 | 69,082 | 69,082 | 69,082 |
| R2 | 0.966 | 0.966 | 0.966 | 0.056 | 0.956 | 0.967 |
* Not reported
Sources: Escobari and Hoover, TSE, OEP, and author’s calculations.
At this point, the natural experiment is basically over. An overabundance of evidence has made it clear that parallel trends are not applicable. Even including a partial fix by allowing year-specific trends over ARRIVAL, the model successfully explains Morales’s first-round victory. Unfortunately, Escobari and Hoover then radically reframe their own experiment to argue in favor of a politically significant 2.86 percentage points of fraud in the election. They do so by shifting the focus from the double-difference to the difference in slopes, as in the right panel of Figure 2.
Figure 2
Escobari and Hoover Offer an Entirely New Interpretation of the Difference-in-Difference
Sources: TSE, OEP, and author’s calculations.
There is an obvious problem with this ex post reinterpretation of the results. Specifically, if there is fraud throughout the entire count in 2019, then we have no baseline from which to work. The TSE announcement no longer offers any opportunity to separate “untreated” (assumed fraud-free) margins from contrasting “treated” margins.
To get around this, Escobari and Hoover simply assume that only the very earliest polling stations to report are free of fraud, and reimpose the invalid parallel trends assumption. These assumptions are simply heroic.
Imagine simply that Morales lost support (relative to the referendum) outside the most rural polling stations; then we might see something like Figure 3 . The gap between the opposition to the referendum (green) and support for Morales (orange) reflects Morales’s overall loss of support. Because the rural stations where Morales better held support tended to arrive late, the gap shrinks as ARRIVAL increases.
Figure 3
Different Slopes May Result from Benign Causes
Source: author’s calculations.
According to Escobari and Hoover, this cannot be an explanation for the shrinking gap. Under their assumptions, Morales could only have maintained support in rural precincts through fraud. That is, the election looks something like the left of Figure 4 , where the shaded orange area represents fraudulent votes counted for Morales in the official data. Once the shaded area is removed, the orange and green trends are made parallel.
Figure 4
Even if Trends Should Be Made Parallel, This Doesn’t Mean Fraud Favored Morales
Source: author’s calculations.
However, there is an equally plausible alternative explanation: that Morales lost support in the capital cities because the opposition stole votes at these generally early polling stations where they controlled the juries. That is, we could see something like the right of Figure 4. Only after adding in the missing support for Morales are parallel trends restored. Escobari and Hoover simply assume that the left is true and the right is false.
Of course, fraud is not even necessary to explain the results at all. It could simply be true that Morales successfully maintained support in rural areas while losing ground in urban areas.
Likewise, Escobari and Hoover simply assume away the possibility that among voters opposed to the referendum Mesa’s support was disproportionately urban and in the capital cities in particular. Though we observe this in the data, Escobari and Hoover’s preferred explanation is on the left of Figure 5 : that other opposing candidates stole support for Mesa among the late arrivals. On the right, we see an alternative explanation that Mesa stole votes from other opposition candidates early.
Figure 5
The Ambiguity Persists with Different Benign Causes of the Difference in Trends
Source: author’s calculations.
Again, the only difference between the two is a presumption that the earliest results — and only the earliest results — are free of fraud.
We emphasize at this point that these two possible explanations for nonparallel trends are neither complete nor exhaustive. They are simply illustrations of benign factors that Escobari and Hoover assume away.
If we accept both of Escobari and Hoover’s incredibly strong assumptions (early arrivals being free of fraud and strict parallel trends), then there isn’t really any point in counting past the first few polling stations. Once we have the difference in margins early, we can simply compute the “fraud-free” result for the entire remainder of the election based on the 2016 results. If we complete the count, we will either find the “correct” result, or we declare fraud.
Hopefully, it is obvious that this is absurd. Escobari and Hoover’s assumptions are far too restrictive, disallowing all kinds of benign explanations of the election results. Simply put, their reinterpretation of the year-specific trends as indicating fraud is not credible.
Before we conclude this series, we will mop up the last of Escobari and Hoover’s models.
This is the twelfth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, part eight, part nine,part ten, and part eleven.
In the previous post , we looked at the difference-in-difference models of Escobari and Hoover. We noted that they all show the same thing, suffering from the same problem. To identify fraud, they require that however much more the results at the late-transmitting polling stations favored Morales when contrasted with the early ones, they ought to do so by no more than the corresponding increase in the Sí vote in 2016. This is the “parallel trends” assumption. Unfortunately, even among the polling stations included in the TSE announcement, the average gap between 2016 and 2019 grows with the lateness of the transmission of results. Because the late polling stations (those excluded from the announcement) were very disproportionately late in their transmissions, we expect that the problem lies in the baseline assumption of parallel trends. Neither the addition of geographic controls nor the inclusion of a common trend improves the model in any important way. Rather, the consequence is that what Escobari and Hoover interpret as fraud in the election is almost entirely a measure of the degree to which the parallel trends assumption is wrong.
Escobari and Hoover do allow for this possibility. In their next models, they account for the linear difference in trends among the polling stations included in the announcement. Incorporating this into the model, we get Figure 1 .
Figure 1
Difference-in-difference Model Allowing Different Slopes in Each Year
Sources: TSE, OEP, and author’s calculations.
For convenience, we estimate the difference-in-difference by turning the computation inside-out. In the previous post, we contrasted the difference across elections for the late-counted votes with the difference across elections for the early-counted votes. However, these differences vary with ARRIVAL. In Figure 1, we contrast the difference across late- and early-counted votes in 2019 with the difference across late- and early-counted votes in 2016. Arithmetically, this double-difference is exactly the same, but the individual differences are constant within each year.
As always, inclusion of geographic effects has no relevant impact on the estimation. We can see that in our replication, with correct valid votes and weights, that we find a much smaller double-difference: about 0.4 percentage points, compared to Escobari and Hoover’s 1 percentage point. Our estimate is not statistically significant. More importantly, none of these estimates are of political significance. Even Escobari and Hoover’s reported result leaves only 0.17 percentage points of Morales’s final margin unaccounted for.
Table 1
| As Published | Replication | Correct Voters | Weighted | |||
|---|---|---|---|---|---|---|
| (1) | (2) | (3) | (4) | (5) | (6) | |
| Variable | ||||||
| SHUTDOWN x Y2019 | 1.064 (0.390) | 0.6970 (0.409) | 0.6691 (0.408) | 0.4348 (0.277) | 0.4160 (0.288) | 0.4151 (0.391) |
| ARRIVAL x Y2019 | 5.722 (1.067) | 5.865 (0.518) | 5.876 (0.518) | 7.246 (0.345) | 7.373 (0.357) | 7.376 (0.486) |
| ARRIVAL | 21.63 (0.914) | -2.781 (0.176) | ||||
| Y2019 | * | 8.786 (0.273) | 8.792 (0.273) | 7.721 (0.180) | 7.636 (0.186) | 7.635 (0.252) |
| SHUTDOWN | 6.137 (0.691) | -0.2592 (0.128) | ||||
| Constant | * | 0.6149 (0.062) | 0.6205 (0.062) | -13.14 (0.483) | 0.5200 (0.090) | -0.9599 (0.060) |
| Fixed Effects | ||||||
| Precinct | Yes | |||||
| Station | Yes | Yes | Yes | Yes | ||
| Observations | 65,811 | 69,064 | 69,086 | 69,082 | 69,082 | 69,082 |
| R2 | 0.966 | 0.966 | 0.966 | 0.056 | 0.956 | 0.967 |
* Not reported
Sources: Escobari and Hoover, TSE, OEP, and author’s calculations.
At this point, the natural experiment is basically over. An overabundance of evidence has made it clear that parallel trends are not applicable. Even including a partial fix by allowing year-specific trends over ARRIVAL, the model successfully explains Morales’s first-round victory. Unfortunately, Escobari and Hoover then radically reframe their own experiment to argue in favor of a politically significant 2.86 percentage points of fraud in the election. They do so by shifting the focus from the double-difference to the difference in slopes, as in the right panel of Figure 2.
Figure 2
Escobari and Hoover Offer an Entirely New Interpretation of the Difference-in-Difference
Sources: TSE, OEP, and author’s calculations.
There is an obvious problem with this ex post reinterpretation of the results. Specifically, if there is fraud throughout the entire count in 2019, then we have no baseline from which to work. The TSE announcement no longer offers any opportunity to separate “untreated” (assumed fraud-free) margins from contrasting “treated” margins.
To get around this, Escobari and Hoover simply assume that only the very earliest polling stations to report are free of fraud, and reimpose the invalid parallel trends assumption. These assumptions are simply heroic.
Imagine simply that Morales lost support (relative to the referendum) outside the most rural polling stations; then we might see something like Figure 3 . The gap between the opposition to the referendum (green) and support for Morales (orange) reflects Morales’s overall loss of support. Because the rural stations where Morales better held support tended to arrive late, the gap shrinks as ARRIVAL increases.
Figure 3
Different Slopes May Result from Benign Causes
Source: author’s calculations.
According to Escobari and Hoover, this cannot be an explanation for the shrinking gap. Under their assumptions, Morales could only have maintained support in rural precincts through fraud. That is, the election looks something like the left of Figure 4 , where the shaded orange area represents fraudulent votes counted for Morales in the official data. Once the shaded area is removed, the orange and green trends are made parallel.
Figure 4
Even if Trends Should Be Made Parallel, This Doesn’t Mean Fraud Favored Morales
Source: author’s calculations.
However, there is an equally plausible alternative explanation: that Morales lost support in the capital cities because the opposition stole votes at these generally early polling stations where they controlled the juries. That is, we could see something like the right of Figure 4. Only after adding in the missing support for Morales are parallel trends restored. Escobari and Hoover simply assume that the left is true and the right is false.
Of course, fraud is not even necessary to explain the results at all. It could simply be true that Morales successfully maintained support in rural areas while losing ground in urban areas.
Likewise, Escobari and Hoover simply assume away the possibility that among voters opposed to the referendum Mesa’s support was disproportionately urban and in the capital cities in particular. Though we observe this in the data, Escobari and Hoover’s preferred explanation is on the left of Figure 5 : that other opposing candidates stole support for Mesa among the late arrivals. On the right, we see an alternative explanation that Mesa stole votes from other opposition candidates early.
Figure 5
The Ambiguity Persists with Different Benign Causes of the Difference in Trends
Source: author’s calculations.
Again, the only difference between the two is a presumption that the earliest results — and only the earliest results — are free of fraud.
We emphasize at this point that these two possible explanations for nonparallel trends are neither complete nor exhaustive. They are simply illustrations of benign factors that Escobari and Hoover assume away.
If we accept both of Escobari and Hoover’s incredibly strong assumptions (early arrivals being free of fraud and strict parallel trends), then there isn’t really any point in counting past the first few polling stations. Once we have the difference in margins early, we can simply compute the “fraud-free” result for the entire remainder of the election based on the 2016 results. If we complete the count, we will either find the “correct” result, or we declare fraud.
Hopefully, it is obvious that this is absurd. Escobari and Hoover’s assumptions are far too restrictive, disallowing all kinds of benign explanations of the election results. Simply put, their reinterpretation of the year-specific trends as indicating fraud is not credible.
Before we conclude this series, we will mop up the last of Escobari and Hoover’s models.
• BoliviaBoliviaLatin America and the CaribbeanAmérica Latina y el Caribe
This is the eleventh in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, part eight, part nine, and part ten.
Earlier, we observed that even if geography is stable from one election to another, there is no guarantee that the effect of geography on vote shares is consistent over time. There are completely ordinary and utterly plausible explanations for why election results would be more or less sensitive to geography from one election to another. This may result in a widening gap between observed margins over the progress of a count even absent any fraud
In the last post, we saw the effect this had on Escobari and Hoover’s “difference-in-difference” models. Specifically, the absence of fraud does not guarantee that the identifying “parallel trends” assumption holds. This causes Escobari and Hoover’s approach to mistakenly find fraud where none exists.
Unfortunately, we also observed that we have reason to believe that the parallel trends assumption does not hold in the official data, either. For example, Morales lost support relative to 2016 outside rural areas of Bolivia, and rural areas tended to be counted late. Thus, his loss shrank as rurality of the vote increased. Mesa’s support among opposition voters was disproportionately concentrated in the capital cities. These tended to report early, meaning Morales’s margin further increased over the count. As a consequence, there was an observable, preexisting widening of the differences between 2016 and 2019 over the count at the time of the TSE announcement. There is every reason to expect that the SHUTDOWN polling stations would on average show a bigger increase in Morales’s margin than the stations included in the announcement.
In Figure 1, we see the application of the difference-in-difference models to the official results.
Figure 1
Difference-in-Difference Model Applied to Official and Adjusted Results
Sources: TSE, OEP, and author’s calculations.
Whether or not we adjust the data by precinct, there is a double-difference of 3 percentage points. However, it seems clear that the trends are not parallel. It is particularly clear in the adjusted data on the right that among polling stations included in the TSE announcement, later reporters have a wider gap between 2016 and 2019 than those polling stations reporting earlier. It is clearly wrong to simply assume that the post-announcement stations, which are largely those that reported even later, should not show a gap at least as wide.
Table 1
Application of Escobari and Hoover’s “Difference-in-Difference Estimates” to Actual Data
|
As Published |
Replication |
|||||
|
(1) |
(2) |
(3) |
(4) |
(5) |
(6) |
|
|
Variable |
||||||
|
SHUTDOWN x Y2019 |
2.964 (0.334) |
2.699 (0.335) |
2.842 (0.459) |
2.996 (0.251) |
3.018 (0.261) |
3.018 (0.355) |
|
SHUTDOWN |
13.30 (0.610) |
-1.090 (0.212) |
13.77 (0.624) |
-1.37 (0.122) |
||
|
Y2019 |
11.99 (0.635) |
11.15 (0.443) |
11.13 (0.612) |
11.06 (0.091) |
11.03 (0.095) |
11.03 (0.129) |
|
Constant |
-2.157 (3.380) |
0.616 (0.298) |
0.438 (0.290) |
-3.173 (0.238) |
-0.742 (0.038) |
-0.960 (0.060) |
|
Fixed Effects |
||||||
|
Precinct |
Yes |
Yes |
||||
|
Station |
Yes |
Yes |
||||
|
Observations |
66,535 |
66,535 |
66,535 |
69,082 |
69,082 |
69,082 |
|
R2 |
0.035 |
0.934 |
0.965 |
0.035 |
0.956 |
0.967 |
Sources: TSE, OEP, and author’s calculations.
Escobari and Hoover next expand their model to include ARRIVAL as a variable — in effect allowing for a trend common to both 2016 and 2019. Because the double-difference is driven by a widening difference between elections, this has no effect on the double-difference. We compare the two unadjusted difference-in-difference models in Figure 2. On the right, we see that the 2016 margins rise more slowly than the common trend, while the 2019 margins rise more quickly.
Figure 2
Allowing an Election-Wide Trend Does Not Change the Double-Difference
In Table 2, we see the model results including the common trend. In columns 1 and 4, we include the results without trend as reference. Column 5 shows the application with no geographic adjustment. Columns 6 and 7 are replications of the results with geographic effects seen in columns 2 and 3. Taken at face value, and accepting the interpretation of the result as fraud, these effects are not large enough to suggest a reversal of Morales’s first-round victory: 3 percentage points applied to 16 percent of the election comes to less than half of 1 percentage point. However, the nonparallel trends suggest that these estimates are in any case inflated.
Table 2
Application of Escobari and Hoover’s Common-Trend Difference-in-Difference Estimates
|
As Published |
Replication |
||||||
|
(1) |
(2) |
(3) |
(4) |
(5) |
(6) |
(7) |
|
|
Variable |
|||||||
|
SHUTDOWN x Y2019 |
2.964 (0.334) |
2.954 (0.433) |
-2.761 (0.967) |
2.996 (0.251) |
2.992 (0.251) |
3.018 (0.261) |
-3.027 (1.109) |
|
SHUTDOWN |
13.30 (0.610) |
-1.577 (0.262) |
13.77 (0.624) |
4.858 (0.696) |
-1.560 (0.126) |
||
|
Y2019 |
11.99 (0.635) |
11.15 (0.443) |
11.14 (0.611) |
11.06 (0.091) |
11.06 (0.091) |
11.03 (0.095) |
11.03 (0.129) |
|
ARRIVAL |
1.340 (0.242) |
25.26 (0.939) |
0.906 (0.160) |
||||
|
ARRIVAL x SHUTDOWN x Y2019 |
7.277 (1.588) |
7.429 (1.315) |
|||||
|
Constant |
-2.157 (3.380) |
-0.831 (0.311) |
0.198 (0.280) |
-3.173 (0.238) |
-14.81 (0.493) |
-1.179 (0.086) |
-0.960 (0.060) |
|
Fixed Effects |
|||||||
|
Precinct |
Yes |
Yes |
|||||
|
Station |
Yes |
Yes |
|||||
|
Observations |
66,535 |
65,811 |
65,811 |
69,082 |
69,082 |
69,082 |
69,082 |
|
R2 |
0.035 |
0.937 |
0.966 |
0.035 |
0.055 |
0.956 |
0.967 |
Eventually, Escobari and Hoover acknowledge that the trends in the early arrivals differ across years. We explore these models in the next post.
This is the eleventh in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, part eight, part nine, and part ten.
Earlier, we observed that even if geography is stable from one election to another, there is no guarantee that the effect of geography on vote shares is consistent over time. There are completely ordinary and utterly plausible explanations for why election results would be more or less sensitive to geography from one election to another. This may result in a widening gap between observed margins over the progress of a count even absent any fraud
In the last post, we saw the effect this had on Escobari and Hoover’s “difference-in-difference” models. Specifically, the absence of fraud does not guarantee that the identifying “parallel trends” assumption holds. This causes Escobari and Hoover’s approach to mistakenly find fraud where none exists.
Unfortunately, we also observed that we have reason to believe that the parallel trends assumption does not hold in the official data, either. For example, Morales lost support relative to 2016 outside rural areas of Bolivia, and rural areas tended to be counted late. Thus, his loss shrank as rurality of the vote increased. Mesa’s support among opposition voters was disproportionately concentrated in the capital cities. These tended to report early, meaning Morales’s margin further increased over the count. As a consequence, there was an observable, preexisting widening of the differences between 2016 and 2019 over the count at the time of the TSE announcement. There is every reason to expect that the SHUTDOWN polling stations would on average show a bigger increase in Morales’s margin than the stations included in the announcement.
In Figure 1, we see the application of the difference-in-difference models to the official results.
Figure 1
Difference-in-Difference Model Applied to Official and Adjusted Results
Sources: TSE, OEP, and author’s calculations.
Whether or not we adjust the data by precinct, there is a double-difference of 3 percentage points. However, it seems clear that the trends are not parallel. It is particularly clear in the adjusted data on the right that among polling stations included in the TSE announcement, later reporters have a wider gap between 2016 and 2019 than those polling stations reporting earlier. It is clearly wrong to simply assume that the post-announcement stations, which are largely those that reported even later, should not show a gap at least as wide.
Table 1
Application of Escobari and Hoover’s “Difference-in-Difference Estimates” to Actual Data
|
As Published |
Replication |
|||||
|
(1) |
(2) |
(3) |
(4) |
(5) |
(6) |
|
|
Variable |
||||||
|
SHUTDOWN x Y2019 |
2.964 (0.334) |
2.699 (0.335) |
2.842 (0.459) |
2.996 (0.251) |
3.018 (0.261) |
3.018 (0.355) |
|
SHUTDOWN |
13.30 (0.610) |
-1.090 (0.212) |
13.77 (0.624) |
-1.37 (0.122) |
||
|
Y2019 |
11.99 (0.635) |
11.15 (0.443) |
11.13 (0.612) |
11.06 (0.091) |
11.03 (0.095) |
11.03 (0.129) |
|
Constant |
-2.157 (3.380) |
0.616 (0.298) |
0.438 (0.290) |
-3.173 (0.238) |
-0.742 (0.038) |
-0.960 (0.060) |
|
Fixed Effects |
||||||
|
Precinct |
Yes |
Yes |
||||
|
Station |
Yes |
Yes |
||||
|
Observations |
66,535 |
66,535 |
66,535 |
69,082 |
69,082 |
69,082 |
|
R2 |
0.035 |
0.934 |
0.965 |
0.035 |
0.956 |
0.967 |
Sources: TSE, OEP, and author’s calculations.
Escobari and Hoover next expand their model to include ARRIVAL as a variable — in effect allowing for a trend common to both 2016 and 2019. Because the double-difference is driven by a widening difference between elections, this has no effect on the double-difference. We compare the two unadjusted difference-in-difference models in Figure 2. On the right, we see that the 2016 margins rise more slowly than the common trend, while the 2019 margins rise more quickly.
Figure 2
Allowing an Election-Wide Trend Does Not Change the Double-Difference
In Table 2, we see the model results including the common trend. In columns 1 and 4, we include the results without trend as reference. Column 5 shows the application with no geographic adjustment. Columns 6 and 7 are replications of the results with geographic effects seen in columns 2 and 3. Taken at face value, and accepting the interpretation of the result as fraud, these effects are not large enough to suggest a reversal of Morales’s first-round victory: 3 percentage points applied to 16 percent of the election comes to less than half of 1 percentage point. However, the nonparallel trends suggest that these estimates are in any case inflated.
Table 2
Application of Escobari and Hoover’s Common-Trend Difference-in-Difference Estimates
|
As Published |
Replication |
||||||
|
(1) |
(2) |
(3) |
(4) |
(5) |
(6) |
(7) |
|
|
Variable |
|||||||
|
SHUTDOWN x Y2019 |
2.964 (0.334) |
2.954 (0.433) |
-2.761 (0.967) |
2.996 (0.251) |
2.992 (0.251) |
3.018 (0.261) |
-3.027 (1.109) |
|
SHUTDOWN |
13.30 (0.610) |
-1.577 (0.262) |
13.77 (0.624) |
4.858 (0.696) |
-1.560 (0.126) |
||
|
Y2019 |
11.99 (0.635) |
11.15 (0.443) |
11.14 (0.611) |
11.06 (0.091) |
11.06 (0.091) |
11.03 (0.095) |
11.03 (0.129) |
|
ARRIVAL |
1.340 (0.242) |
25.26 (0.939) |
0.906 (0.160) |
||||
|
ARRIVAL x SHUTDOWN x Y2019 |
7.277 (1.588) |
7.429 (1.315) |
|||||
|
Constant |
-2.157 (3.380) |
-0.831 (0.311) |
0.198 (0.280) |
-3.173 (0.238) |
-14.81 (0.493) |
-1.179 (0.086) |
-0.960 (0.060) |
|
Fixed Effects |
|||||||
|
Precinct |
Yes |
Yes |
|||||
|
Station |
Yes |
Yes |
|||||
|
Observations |
66,535 |
65,811 |
65,811 |
69,082 |
69,082 |
69,082 |
69,082 |
|
R2 |
0.035 |
0.937 |
0.966 |
0.035 |
0.055 |
0.956 |
0.967 |
Eventually, Escobari and Hoover acknowledge that the trends in the early arrivals differ across years. We explore these models in the next post.
• BoliviaBoliviaElectionsEleccionesLatin America and the CaribbeanAmérica Latina y el CaribeWorldEl Mundo
This is the tenth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, part eight, and part nine.
In the previous post, we observed that even if geography is stable from one election to another, there is no guarantee that the effect of geography on vote shares is consistent over time. This may result in a widening gap between observed margins over the progress of a count, even absent any fraud. We also observed that the results at the polling stations included in the TSE announcement exhibited such a trend. We asserted that this poses a problem for Escobari and Hoover. In this post, we demonstrate how their “difference-in-difference” models mistakenly identify fraud when the difference in trends interacts with the counting bias — even when the resulting trends are linear.
One way of demonstrating this is to apply Escobari and Hoover to synthetic election data where we have total control of the amount of fraud. We may create a “geography” variable. We will not be able to observe this variable directly in our analysis, but it will be constant across precincts and will correlate with ARRIVAL. We may then generate election results based on the hidden geography alone, with no consideration to ARRIVAL. For clarity of illustration, we will then break out the SHUTDOWN group as polling stations in the last sixth of ARRIVAL.
In Figure 1, we see two different synthetic election results. Both the results on the left and the results on the right have the same overall trend over the order in which polling stations transmitted. In each, we have marked out the SHUTDOWN polling stations in green. There is no within-precinct-trend in either the left or right, nor does SHUTDOWN have any impact on the margin. The only difference between left and right is how much geography explains the differences across precincts.
Figure 1
Two Examples of Synthetic Election Data
Source: author’s calculations.
On the right, geography explains quite a bit and therefore the overall trend is clearer because the order of transmission is an imperfect proxy for geography.
We may produce difference estimates on this data just as we had on the actual election results of 2019. In both the left and the right, Escobari and Hoover’s difference estimates identify fraud where none exists — unless we adjust at the precinct level.
Table 1
| (1) | (2) | (3) | (4) | |
| SHUTDOWN | 15.46 (0.706) | 0.233 (0.110) | 16.26 (0.141) | 0.132 (0.706) |
| Constant | 7.780 (0.298) | 10.38 (0.037) | 7.370 (0.076) | 10.11 (0.044) |
| Precinct | No | Yes | No | Yes |
| 35,000 | 35,000 | 35,000 | 35,000 | |
| 0.0136 | 0.988 | 0.199 | 0.773 |
Source: author’s calculations.
In generating the synthetic election results, we have not even identified precincts as counted entirely late, so we know there is no late precinct-level fraud. However, that should not stop up from adding in synthetic 2016 data. As we see in Figure 2, the synthetic 2019 results are more sensitive to geography than the 2016 results.
Figure 2
Example of Synthetic Data Covering Multiple Elections
Source: author’s calculations.
So what do Escobari and Hoover do with this added information? In the simple difference models, they simply compared the average margin among those polling stations included in the TSE announcement to those that were excluded. They assume that absent fraud there is no difference between the two. In the “difference-in-difference” models, the baseline for fraud in 2019 is not zero, but whatever difference is observed in 2016. Graphically, we may see Escobari and Hoover’s baseline difference-in-difference model applied to the synthetic data in Figure 3. The thin dashed lines mark the trends for each election and the thick solid lines indicate the model predictions.
Figure 3
Simple Difference-in-Difference Fails on Synthetic Election Data
Source: author’s calculations.
Because the trends are not parallel, the average gap between 2016 and 2019 is 11.8 percentage points among polling stations included in the TSE announcement, but 15.1 percentage points among the SHUTDOWN stations. This results in a “difference-in-difference” of 3.3 percentage points (the difference between the late-ARRIVAL difference of 15.1 percentage points and the early-ARRIVAL difference of 11.8 percentage points).
Escobari and Hoover’s interpretation of the 3.3 percentage point double-difference is that there is fraud in the late-ARRIVING polling stations. But there is nothing interesting about these at all, except that they are “geographically” more favorable to the incumbent. The double-difference arises because the margins are more sensitive to geography in 2019 — something that exists throughout the data.
We may, as before, include geographic controls by adjusting at the precinct level. While this will remove the overall trend across elections, it will preserve the cross-election differences; the difference in trends will not change. This means that the difference-in-difference also will not change, and therefore Escobari and Hoover would again misinterpret this increased geographic sensitivity as “fraud.”
Figure 4
Precinct-Level Adjustment of Synthetic Data Has No Effect on the Estimate
Source: author’s calculations.
In Table 2, we see the statistical output from applying the difference-in-difference model to both the “left” and “right” data and with varying levels of geographic controls. The difference-in-difference is listed as “SHUTDOWN x Y2019.”
Table 2
| (1) | (2) | (3) | (4) | (5) | (6) | |
| SHUTDOWN x Y2019 | 4.703 (1.03) | 4.703 (1.069) | 4.703 (1.456) | 3.343 (0.150) | 3.343 (0.156) | 3.343 (0.212) |
| SHUTDOWN | 10.75 (0.784) | -2.202 (0.543) | 12.91 (0.137) | -1.687 (0.127) | ||
| Y2019 | 11.74 (0.437) | 11.74 (0.436) | 11.74 (0.618) | 11.79 (0.063) | 11.79 (0.065) | 11.79 (0.089) |
| Constant | -3.963 (0.331) | -1.754 (0.229) | -2.129 (0.280) | -4.442 (0.068) | -1.956 (0.047) | -2.241 (0.040) |
| Precinct | No | Yes | Yes | No | Yes | Yes |
| Tally Sheet | No | No | Yes | No | No | Yes |
| 70,000 | 70,000 | 70,000 | 70,000 | 70,000 | 70,000 | |
| 0.0231 | 0.523 | 0.530 | 0.332 | 0.746 | 0.861 | |
Source: author’s calculations.
Note that within each data set, the level of geographic control does not change the point estimate of the double-difference at all, but the uncertainty is greater in the “left” data — which is less fully explained by geography.
The reason that the point estimates are constant within each data set is that our data is complete: we have data for 2016 and 2019 at each polling station. The difference-in-difference model applied to the vote margins reduces exactly to an ordinary difference model applied to the increases in margins from 2016 to 2019. In other words, with the difference-in-difference model we are asking simply by how much more margins rose on average in the SHUTDOWN group compared to the rise in the earlier polling stations. If there is any benign reason why the SHUTDOWN margins would rise by more, then the double-difference overestimates fraud.
There is no fraud in the synthetic data. Rather, the identifying “parallel trends” assumption does not hold. The interpretation of the double-difference as fraud is entirely mistaken.
In the next post, we will apply these methods to the actual election data and observe that the problem of nonparallel trends persists.
This is the tenth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, part eight, and part nine.
In the previous post, we observed that even if geography is stable from one election to another, there is no guarantee that the effect of geography on vote shares is consistent over time. This may result in a widening gap between observed margins over the progress of a count, even absent any fraud. We also observed that the results at the polling stations included in the TSE announcement exhibited such a trend. We asserted that this poses a problem for Escobari and Hoover. In this post, we demonstrate how their “difference-in-difference” models mistakenly identify fraud when the difference in trends interacts with the counting bias — even when the resulting trends are linear.
One way of demonstrating this is to apply Escobari and Hoover to synthetic election data where we have total control of the amount of fraud. We may create a “geography” variable. We will not be able to observe this variable directly in our analysis, but it will be constant across precincts and will correlate with ARRIVAL. We may then generate election results based on the hidden geography alone, with no consideration to ARRIVAL. For clarity of illustration, we will then break out the SHUTDOWN group as polling stations in the last sixth of ARRIVAL.
In Figure 1, we see two different synthetic election results. Both the results on the left and the results on the right have the same overall trend over the order in which polling stations transmitted. In each, we have marked out the SHUTDOWN polling stations in green. There is no within-precinct-trend in either the left or right, nor does SHUTDOWN have any impact on the margin. The only difference between left and right is how much geography explains the differences across precincts.
Figure 1
Two Examples of Synthetic Election Data
Source: author’s calculations.
On the right, geography explains quite a bit and therefore the overall trend is clearer because the order of transmission is an imperfect proxy for geography.
We may produce difference estimates on this data just as we had on the actual election results of 2019. In both the left and the right, Escobari and Hoover’s difference estimates identify fraud where none exists — unless we adjust at the precinct level.
Table 1
| (1) | (2) | (3) | (4) | |
| SHUTDOWN | 15.46 (0.706) | 0.233 (0.110) | 16.26 (0.141) | 0.132 (0.706) |
| Constant | 7.780 (0.298) | 10.38 (0.037) | 7.370 (0.076) | 10.11 (0.044) |
| Precinct | No | Yes | No | Yes |
| 35,000 | 35,000 | 35,000 | 35,000 | |
| 0.0136 | 0.988 | 0.199 | 0.773 |
Source: author’s calculations.
In generating the synthetic election results, we have not even identified precincts as counted entirely late, so we know there is no late precinct-level fraud. However, that should not stop up from adding in synthetic 2016 data. As we see in Figure 2, the synthetic 2019 results are more sensitive to geography than the 2016 results.
Figure 2
Example of Synthetic Data Covering Multiple Elections
Source: author’s calculations.
So what do Escobari and Hoover do with this added information? In the simple difference models, they simply compared the average margin among those polling stations included in the TSE announcement to those that were excluded. They assume that absent fraud there is no difference between the two. In the “difference-in-difference” models, the baseline for fraud in 2019 is not zero, but whatever difference is observed in 2016. Graphically, we may see Escobari and Hoover’s baseline difference-in-difference model applied to the synthetic data in Figure 3. The thin dashed lines mark the trends for each election and the thick solid lines indicate the model predictions.
Figure 3
Simple Difference-in-Difference Fails on Synthetic Election Data
Source: author’s calculations.
Because the trends are not parallel, the average gap between 2016 and 2019 is 11.8 percentage points among polling stations included in the TSE announcement, but 15.1 percentage points among the SHUTDOWN stations. This results in a “difference-in-difference” of 3.3 percentage points (the difference between the late-ARRIVAL difference of 15.1 percentage points and the early-ARRIVAL difference of 11.8 percentage points).
Escobari and Hoover’s interpretation of the 3.3 percentage point double-difference is that there is fraud in the late-ARRIVING polling stations. But there is nothing interesting about these at all, except that they are “geographically” more favorable to the incumbent. The double-difference arises because the margins are more sensitive to geography in 2019 — something that exists throughout the data.
We may, as before, include geographic controls by adjusting at the precinct level. While this will remove the overall trend across elections, it will preserve the cross-election differences; the difference in trends will not change. This means that the difference-in-difference also will not change, and therefore Escobari and Hoover would again misinterpret this increased geographic sensitivity as “fraud.”
Figure 4
Precinct-Level Adjustment of Synthetic Data Has No Effect on the Estimate
Source: author’s calculations.
In Table 2, we see the statistical output from applying the difference-in-difference model to both the “left” and “right” data and with varying levels of geographic controls. The difference-in-difference is listed as “SHUTDOWN x Y2019.”
Table 2
| (1) | (2) | (3) | (4) | (5) | (6) | |
| SHUTDOWN x Y2019 | 4.703 (1.03) | 4.703 (1.069) | 4.703 (1.456) | 3.343 (0.150) | 3.343 (0.156) | 3.343 (0.212) |
| SHUTDOWN | 10.75 (0.784) | -2.202 (0.543) | 12.91 (0.137) | -1.687 (0.127) | ||
| Y2019 | 11.74 (0.437) | 11.74 (0.436) | 11.74 (0.618) | 11.79 (0.063) | 11.79 (0.065) | 11.79 (0.089) |
| Constant | -3.963 (0.331) | -1.754 (0.229) | -2.129 (0.280) | -4.442 (0.068) | -1.956 (0.047) | -2.241 (0.040) |
| Precinct | No | Yes | Yes | No | Yes | Yes |
| Tally Sheet | No | No | Yes | No | No | Yes |
| 70,000 | 70,000 | 70,000 | 70,000 | 70,000 | 70,000 | |
| 0.0231 | 0.523 | 0.530 | 0.332 | 0.746 | 0.861 | |
Source: author’s calculations.
Note that within each data set, the level of geographic control does not change the point estimate of the double-difference at all, but the uncertainty is greater in the “left” data — which is less fully explained by geography.
The reason that the point estimates are constant within each data set is that our data is complete: we have data for 2016 and 2019 at each polling station. The difference-in-difference model applied to the vote margins reduces exactly to an ordinary difference model applied to the increases in margins from 2016 to 2019. In other words, with the difference-in-difference model we are asking simply by how much more margins rose on average in the SHUTDOWN group compared to the rise in the earlier polling stations. If there is any benign reason why the SHUTDOWN margins would rise by more, then the double-difference overestimates fraud.
There is no fraud in the synthetic data. Rather, the identifying “parallel trends” assumption does not hold. The interpretation of the double-difference as fraud is entirely mistaken.
In the next post, we will apply these methods to the actual election data and observe that the problem of nonparallel trends persists.
• BoliviaBoliviaElectionsEleccionesLatin America and the CaribbeanAmérica Latina y el CaribeWorldEl Mundo
This is the ninth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, and part eight.
In the previous post, we adjusted the average for a precinct to match the average for the entire election while preserving the variability of polling stations within each precinct. In doing so, we eliminated the cross-precinct trend and saw that exclusion from the TSE announcement explained very little of the differences between polling stations within precincts.
The problem with this approach — as Escobari and Hoover note — is that it fails to distinguish fraud applied to an entire precinct from a precinct that for benign reasons happens to be particularly supportive of Morales. That is, while we eliminated the cross-precinct trend on the grounds that it controls for a range of benign geographic and socioeconomic explanations for the differences in support among precincts, it eliminates all illicit explanations for these precinct-level differences as well.
By way of illustration, let us add a lot (almost half a percentage point) of artificial fraud to all the polling stations that were entirely excluded from the TSE announcement. On the left of Figure 1, we see that this causes the trend to swing even more sharply upward (because these precincts tend to report late). On the right, we see that in the adjusted data there is a shift in trend among the late-transmitting polling stations (because these were disproportionately excluded from the announcement). This is the kind of “fraud” that the difference models discussed in the previous post are good at detecting.
Figure 1
Addition of Fraudulent Votes to Late Polling Stations Shows Up Despite Precinct-Level Adjustment
Sources: TSE and author’s calculations.
On the other hand, we can add even more fraud, but if we apply it uniformly at the precinct level among just those precincts entirely excluded from the TSE announcement, it no longer appears as a shift in the adjusted data. Instead, as we see in the right of Figure 2, it merely shifts the entire trend upward. The earlier difference models are unable to detect the addition.
Figure 2
Addition of Fraudulent Votes to Late Precincts Does Not Show Up With Precinct-Level Adjustment
Sources: TSE and author’s calculations.
The problem is not mathematical, but conceptual. If we simply use geographic identifiers to infer differences in support for Morales among precincts, we cannot distinguish benign precinct-level differences from illicit ones. To make that distinction, we require additional information: either additional data or additional assumptions. Escobari and Hoover add both.
So what do we have? We could look for census data on population density, Internet access, Spanish fluency, educational attainment, and the like, but we are not likely to find these at a sufficiently fine geographic level to disentangle the effects. We might consider ARRIVAL itself as a control. One problem with conditioning the fraud estimate on each polling station’s position in the order of transmission is that the effect of the bias can be nonlinear over the count, even if the bias is constant. If, as we saw in post #2, the bias plausibly results in a benign, late, sharp swing in the trend, this will be difficult to disentangle from SHUTDOWN.
Escobari and Hoover chose the results of the 2016 referendum to supplement their data. As we observed in post #5, precincts with voters that more heavily favored the referendum were less fully included in the initial preliminary results announced by the TSE. In the extreme, voters at precincts entirely excluded from that announcement were on balance much more likely to support the referendum than those at precincts that were at least partially included. Support for the referendum is also a reasonable, if imperfect, proxy for political leanings. That voters supportive of the referendum might also support Morales is not surprising; approval of the referendum would have removed term limits, and Morales was the incumbent president and, at the time, was term-limited under the constitution. Geographic and socioeconomic determinants of voter behavior may affect support for both.
There are several problems with using the 2016 margins in lieu of geography. The first is it is not possible to match polling stations in 2016 to polling stations in 2019 in any simple fashion. Most obviously, the 2019 election had 34,555 tally sheets — 33,048 from within the country, and 1,507 representing voters abroad. By contrast, the 2016 vote had only 29,224 tally sheets within the country, and 1,143 abroad. There is literally no way to directly match sheets. There isn’t even a clear way to match precincts, as precinct borders moved, split, and merged between 2016 and 2019. Even precincts with the same names across different elections sometimes had different numbers of polling stations. Escobari and Hoover claim to have performed a matching of polling stations, but it is not clear how they managed it.[1]
Second, the relationship between margins in 2016 and margins in 2019 is indirect, as indicated in the above diagram. By itself, a vote for the referendum in 2016 does not cause a voter to support Morales in 2019. We might guess that factors (such as socioeconomic status) that may help explain voter behavior are, over time, stable within small geographic areas such as precincts.
However, even if this is true, it doesn’t mean that the impacts of these factors on voter behavior are stable as well. The relationship between rurality and vote margin in 2016 need not be the same as the relationship between rurality and vote margin in 2019. For example, the electorate may have become more polarized, geographically, in those three years. If Morales played to a base of support in rural areas while neglecting urban areas, the change in margins across the years will vary by geography and thereby trend over the count.
There are issues with turnout as well. Turnout for the referendum was unusually low among voters residing outside Bolivia. According to a study by the OEP, reasons included nonmandatory participation, general disinterest, and seasonal travel. Those residing abroad who did turn out favored the referendum, on balance.
That aside, to the extent one might think of 2016 as a proxy for support for Morales, the relationship between the vote in 2016 and the vote in 2019 is more complex. Suppose we believe that anyone voting for the referendum would also vote for Morales. What then of those opposed to the referendum? In 2019, the opposition split among several candidates. Even if a vote for the referendum translated into a vote for Morales, it is not the case that a vote against the referendum translated into a vote for Mesa.
The split was not even stable across geographies. Mesa received 68 percent of the non-Morales valid votes in early, urban localities, compared to only 44 percent in rural localities. Mesa picked up an even larger share of the non-Morales vote in the department capitals.
| Percent of Announced Vote | Sí Share of Valid | Morales Share of Valid | Mesa Share of Non-Morales | |
|---|---|---|---|---|
| Rural | 21.47 | 65.6 | 65.3 | 45.1 |
| Foreign | 3.29 | 60.8 | 58.6 | 55.5 |
| El Alto | 10.74 | 58.5 | 55.1 | 50.8 |
| Other Urban Non-Capital | 19.37 | 49.7 | 47.5 | 60.3 |
| Capital | 45.13 | 36.4 | 32.5 | 75.9 |
| Total | 100.00 | 48.4 | 45.7 | 62.9 |
Sources: TSE and author’s calculations.
That is, from the perspective of 2016 as a proxy, Morales managed to hold ground in rural areas, though he lost a little support elsewhere. More significantly, the opposition split along geographic lines. Even within individual capital cities, Mesa picked up a lesser share of the opposition vote in precincts favoring the referendum.
Figure 3
Mesa Shares of Opposition Votes Fall With Support for Referendum (TSE Announcement)
Sources: TSE and author’s calculations.
This drives a wedge between the 2016 and 2019 margins. Compared to 2016, Morales’s margin is a few percentage points better in the capital cities, but quite a bit larger in rural areas. Because of the timing differences, the trends will not be parallel. Instead, Morales’s margin will be even larger compared to the referendum among the later arrivals, and we might see something like Figure 4.
Figure 4
Margins for Morales and the Referendum at the Time of the TSE Announcement
Sources: TSE, OEP, and author’s calculations.
So why does any of this matter in our understanding of Escobari and Hoover? The TSE announcement disproportionately excluded precincts where voters supported the referendum, so we expect a larger gap between 2016 and 2019 among SHUTDOWN polling stations than those included in the announcement. Yet the fundamental assumption made by Escobari and Hoover is that absent fraud, the gap should be — on average — no larger in the SHUTDOWN group. They therefore dismiss all of the above as fraud.
Regardless, Escobari and Hoover’s analyses all depend on parallel trends in fraud-free areas in order to identify fraud elsewhere. In the next post, we will see how nonparallel trends cause their models to misidentify fraud even where none exists.
Bonus Notes:
Escobari and Hoover note that “only MAS and CC appear to have experienced relatively big changes at the shutdown as the other shares of political parties in the election appear relatively stable.” In fact, as a share of the opposition vote, the CC share is one of the most stable, having fallen by 8.1 percent from 69.8 to 64.1 percent of the non-MAS valid vote. The PDC rose by 21 percent, from 16.1 to 19.5 percent; MTS by 31 percent, from 2.3 to 3.0; and MNR 27 percent, from 1.3 to 1.6. The apparent stability of the minor-party shares is precisely because they are minor parties. This will be important later on.
[1] We also performed a matching between 2016 and 2019. Our matches were sometimes approximate and were in any case at the precinct-level only. The matches we made (as well as the code to produce them) are included in the data set archived at https://github.com/ViscidKonrad/Bolivia-Escobari-Hoover.
This is the ninth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to posts: part one, part two, part three, part four, part five, part six, part seven, and part eight.
In the previous post, we adjusted the average for a precinct to match the average for the entire election while preserving the variability of polling stations within each precinct. In doing so, we eliminated the cross-precinct trend and saw that exclusion from the TSE announcement explained very little of the differences between polling stations within precincts.
The problem with this approach — as Escobari and Hoover note — is that it fails to distinguish fraud applied to an entire precinct from a precinct that for benign reasons happens to be particularly supportive of Morales. That is, while we eliminated the cross-precinct trend on the grounds that it controls for a range of benign geographic and socioeconomic explanations for the differences in support among precincts, it eliminates all illicit explanations for these precinct-level differences as well.
By way of illustration, let us add a lot (almost half a percentage point) of artificial fraud to all the polling stations that were entirely excluded from the TSE announcement. On the left of Figure 1, we see that this causes the trend to swing even more sharply upward (because these precincts tend to report late). On the right, we see that in the adjusted data there is a shift in trend among the late-transmitting polling stations (because these were disproportionately excluded from the announcement). This is the kind of “fraud” that the difference models discussed in the previous post are good at detecting.
Figure 1
Addition of Fraudulent Votes to Late Polling Stations Shows Up Despite Precinct-Level Adjustment
Sources: TSE and author’s calculations.
On the other hand, we can add even more fraud, but if we apply it uniformly at the precinct level among just those precincts entirely excluded from the TSE announcement, it no longer appears as a shift in the adjusted data. Instead, as we see in the right of Figure 2, it merely shifts the entire trend upward. The earlier difference models are unable to detect the addition.
Figure 2
Addition of Fraudulent Votes to Late Precincts Does Not Show Up With Precinct-Level Adjustment
Sources: TSE and author’s calculations.
The problem is not mathematical, but conceptual. If we simply use geographic identifiers to infer differences in support for Morales among precincts, we cannot distinguish benign precinct-level differences from illicit ones. To make that distinction, we require additional information: either additional data or additional assumptions. Escobari and Hoover add both.
So what do we have? We could look for census data on population density, Internet access, Spanish fluency, educational attainment, and the like, but we are not likely to find these at a sufficiently fine geographic level to disentangle the effects. We might consider ARRIVAL itself as a control. One problem with conditioning the fraud estimate on each polling station’s position in the order of transmission is that the effect of the bias can be nonlinear over the count, even if the bias is constant. If, as we saw in post #2, the bias plausibly results in a benign, late, sharp swing in the trend, this will be difficult to disentangle from SHUTDOWN.
Escobari and Hoover chose the results of the 2016 referendum to supplement their data. As we observed in post #5, precincts with voters that more heavily favored the referendum were less fully included in the initial preliminary results announced by the TSE. In the extreme, voters at precincts entirely excluded from that announcement were on balance much more likely to support the referendum than those at precincts that were at least partially included. Support for the referendum is also a reasonable, if imperfect, proxy for political leanings. That voters supportive of the referendum might also support Morales is not surprising; approval of the referendum would have removed term limits, and Morales was the incumbent president and, at the time, was term-limited under the constitution. Geographic and socioeconomic determinants of voter behavior may affect support for both.
There are several problems with using the 2016 margins in lieu of geography. The first is it is not possible to match polling stations in 2016 to polling stations in 2019 in any simple fashion. Most obviously, the 2019 election had 34,555 tally sheets — 33,048 from within the country, and 1,507 representing voters abroad. By contrast, the 2016 vote had only 29,224 tally sheets within the country, and 1,143 abroad. There is literally no way to directly match sheets. There isn’t even a clear way to match precincts, as precinct borders moved, split, and merged between 2016 and 2019. Even precincts with the same names across different elections sometimes had different numbers of polling stations. Escobari and Hoover claim to have performed a matching of polling stations, but it is not clear how they managed it.[1]
Second, the relationship between margins in 2016 and margins in 2019 is indirect, as indicated in the above diagram. By itself, a vote for the referendum in 2016 does not cause a voter to support Morales in 2019. We might guess that factors (such as socioeconomic status) that may help explain voter behavior are, over time, stable within small geographic areas such as precincts.
However, even if this is true, it doesn’t mean that the impacts of these factors on voter behavior are stable as well. The relationship between rurality and vote margin in 2016 need not be the same as the relationship between rurality and vote margin in 2019. For example, the electorate may have become more polarized, geographically, in those three years. If Morales played to a base of support in rural areas while neglecting urban areas, the change in margins across the years will vary by geography and thereby trend over the count.
There are issues with turnout as well. Turnout for the referendum was unusually low among voters residing outside Bolivia. According to a study by the OEP, reasons included nonmandatory participation, general disinterest, and seasonal travel. Those residing abroad who did turn out favored the referendum, on balance.
That aside, to the extent one might think of 2016 as a proxy for support for Morales, the relationship between the vote in 2016 and the vote in 2019 is more complex. Suppose we believe that anyone voting for the referendum would also vote for Morales. What then of those opposed to the referendum? In 2019, the opposition split among several candidates. Even if a vote for the referendum translated into a vote for Morales, it is not the case that a vote against the referendum translated into a vote for Mesa.
The split was not even stable across geographies. Mesa received 68 percent of the non-Morales valid votes in early, urban localities, compared to only 44 percent in rural localities. Mesa picked up an even larger share of the non-Morales vote in the department capitals.
| Percent of Announced Vote | Sí Share of Valid | Morales Share of Valid | Mesa Share of Non-Morales | |
|---|---|---|---|---|
| Rural | 21.47 | 65.6 | 65.3 | 45.1 |
| Foreign | 3.29 | 60.8 | 58.6 | 55.5 |
| El Alto | 10.74 | 58.5 | 55.1 | 50.8 |
| Other Urban Non-Capital | 19.37 | 49.7 | 47.5 | 60.3 |
| Capital | 45.13 | 36.4 | 32.5 | 75.9 |
| Total | 100.00 | 48.4 | 45.7 | 62.9 |
Sources: TSE and author’s calculations.
That is, from the perspective of 2016 as a proxy, Morales managed to hold ground in rural areas, though he lost a little support elsewhere. More significantly, the opposition split along geographic lines. Even within individual capital cities, Mesa picked up a lesser share of the opposition vote in precincts favoring the referendum.
Figure 3
Mesa Shares of Opposition Votes Fall With Support for Referendum (TSE Announcement)
Sources: TSE and author’s calculations.
This drives a wedge between the 2016 and 2019 margins. Compared to 2016, Morales’s margin is a few percentage points better in the capital cities, but quite a bit larger in rural areas. Because of the timing differences, the trends will not be parallel. Instead, Morales’s margin will be even larger compared to the referendum among the later arrivals, and we might see something like Figure 4.
Figure 4
Margins for Morales and the Referendum at the Time of the TSE Announcement
Sources: TSE, OEP, and author’s calculations.
So why does any of this matter in our understanding of Escobari and Hoover? The TSE announcement disproportionately excluded precincts where voters supported the referendum, so we expect a larger gap between 2016 and 2019 among SHUTDOWN polling stations than those included in the announcement. Yet the fundamental assumption made by Escobari and Hoover is that absent fraud, the gap should be — on average — no larger in the SHUTDOWN group. They therefore dismiss all of the above as fraud.
Regardless, Escobari and Hoover’s analyses all depend on parallel trends in fraud-free areas in order to identify fraud elsewhere. In the next post, we will see how nonparallel trends cause their models to misidentify fraud even where none exists.
Bonus Notes:
Escobari and Hoover note that “only MAS and CC appear to have experienced relatively big changes at the shutdown as the other shares of political parties in the election appear relatively stable.” In fact, as a share of the opposition vote, the CC share is one of the most stable, having fallen by 8.1 percent from 69.8 to 64.1 percent of the non-MAS valid vote. The PDC rose by 21 percent, from 16.1 to 19.5 percent; MTS by 31 percent, from 2.3 to 3.0; and MNR 27 percent, from 1.3 to 1.6. The apparent stability of the minor-party shares is precisely because they are minor parties. This will be important later on.
[1] We also performed a matching between 2016 and 2019. Our matches were sometimes approximate and were in any case at the precinct-level only. The matches we made (as well as the code to produce them) are included in the data set archived at https://github.com/ViscidKonrad/Bolivia-Escobari-Hoover.
• BoliviaBoliviaElectionsEleccionesLatin America and the CaribbeanAmérica Latina y el CaribeWorldEl Mundo
This is the eighth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to other posts: part one, part two, part three, part four, part five, part six, part seven, and part nine.
In the previous post, we investigated the confounding effects of rurality and socioeconomic status on a naive estimate of fraud. We noted that if we could divide polling stations into groups such that the confounding effects do not vary within each group, then we may begin to disentangle the effects of, say, Internet connectivity, on whether a station was included in the TSE announcement.
The most obvious way to manage such factors is to assume that voters within small geographic areas are effectively indistinguishable. Of course, the smaller the geographic area, the more truth there will be in this assumption. To a first approximation, we would expect voters of the same precinct to have more similar socioeconomic status or rurality than voters at different precincts, even within the same municipality. At least, we may hope.
Consider column 4 of Escobari and Hoover’s difference estimates. There, the polling stations are grouped by municipality. In practice, this meant performing exactly the same analysis as before, but only after eliminating the average differences in vote margins across municipalities. Importantly, the adjustment comes at the municipality level so that differences between polling stations of the same municipality are preserved. Because the averages depend on the weighting scheme, the adjustments will be different if we take into account the number of valid votes at each polling station.
In Figure 1, we see the unadjusted and (weighted) adjustments for polling stations at two municipalities: New York (United States) and Acasio (Potosí). On the left, we see that New York went heavily for Mesa, while Acasio greatly favored Morales. On the right, we have adjusted the margins to take into account only the differences across municipalities.
Figure 1
Official and Municipality-Adjusted Results in New York and Acasio
Sources: TSE and author’s calculations.
In Figure 2, we see how applying the adjustment to all municipalities affects the overall trend. We see that municipality explains most, but not all of the trend in support across ARRIVAL.
Figure 2
Official and Municipality-Adjusted Trends in Vote Margin
Sources: TSE and author’s calculations.
Our replication and corrections to the municipality-adjusted results are presented alongside their reported results in Table 1.
Table 1
Replication and Reanalyses of Escobari and Hoover’s Municipality-Adjusted Difference Model
| As Published | Replication | Correct Voters | Weighted | |
|---|---|---|---|---|
| (1) | (2) | (3) | (4) | |
| Variable | ||||
| SHUTDOWN | 7.243 (0.437) | 7.243 (0.437) | 7.214 (0.436) | 7.766 (0.447) |
| Constant | 11.28 (0.162) | 11.28 (0.162) | 11.31 (0.162) | 9.32 (0.166) |
| Observations | 34,529 | 34,529 | 34,551 | 34,551 |
| R2 | 0.640 | 0.640 | 0.640 | 0.627 |
Sources: TSE and author’s calculations.
Due to the way that the municipality adjustments (the “fixed effects”) are identified in this particular model, the coefficients are harder to interpret, even when the polling stations are weighted by size. However, when weighted, the mean margin still matches the data: 9.32+0.16 x 7.7766 = 10.56, where 16 percent of votes were excluded from the TSE announcement. More importantly, the effect of the SHUTDOWN is greatly reduced. More than half of the measured effect was coming from differences in municipalityrather than being verified late in the count. This offers considerable evidence that the problem of nickels before dimes is a serious issue that must be completely accounted for when estimating fraud.
As we shrink the group size, the differences between polling stations within each group also narrow. Correspondingly, there is less variation within each group left to explain by exclusion from the TSE announcement. The more we account for confounding factors, the more the SHUTDOWN effect shrinks. The smallest practical unit we may employ is the precinct.
In Figure 3, we see that precincts account for the overall trend almost entirely. Very little is left in the adjusted results.
Figure 3
Official and Precinct-Adjusted Trends in Vote Margin
Sources: TSE and author’s calculations.
Table 2
Replication and Reanalyses of Escobari and Hoover’s Precinct-Adjusted Difference Model
| As Published | Replication | Correct Voters | Weighted | |
|---|---|---|---|---|
| (1) | (2) | (3) | (4) | |
| Variable | ||||
| SHUTDOWN | 0.365 (0.194) | 0.377 (0.194) | 0.360 (0.193) | 0.287 (0.192) |
| Constant | 12.39 (0.0631) | 12.39 (0.0631) | 12.41 (0.0630) | 10.52 (0.0632) |
| Observations | 34,529 | 34,529 | 34,551 | 34,551 |
| R2 | 0.958 | 0.958 | 0.958 | 0.958 |
Sources: TSE and author’s calculations.
Rather than an increase of 16 percentage points, we see that on average in a given split precinct, the excluded polling stations favored Morales by only an additional 0.3 percentage points — or about 3,000 net votes. This difference is far from politically significant, with the model otherwise explaining all but 0.046 percentage points of Morales’s final margin.
Again, this doesn’t mean that the 3,000 votes amounted to fraud; it means that we have yet to offer any alternative explanation for the otherwise unexpected difference. For example, if surname is associated with both support for Morales and delays in reporting, the difference could be accounted for there. According to Escobari and Hoover, voters with surnames starting with “Z” voted more strongly in favor of Mesa when compared to others. Perhaps such surnames are associated with a particular socioeconomic status. In any case, coming at the end of the alphabet, “Z” voters voters are assigned to the highest-numbered polling stations in a given precinct, and therefore tend to report early (subject to the “small-station” effect discussed in post #3). Thus, we observe a bias in polling stations with opposition-heavy “Z” voters reporting disproportionately early and so more likely to be included in the TSE announcement. In turn, this would cause us to underestimate Morales’s support in polling stations excluded from the announcement. Worse, Escobari and Hoover do not weigh polling stations by the number of voters, so smaller “Z” surname-heavy stations have oversized impacts on the analysis, exaggerating the difference.[1]
We do not explore whether the 3,000 votes may be accounted for in this manner or, alternatively, if other explanations (potentially including fraud) are required. However, even if we assume that the entire 3,000-vote difference came from fraud, this only would knock Morales’s margin down to 10.52 percentage points — not nearly enough to change the outcome of the election.
This has two important takeaways. First, geography has the capacity to explain much of the increase in Morales’s support. Even if we assume the worst interpretation of the results, the increase in support officially reported for Morales among the late polling stations in split precincts is very small. Second, because 84 percent of the late polling stations came from these split precincts, there is little room left for politically relevant fraud. Escobari and Hoover’s 16.26 percentage points do not form a credible estimate of fraud in the late polling stations.
To illustrate, note that Escobari and Hoover pin their fraud estimate at just under 160,000 votes. If we generously quadruple the number of unexpected votes coming from late polling stations in split precincts (12,000 instead of only 3,000), this leaves 148,000 “fraudulent” votes among the 153,890 valid votes cast in the late precincts. In other words, to reconcile with Escobari and Hoover’s estimate of 16.26 percentage points (see previous post), the late polling stations — officially reported as having supported Morales by more than 50 percentage points — actually supported Mesa by about 45 percentage points.
This would be not only a shockingly obvious level of manipulation, it would need to be vote manipulation in polling stations overwhelmingly in favor of Mesa. These would be stations where most jurors would be Mesa voters, where Mesa voters would witness ballot counting, and where a CC (Mesa’s party) representative likely would be present and offered a copy of the acta as a security measure. It would make no sense for Morales-friendly fraud to take place at these polling stations. And if forged actas were later offered as substitutes, where are the copies of the originals? The OAS audit report indicates that its audit team compared official actas with copies it obtained, but the report did not mention any instances of numerical discrepancy.
Absent any credible way that this level of vote manipulation could have been carried out, we fall back on a much more plausible explanation: Escobari and Hoover’s interpretation of their results as a measure of fraud is incorrect.
In the next post, we will begin to explore approaches to disentangle the effect of geography from that of possible precinct-level manipulations.
[1] On the other hand, they report that “Y” voters favored Morales, so in many areas the effect may be reversed.
This is the eighth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to other posts: part one, part two, part three, part four, part five, part six, part seven, and part nine.
In the previous post, we investigated the confounding effects of rurality and socioeconomic status on a naive estimate of fraud. We noted that if we could divide polling stations into groups such that the confounding effects do not vary within each group, then we may begin to disentangle the effects of, say, Internet connectivity, on whether a station was included in the TSE announcement.
The most obvious way to manage such factors is to assume that voters within small geographic areas are effectively indistinguishable. Of course, the smaller the geographic area, the more truth there will be in this assumption. To a first approximation, we would expect voters of the same precinct to have more similar socioeconomic status or rurality than voters at different precincts, even within the same municipality. At least, we may hope.
Consider column 4 of Escobari and Hoover’s difference estimates. There, the polling stations are grouped by municipality. In practice, this meant performing exactly the same analysis as before, but only after eliminating the average differences in vote margins across municipalities. Importantly, the adjustment comes at the municipality level so that differences between polling stations of the same municipality are preserved. Because the averages depend on the weighting scheme, the adjustments will be different if we take into account the number of valid votes at each polling station.
In Figure 1, we see the unadjusted and (weighted) adjustments for polling stations at two municipalities: New York (United States) and Acasio (Potosí). On the left, we see that New York went heavily for Mesa, while Acasio greatly favored Morales. On the right, we have adjusted the margins to take into account only the differences across municipalities.
Figure 1
Official and Municipality-Adjusted Results in New York and Acasio
Sources: TSE and author’s calculations.
In Figure 2, we see how applying the adjustment to all municipalities affects the overall trend. We see that municipality explains most, but not all of the trend in support across ARRIVAL.
Figure 2
Official and Municipality-Adjusted Trends in Vote Margin
Sources: TSE and author’s calculations.
Our replication and corrections to the municipality-adjusted results are presented alongside their reported results in Table 1.
Table 1
Replication and Reanalyses of Escobari and Hoover’s Municipality-Adjusted Difference Model
| As Published | Replication | Correct Voters | Weighted | |
|---|---|---|---|---|
| (1) | (2) | (3) | (4) | |
| Variable | ||||
| SHUTDOWN | 7.243 (0.437) | 7.243 (0.437) | 7.214 (0.436) | 7.766 (0.447) |
| Constant | 11.28 (0.162) | 11.28 (0.162) | 11.31 (0.162) | 9.32 (0.166) |
| Observations | 34,529 | 34,529 | 34,551 | 34,551 |
| R2 | 0.640 | 0.640 | 0.640 | 0.627 |
Sources: TSE and author’s calculations.
Due to the way that the municipality adjustments (the “fixed effects”) are identified in this particular model, the coefficients are harder to interpret, even when the polling stations are weighted by size. However, when weighted, the mean margin still matches the data: 9.32+0.16 x 7.7766 = 10.56, where 16 percent of votes were excluded from the TSE announcement. More importantly, the effect of the SHUTDOWN is greatly reduced. More than half of the measured effect was coming from differences in municipalityrather than being verified late in the count. This offers considerable evidence that the problem of nickels before dimes is a serious issue that must be completely accounted for when estimating fraud.
As we shrink the group size, the differences between polling stations within each group also narrow. Correspondingly, there is less variation within each group left to explain by exclusion from the TSE announcement. The more we account for confounding factors, the more the SHUTDOWN effect shrinks. The smallest practical unit we may employ is the precinct.
In Figure 3, we see that precincts account for the overall trend almost entirely. Very little is left in the adjusted results.
Figure 3
Official and Precinct-Adjusted Trends in Vote Margin
Sources: TSE and author’s calculations.
Table 2
Replication and Reanalyses of Escobari and Hoover’s Precinct-Adjusted Difference Model
| As Published | Replication | Correct Voters | Weighted | |
|---|---|---|---|---|
| (1) | (2) | (3) | (4) | |
| Variable | ||||
| SHUTDOWN | 0.365 (0.194) | 0.377 (0.194) | 0.360 (0.193) | 0.287 (0.192) |
| Constant | 12.39 (0.0631) | 12.39 (0.0631) | 12.41 (0.0630) | 10.52 (0.0632) |
| Observations | 34,529 | 34,529 | 34,551 | 34,551 |
| R2 | 0.958 | 0.958 | 0.958 | 0.958 |
Sources: TSE and author’s calculations.
Rather than an increase of 16 percentage points, we see that on average in a given split precinct, the excluded polling stations favored Morales by only an additional 0.3 percentage points — or about 3,000 net votes. This difference is far from politically significant, with the model otherwise explaining all but 0.046 percentage points of Morales’s final margin.
Again, this doesn’t mean that the 3,000 votes amounted to fraud; it means that we have yet to offer any alternative explanation for the otherwise unexpected difference. For example, if surname is associated with both support for Morales and delays in reporting, the difference could be accounted for there. According to Escobari and Hoover, voters with surnames starting with “Z” voted more strongly in favor of Mesa when compared to others. Perhaps such surnames are associated with a particular socioeconomic status. In any case, coming at the end of the alphabet, “Z” voters voters are assigned to the highest-numbered polling stations in a given precinct, and therefore tend to report early (subject to the “small-station” effect discussed in post #3). Thus, we observe a bias in polling stations with opposition-heavy “Z” voters reporting disproportionately early and so more likely to be included in the TSE announcement. In turn, this would cause us to underestimate Morales’s support in polling stations excluded from the announcement. Worse, Escobari and Hoover do not weigh polling stations by the number of voters, so smaller “Z” surname-heavy stations have oversized impacts on the analysis, exaggerating the difference.[1]
We do not explore whether the 3,000 votes may be accounted for in this manner or, alternatively, if other explanations (potentially including fraud) are required. However, even if we assume that the entire 3,000-vote difference came from fraud, this only would knock Morales’s margin down to 10.52 percentage points — not nearly enough to change the outcome of the election.
This has two important takeaways. First, geography has the capacity to explain much of the increase in Morales’s support. Even if we assume the worst interpretation of the results, the increase in support officially reported for Morales among the late polling stations in split precincts is very small. Second, because 84 percent of the late polling stations came from these split precincts, there is little room left for politically relevant fraud. Escobari and Hoover’s 16.26 percentage points do not form a credible estimate of fraud in the late polling stations.
To illustrate, note that Escobari and Hoover pin their fraud estimate at just under 160,000 votes. If we generously quadruple the number of unexpected votes coming from late polling stations in split precincts (12,000 instead of only 3,000), this leaves 148,000 “fraudulent” votes among the 153,890 valid votes cast in the late precincts. In other words, to reconcile with Escobari and Hoover’s estimate of 16.26 percentage points (see previous post), the late polling stations — officially reported as having supported Morales by more than 50 percentage points — actually supported Mesa by about 45 percentage points.
This would be not only a shockingly obvious level of manipulation, it would need to be vote manipulation in polling stations overwhelmingly in favor of Mesa. These would be stations where most jurors would be Mesa voters, where Mesa voters would witness ballot counting, and where a CC (Mesa’s party) representative likely would be present and offered a copy of the acta as a security measure. It would make no sense for Morales-friendly fraud to take place at these polling stations. And if forged actas were later offered as substitutes, where are the copies of the originals? The OAS audit report indicates that its audit team compared official actas with copies it obtained, but the report did not mention any instances of numerical discrepancy.
Absent any credible way that this level of vote manipulation could have been carried out, we fall back on a much more plausible explanation: Escobari and Hoover’s interpretation of their results as a measure of fraud is incorrect.
In the next post, we will begin to explore approaches to disentangle the effect of geography from that of possible precinct-level manipulations.
[1] On the other hand, they report that “Y” voters favored Morales, so in many areas the effect may be reversed.
• BoliviaBoliviaElectionsEleccionesLatin America and the CaribbeanAmérica Latina y el CaribeWorldEl Mundo
This is the seventh in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to other posts: part one, part two, part three, part four, part five, part six, part eight, and part nine.
In the previous post, we took note of an error in the margin calculations by Escobari and Hoover. Though the effect on their calculations was small, the incorrect use of Válidos En Acta by Escobari and Hoover (among many others) generated controversy by making it appear that official vote totals did not correctly sum. Rather, these reflect clerical errors made by jurors at individual polling stations. We now pick up where we left off in post #5 when we noted that there was counting bias in the election. Here, we delve into the effects that bias had on the first results produced by Escobari and Hoover.
We begin with their “Difference Estimates,” almost exactly reproduced below. We attribute the discrepancies (indicated in red) to differences in assigning polling stations to precincts — a problem we identified with the first version of their 2019 paper, and that was likely not completely corrected.
Table 1
| CC | MAS | MAS-CC | ||||
|---|---|---|---|---|---|---|
| (1) | (2) | (3) | (4) | (5) | (6) | |
| Variable | ||||||
| SHUTDOWN | -8.286 (0.324) | 7.975 (0.343) | 16.26 (0.653) | 7.243 (0.437) | 6.762 (0.464) | 0.377 (0.194) |
| Constant | 36.86 (0.136) | 46.69 (0.134) | 9.830 (0.266) | 11.28 (0.162) | 11.36 (0.151) | 12.39 (0.063) |
| Fixed Effects[1] | ||||||
| Municipality | 129.6 | |||||
| Locality | 23.49 | |||||
| Precinct | 124.7 | |||||
| Observations | 34,529 | 34,529 | 34,529 | 34,529 | 34,529 | 34,529 |
| R2 | 0.017 | 0.016 | 0.017 | 0.640 | 0.740 | 0.958 |
Source: TSE and author’s calculations.
Notes: Dependent variables are percentages of Válidos En Acta (frequently missing or otherwise misreported on the tally sheets) and not of official valid votes. Standard errors in parentheses are robust. Differences from Escobari and Hoover noted in red.
[1] F-test statistics for fixed effects are not robust.
Note also that the analysis is not weighted by the number of voters at each station. For example, the constant for column 3 indicates the simple average margin for Morales across polling stations included in the TSE announcement was 9.83 percentage points — almost 2 percentage points above the official result at the time. Likewise, Escobari and Hoover’s result implies the simple average margin for Morales across all polling stations was 12.45 percentage points — again nearly 2 percentage points above the official result. This is because in actual elections, overall vote shares are not calculated the way Escobari and Hoover do. In actual elections, it is the vote totals, and not the average margins, that matter. Thus, polling stations with fewer votes have less impact on the final vote than do polling stations with more votes. Ignoring this makes Escobari and Hoover’s results difficult to put in the proper context.
Consider the two-precinct example of Table 2. In the rural precinct, there were 100 valid votes, which Morales won by 40 votes. In the urban precinct, Mesa won by 25 votes out of 250. The simple average margin is (40-10)/2 = 15 percentage points. But overall (taking both stations as a single group) Morales won by 40-25 = 15 votes out of 350, or only 4.3 percentage points.
Table 2
| Voters | Net Votes | Margin | |
|---|---|---|---|
| Rural | 100 | 40 | +40 |
| Urban | 250 | -25 | -10 |
| Combined | 350 | 15 | 4 |
We turn to column 3 of Table 1 above. In Table 3, we present the results of Escobari and Hoover alongside our replication and corrections to ease context. First, we note that our replication (column 2) exactly reproduces the published results (column 1). Second, we see that in employing the correct number of valid voters in the calculation, we have 22 more observations, missing only four polling stations that were annulled. Third, we note that once we weigh polling station data by the number of valid voters (column 4), the “Constant” falls by nearly 2 percentage points. This reflects putting the numbers in their proper context. Morales’s lead (based on the official numbers) at the polling stations included in the TSE announcement was 7.9 percent of the valid vote.
Likewise, in moving from column 3 to column 4, “SHUTDOWN” grows by 0.5, meaning that in giving too much importance to small polling stations, Escobari and Hoover wind up underestimating the increase in support when moving from stations included in the TSE announcement to those outstanding. Taken as a group, Morales’s margin on outstanding polling stations is 7.883+16.77 = 24.65 percentage points, and not 9.843+16.27 = 26.12.
Table 3
| As Published | Replication | Correct Voters | Weighted | |
|---|---|---|---|---|
| (1) | (2) | (3) | (4) | |
| Variable | ||||
| SHUTDOWN | 16.26 (0.653) | 16.26 (0.653) | 16.27 (0.653) | 16.77 (0.663) |
| Constant | 9.830 (0.266) | 9.830 (0.266) | 9.843 (0.266) | 7.883 (0.264) |
| Observations | 34,529 | 34,529 | 34,551 | 34,551 |
| R2 | 0.017 | 0.017 | 0.017 | 0.019 |
There are several ways of interpreting these results. One is to simply say that they measure the amount by which late polling stations more heavily favored Morales, and make no attribution as to the cause. This analysis is merely descriptive.
Another is to say that these results measure the bias in counting opposition polling stations disproportionately early. Perhaps rural stations that happen to favor Morales were simply more likely to be late and were therefore excluded from the TSE announcement — that is, nickels before dimes.
A third is to say that the announcement itself marked a division: the mere fact that a polling station was not included in the announcement explains the increase in support and that if all had been included, Morales would have won by only 7.9 percentage points. Because voting took place before the announcement, exclusion from the announcement should not by itself cause Morales’s support in those polling stations to rise. The implication is that the rise must be due to the addition of fraud, either committed after the announcement or in a deliberate delay in reporting polling station results already known to contain fraud. That is, in this interpretation SHUTDOWN would be a proxy for fraud.
In this figure, we are interested in the connection from fraud to margin, highlighted in red. Fraud is not something we can directly observe in the data, but one proposed mechanism is that the time required to implement fraud required delaying verification of those tally sheets until after the TSE announcement (hence whether or not it was included in the post-announcement “SHUTDOWN” group).
Note that the published result is inconsistent with respect to this interpretation. Escobari and Hoover argue in favor of the 7.9 percentage point counterfactual, but the constant in the model implies a projected margin of 9.8 percentage points — not statistically distinct from the election-determining 10 percentage point threshold. This reinforces our point that the use of weights in the analysis is important when one wishes to interpret the results.
This third explanation of the 16 percentage point difference as a measure of actual fraud is difficult to defend because of the confounding explanations in the second analysis. That is, in the model, SHUTDOWN captures everything impacting Morales’s margin that varies across the groups. There exists a whole apparatus of factors all complicating the interpretation of the 16 percentage point difference as fraud.
We are still only interested in the effect of fraud indicated in red. Of course, tally sheets wound up in the SHUTDOWN group for benign reasons as well as because of any putative malice. Consider those that transmitted late (late “ARRIVAL” to the electoral authorities) and those that transmitted their transcriptions but could not be verified in a timely manner. We tie both ARRIVAL and SHUTDOWN to rurality, but here “Rural” is a stand-in for a battery of various geographic or socioeconomic factors, any of which may have a different effect on each. Importantly, these same factors carry information about support for Morales, and so impact the observed margin. Finally, the number of voters at any given polling station helps to determine the order of ARRIVAL as smaller stations are able to complete their votes counts more rapidly.
The problem is that if we control for SHUTDOWN alone, that carries with it information about all the geographic factors. For example, given that a station is in the SHUTDOWN group, we can infer that it is more rural and therefore more heavily in favor of Morales. We can’t say if the 16 percentage point difference is all due to the “fraud” Escobari and Hoover seek to measure, or if it is all due to differences in geographic/socioeconomic factors. A more complex statistical model is required.
Of course, it is not easy to quantify — let alone identify — every confounding factor. We must bend somewhat to the reality of data availability. We must recognize that SHUTDOWN is a residual effect. Everything that accounts for the late increase in margin that is not expressly modeled is captured by SHUTDOWN. That includes both possible fraud and any overlooked nickels before dimes. A “statistically significant” SHUTDOWN coefficient doesn’t indicate the existence of fraud, specifically, unless we can adequately disentangle the effects.
To that point, an unexplained 16.77 percentage point difference would be politically worrisome in the absence of other information. Applied to the 16 percent of the election included in the SHUTDOWN group, this implies the exceedingly simple model fails to explain 2.7 percentage points of Morales’s final margin. We can see this directly in the estimated constant of Table 3, Column 4, which says that the non-SHUTDOWN group favored Morales by 7.9 percentage points. If the SHUTDOWN group is effectively identical, then the final election margin should have been close to 7.9 percentage points and not the official 10.56. Thus, the model leaves a politically significant unexplained residual that Escobari and Hoover interpret to be fraud. However, we know for a fact that the critical assumption that the SHUTDOWN group is identical is false. The model does not take into account important differences between the SHUTDOWN and non-SHUTDOWN groups. Nickels before dimes.
One way to cope with a dizzying array of possible differences is to divide polling stations into smaller groups. In doing so, we may hope to make assignments so that within each group these confounding factors are more or less constant — that we cannot easily distinguish one polling station from another except by their inclusion or exclusion from the TSE announcement.
As we will see in the next post, this is the reasoning behind columns 4–6 of Table 1.
This is the seventh in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to other posts: part one, part two, part three, part four, part five, part six, part eight, and part nine.
In the previous post, we took note of an error in the margin calculations by Escobari and Hoover. Though the effect on their calculations was small, the incorrect use of Válidos En Acta by Escobari and Hoover (among many others) generated controversy by making it appear that official vote totals did not correctly sum. Rather, these reflect clerical errors made by jurors at individual polling stations. We now pick up where we left off in post #5 when we noted that there was counting bias in the election. Here, we delve into the effects that bias had on the first results produced by Escobari and Hoover.
We begin with their “Difference Estimates,” almost exactly reproduced below. We attribute the discrepancies (indicated in red) to differences in assigning polling stations to precincts — a problem we identified with the first version of their 2019 paper, and that was likely not completely corrected.
Table 1
| CC | MAS | MAS-CC | ||||
|---|---|---|---|---|---|---|
| (1) | (2) | (3) | (4) | (5) | (6) | |
| Variable | ||||||
| SHUTDOWN | -8.286 (0.324) | 7.975 (0.343) | 16.26 (0.653) | 7.243 (0.437) | 6.762 (0.464) | 0.377 (0.194) |
| Constant | 36.86 (0.136) | 46.69 (0.134) | 9.830 (0.266) | 11.28 (0.162) | 11.36 (0.151) | 12.39 (0.063) |
| Fixed Effects[1] | ||||||
| Municipality | 129.6 | |||||
| Locality | 23.49 | |||||
| Precinct | 124.7 | |||||
| Observations | 34,529 | 34,529 | 34,529 | 34,529 | 34,529 | 34,529 |
| R2 | 0.017 | 0.016 | 0.017 | 0.640 | 0.740 | 0.958 |
Source: TSE and author’s calculations.
Notes: Dependent variables are percentages of Válidos En Acta (frequently missing or otherwise misreported on the tally sheets) and not of official valid votes. Standard errors in parentheses are robust. Differences from Escobari and Hoover noted in red.
[1] F-test statistics for fixed effects are not robust.
Note also that the analysis is not weighted by the number of voters at each station. For example, the constant for column 3 indicates the simple average margin for Morales across polling stations included in the TSE announcement was 9.83 percentage points — almost 2 percentage points above the official result at the time. Likewise, Escobari and Hoover’s result implies the simple average margin for Morales across all polling stations was 12.45 percentage points — again nearly 2 percentage points above the official result. This is because in actual elections, overall vote shares are not calculated the way Escobari and Hoover do. In actual elections, it is the vote totals, and not the average margins, that matter. Thus, polling stations with fewer votes have less impact on the final vote than do polling stations with more votes. Ignoring this makes Escobari and Hoover’s results difficult to put in the proper context.
Consider the two-precinct example of Table 2. In the rural precinct, there were 100 valid votes, which Morales won by 40 votes. In the urban precinct, Mesa won by 25 votes out of 250. The simple average margin is (40-10)/2 = 15 percentage points. But overall (taking both stations as a single group) Morales won by 40-25 = 15 votes out of 350, or only 4.3 percentage points.
Table 2
| Voters | Net Votes | Margin | |
|---|---|---|---|
| Rural | 100 | 40 | +40 |
| Urban | 250 | -25 | -10 |
| Combined | 350 | 15 | 4 |
We turn to column 3 of Table 1 above. In Table 3, we present the results of Escobari and Hoover alongside our replication and corrections to ease context. First, we note that our replication (column 2) exactly reproduces the published results (column 1). Second, we see that in employing the correct number of valid voters in the calculation, we have 22 more observations, missing only four polling stations that were annulled. Third, we note that once we weigh polling station data by the number of valid voters (column 4), the “Constant” falls by nearly 2 percentage points. This reflects putting the numbers in their proper context. Morales’s lead (based on the official numbers) at the polling stations included in the TSE announcement was 7.9 percent of the valid vote.
Likewise, in moving from column 3 to column 4, “SHUTDOWN” grows by 0.5, meaning that in giving too much importance to small polling stations, Escobari and Hoover wind up underestimating the increase in support when moving from stations included in the TSE announcement to those outstanding. Taken as a group, Morales’s margin on outstanding polling stations is 7.883+16.77 = 24.65 percentage points, and not 9.843+16.27 = 26.12.
Table 3
| As Published | Replication | Correct Voters | Weighted | |
|---|---|---|---|---|
| (1) | (2) | (3) | (4) | |
| Variable | ||||
| SHUTDOWN | 16.26 (0.653) | 16.26 (0.653) | 16.27 (0.653) | 16.77 (0.663) |
| Constant | 9.830 (0.266) | 9.830 (0.266) | 9.843 (0.266) | 7.883 (0.264) |
| Observations | 34,529 | 34,529 | 34,551 | 34,551 |
| R2 | 0.017 | 0.017 | 0.017 | 0.019 |
There are several ways of interpreting these results. One is to simply say that they measure the amount by which late polling stations more heavily favored Morales, and make no attribution as to the cause. This analysis is merely descriptive.
Another is to say that these results measure the bias in counting opposition polling stations disproportionately early. Perhaps rural stations that happen to favor Morales were simply more likely to be late and were therefore excluded from the TSE announcement — that is, nickels before dimes.
A third is to say that the announcement itself marked a division: the mere fact that a polling station was not included in the announcement explains the increase in support and that if all had been included, Morales would have won by only 7.9 percentage points. Because voting took place before the announcement, exclusion from the announcement should not by itself cause Morales’s support in those polling stations to rise. The implication is that the rise must be due to the addition of fraud, either committed after the announcement or in a deliberate delay in reporting polling station results already known to contain fraud. That is, in this interpretation SHUTDOWN would be a proxy for fraud.
In this figure, we are interested in the connection from fraud to margin, highlighted in red. Fraud is not something we can directly observe in the data, but one proposed mechanism is that the time required to implement fraud required delaying verification of those tally sheets until after the TSE announcement (hence whether or not it was included in the post-announcement “SHUTDOWN” group).
Note that the published result is inconsistent with respect to this interpretation. Escobari and Hoover argue in favor of the 7.9 percentage point counterfactual, but the constant in the model implies a projected margin of 9.8 percentage points — not statistically distinct from the election-determining 10 percentage point threshold. This reinforces our point that the use of weights in the analysis is important when one wishes to interpret the results.
This third explanation of the 16 percentage point difference as a measure of actual fraud is difficult to defend because of the confounding explanations in the second analysis. That is, in the model, SHUTDOWN captures everything impacting Morales’s margin that varies across the groups. There exists a whole apparatus of factors all complicating the interpretation of the 16 percentage point difference as fraud.
We are still only interested in the effect of fraud indicated in red. Of course, tally sheets wound up in the SHUTDOWN group for benign reasons as well as because of any putative malice. Consider those that transmitted late (late “ARRIVAL” to the electoral authorities) and those that transmitted their transcriptions but could not be verified in a timely manner. We tie both ARRIVAL and SHUTDOWN to rurality, but here “Rural” is a stand-in for a battery of various geographic or socioeconomic factors, any of which may have a different effect on each. Importantly, these same factors carry information about support for Morales, and so impact the observed margin. Finally, the number of voters at any given polling station helps to determine the order of ARRIVAL as smaller stations are able to complete their votes counts more rapidly.
The problem is that if we control for SHUTDOWN alone, that carries with it information about all the geographic factors. For example, given that a station is in the SHUTDOWN group, we can infer that it is more rural and therefore more heavily in favor of Morales. We can’t say if the 16 percentage point difference is all due to the “fraud” Escobari and Hoover seek to measure, or if it is all due to differences in geographic/socioeconomic factors. A more complex statistical model is required.
Of course, it is not easy to quantify — let alone identify — every confounding factor. We must bend somewhat to the reality of data availability. We must recognize that SHUTDOWN is a residual effect. Everything that accounts for the late increase in margin that is not expressly modeled is captured by SHUTDOWN. That includes both possible fraud and any overlooked nickels before dimes. A “statistically significant” SHUTDOWN coefficient doesn’t indicate the existence of fraud, specifically, unless we can adequately disentangle the effects.
To that point, an unexplained 16.77 percentage point difference would be politically worrisome in the absence of other information. Applied to the 16 percent of the election included in the SHUTDOWN group, this implies the exceedingly simple model fails to explain 2.7 percentage points of Morales’s final margin. We can see this directly in the estimated constant of Table 3, Column 4, which says that the non-SHUTDOWN group favored Morales by 7.9 percentage points. If the SHUTDOWN group is effectively identical, then the final election margin should have been close to 7.9 percentage points and not the official 10.56. Thus, the model leaves a politically significant unexplained residual that Escobari and Hoover interpret to be fraud. However, we know for a fact that the critical assumption that the SHUTDOWN group is identical is false. The model does not take into account important differences between the SHUTDOWN and non-SHUTDOWN groups. Nickels before dimes.
One way to cope with a dizzying array of possible differences is to divide polling stations into smaller groups. In doing so, we may hope to make assignments so that within each group these confounding factors are more or less constant — that we cannot easily distinguish one polling station from another except by their inclusion or exclusion from the TSE announcement.
As we will see in the next post, this is the reasoning behind columns 4–6 of Table 1.
