May 27, 2020
Yet Another Analysis Fails to Support the OAS’ Delegitimization of Bolivian Elections
The Bolivian mission to the Organization of American States (OAS) promoted a new study purportedly proving that the results of the October election in Bolivia were fraudulent.1 This study, “The OAS Conclusions about the election integrity of the Bolivian election are correct” by John Newman, primarily expresses concern that in the preliminary count the late votes are distributed differently than the votes counted prior to the interruption in reporting. 2 Newman argues that the unexplained difference is necessary in order to account for Evo Morales’s victory. That is, if not for the unexplained increase in his votes late in the count, Morales’s margin would have fallen short of the10 percentage points necessary to avoid a runoff election.
Unfortunately, Newman’s statistical tests are uninformative. By conditioning samples on the early vote margins, the tests ought to come back positive ― with his split Newman creates a difference to be detected and a test should detect that difference. All negative results — failures to distinguish statistically between the distributions — likewise, are false negatives even in the absence of any fraud.
We suggest an alternative split of the data that suffers less of this conditioning problem while maintaining the spirit of the exercise. When we apply the tests on this alternative split, we expect to find that all results are negative and — following Newman’s approach — the election result is entirely explained.
Further, Newman’s choice of cutoff increases the relative likelihood of finding a false negative in cases where a positive result would lend counterfactual support to Morales over the runner-up, Carlos Mesa.
Finally, we note that Newman builds the final counterfactual based on municipality-level analysis. By ignoring intra-municipality variation, this necessarily overestimates the unexplained difference.
We therefore conclude that Newman’s study is fatally flawed and should be retracted.
There are a number of ways in which researchers have attacked the question of whether the post-interruption results in Bolivia’s October 2019 elections were predictable. For example, we projected directly the late votes using multiple imputations of tally sheets based on geographic indicators and the pre-interruption results; this led us to reject the idea that something inexplicable drove Morales’s final margin above the 10 percentage point threshold required for a first-round victory.3 That Morales was headed for a decisive victory appeared inevitable based on the early reporting.
Taking a different approach, Escobari and Hoover construct an implicit counterfactual using regression analysis. Seeking to quantify the degree to which the late vote is inexplicable, they find that incorporating full geographic information into their model leaves very little unexplained. In other words, once they take geography into full account, their model shows Morales’s victory was predictable.
John Newman takes a third approach. Newman first breaks down the data into three sets: geographical areas where all tally sheets were counted pre-interruption, areas where all tally sheets were counted after the interruption, and areas which were partially but not completely counted after the interruption. Newman accepts the results from the first and second groups; for the third group, two questions are posed. First, do the late tally sheets look different than the early ones? Second, if they do look different, how much of this difference is accounted for by a shift in the mix of geographies within the group? Newman argues that absent non-geographical differences, Morales’s margin would have fallen short of the threshold for a first-round victory.
As it happens, with a careful understanding of the first question, Newman’s second question becomes moot.
