November 16, 2022
This is the fourth in a series of blog posts addressing a report by Diego Escobari and Gary Hoover covering the 2019 presidential election in Bolivia. Their conclusions do not hold up to scrutiny, as we observe in our report Nickels Before Dimes. Here, we expand upon various claims and conclusions that Escobari and Hoover make in their paper. Links to other posts: part one, part two, part three, part five, part six, part seven, part eight, and part nine.
In the previous post, we observed two biases affecting the preliminary count in Bolivia’s 2019 elections: a bias toward counting small (disproportionately rural) polling stations early, and a general bias against counting rural polling stations early. On balance, these biases meant late-counted polling stations were disproportionately rural and not representative of the stations that had been counted earlier.
After the polls closed on October 21, 2019 and jurors at each station tallied the votes, they submitted the results via cell phone app for inclusion in a preliminary count of the election results: the Transmisión de Resultados Electorales Preliminares (TREP). The TREP had no legal standing, and for good reason: it was designed only to give an early sense of the results while the careful and deliberate official count — the cómputo — took several days to complete.
The TREP relied on transcription of electronically submitted images of the actas (tally sheets), many of which were not correctly filled out by the citizens selected at random to manage the operation of their polling station. Indeed, as we noted in the previous post, training was in many cases insufficient.
Sometimes the transcription produced at the polling station conflicted with the transcription produced by workers at the Servicio de Registro Cívico (the Civil Service Registry, SERECI). If the transcript conflicted, the workers could flag the acta for further scrutiny. Many actas contained notes made at the polling stations — observaciones. Observaciones often contained corrections to individual party vote totals listed on the acta. Often these acknowledged utterly benign clerical mistakes, such as starting to record one party’s vote total in the space designated for another. These had potential to cause transcription conflicts as the decision whether to take into account observaciones was not uniformly applied. Observaciones and transcription conflicts delayed the verification of many actas.
As the TREP had zero legal standing, it was neither important to resolve all conflicts and questions resulting from the electronic submissions nor was it important to complete the count by including all polling stations. We find no record of a preliminary count in Bolivia ever concluding with 100 percent of the actas.
The importance of the TREP in the electoral process was, nevertheless, greatly inflated in 2019. Lest there be any confusion on this point, we note there has been no preliminary count at all in Bolivia since. (The replacement for the TREP was scrapped just days prior to the 2020 election.) The hard work of formally resolving conflicts and questions has always fallen to the electoral authorities of every department performing the official cómputo.
So what happened with the TREP that it became the focal point for allegations of fraud?
At 19:50 on election day, the TSE held a press conference to report that with 83.9 percent of polling stations included in the preliminary count, Morales led runner-up Carlos Mesa by 7.87 percent of the valid votes. If this result held in the completed official count, with Morales lacking either a majority of the valid votes or 40 percent with a 10 percentage point lead, then a second-round vote would be required. The public website showed the current results and made available records of the progression of the count: an Excel document had been produced every three minutes until 19:40:57, containing the list of verified actas and the corresponding unofficial polling-station level results. Further updates to the website were paused so the TSE could present a stable snapshot of the incomplete results. Importantly, the preliminary count progressed without interruption.
Combined with the electoral registry, the 19:40:57 snapshot offered a basis for which projections for the full count could be made. Without doubt, the partial results were not representative of the entire electorate. Nickels before dimes.
Fortunately, most of the outstanding voters were in precincts with at least partial results announced. Assuming the polling stations included were representative of their precincts, we can forecast the vote in about five-sixths of the remaining polling stations.
A crude projection based on the above observations yields a projected margin of about 10.4 percentage points. A more careful analysis attempting to account for the greater uncertainty from polling stations with no results suggests that Morales’s lead would grow to 10.2–10.5 percentage points. In Figure 1, we see forecasts of Morales’s vote margin based entirely on 2019 data available at the time of the TSE announcement. We have assumed that the order in which the polling stations were verified in the count is unchanged. That is, if we knew in advance the order of the count, the results announced by the TSE alone would suggest the trend we see in Figure 1.
Figure 1
All Forecasts Based on Partial Count Show Morales Winning in the First Round
Sources: OEP, TSE, and author’s calculations.
Another independent analyst forecast in real time an eventual 11-point lead. In any case, a review of the past snapshots revealed that Morales had come behind from an initial deficit to take a lead that had been growing almost without interruption for most of the preliminary count. Given the even more unrepresentative precincts that remained, there existed every reason to doubt that a second round would be required.
Although the TSE president warned that many rural, presumably Morales-heavy precincts still remained, the unrepresentative nature of the announced results was largely missed or ignored. An hour later, with the official count only just begun, Mesa claimed prematurely that he had moved on to a second round. The newspapers accepted the announcement with no skepticism.
Greatly complicating matters, at 19:30 — just 20 minutes before the original announcement — Ethical Hacking Consultants (EHC) detected a spike in activity from an internal IP address that was not on their audit list. Seemingly unaware of the cause, EHC sent a “maximum alert” to the TSE a few minutes before 20:00.
According to EHC, the resulting meeting was very tense, with TSE members making accusations of fraud and threatening to call the attorney general. At approximately 20:07, the TSE interrupted the process of verifying actas in the TREP.
The decision to stop the TREP was debatable. Certainly, the European Union Electoral Mission (EEM) had already argued the importance of a more complete preliminary count. The EEM was certainly justified in this regard, as the observed bias was significant enough to mislead. On the other hand, the cómputo was underway, so having multiple counts updating in real time could add to the confusion. As EHC put the security of the TREP in question, it was not obvious that the unofficial count was worth continuing. Again, the TREP of 2019 would be Bolivia’s last preliminary count, so even a partial count would be an improvement over future elections — but for the biased sample reported.
In any case, the opposition seized on the TREP interruption, claiming that the TSE had pulled the plug in order to buy time to manipulate the results. The EOM leaned on the TSE to restart the process. Approaching 1:00 a.m., Marco Antonio Pumari, president of the Comité Cívico Potosinista (Postosí Civic Committee, or Comcipo, an opposition organization) released a series of videos claiming to show fraudulent election materials. Though Pumari retracted those claims just a few hours later after meeting with an observer for the OAS, the videos continue to circulate on social media. Reminiscent of Florida in 2000, angry crowds shut down the official count in Potosí, setting fire to the department office.
The TSE relented the next day, restarting the count. When the public website updated at 18:29 with 94.9 percent of actas verified, the newly added polling stations were predictably favorable to Morales, from areas that had supported the failed 2016 referendum by more than 12 percentage points.[1] Morales now led Mesa in the TREP by 10.15 percentage points.
The OAS — in its role as election observer — publicly challenged the election results, irresponsibly legitimizing the fears of the opposition. In a press release, issued at 21:30 the day after the election, and before all the votes had even been counted, the EOM wrote:
The OAS Mission expresses its deep concern and surprise at the drastic and hard-to-explain change in the trend of the preliminary results revealed after the closing of the polls.
[… Today] the TSE presented data with an inexplicable change in trend that drastically modifies the fate of the election and generates a loss of confidence in the electoral process.
At an appropriate time, the Mission will issue a report with recommendations ahead of a second round.
[emphasis added]
The EOM expressed confidence that a count of legitimate ballots would result in an outcome other than the apparent first-round victory suggested by the preliminary, and wholly unofficial, results. The EOM was so confident in this declaration that, before the official results had been tabulated, it publicly stated that there would be a second round. No analysis was offered. To date, the EOM has yet to provide the reasoning behind their declarations that day. Instead, its press release solidified the loss of confidence in the process. By 21:15, opposition protesters had set fire to the TED offices in Sucre.
The EOM statement solidified a narrative of fraud — that the preliminary count had been reliable up to the interruption of the TREP, but the results that came after were inexplicable. Implicitly, only fraud could explain how these late-counted polling stations put Morales over the top for a first-round victory. The fact that the first-round victory has proven predictable puts the lie to the narrative, but the focus on the interruption of the TREP remains, even today.
Today, the OAS insists that its claim of an inexplicable change in trend is irrelevant — that its audit of the election turned up irregularities and security holes sufficient to doubt the official results. However, irregularities happen in every election of meaningful size. Many of the same allegedly fatal irregularities observed in 2019 happened in 2016, 2017, and 2020. The audit team offered zero examples of any official numbers resulting from efforts to manipulate the results. Of course, election security is vital to democracy, but over and over again the findings of the audit report rely on the alleged inexplicability of the post-interruption results to cast suspicion on otherwise ordinary clerical errors or lapses.
It is that same interruption that Escobari and Hoover choose to exploit in their papers, writing:
A shutdown of the official preliminary vote counting system Transmisión de Resultados Electorales Preliminares (TREP) gave us a rare opportunity that created a natural experiment to formally test for electoral fraud. We take advantage of the shutdown to separate polling stations between a control group that is less likely to be associated with fraud, and a treatment group where fraud is more likely to have occurred.
Escobari and Hoover do this by separating polling stations that were included in the TSE announcement from those excluded. However, this largely begs the question of fraud in the election. In a classical natural experiment, we know which observations are treated. For example, in a cross-border minimum wage study, we know which firms are subject to the imposition (or increase of) the minimum wage and when. Or, to take another, the Vietnam War draft lottery randomly sorted young men into those who served and those who did not, but in studying the effect of service on their lifetime earnings we start knowing already who did or did not serve. Here, not knowing which stations are treated makes it very hard to discern the effects of treatment. Escobari and Hoover merely proceed as if the late-reported stations have been definitively contaminated by fraud.
How do Escobari and Hoover know that the excluded polling stations are more likely to be associated with fraud? The groups are not otherwise similar. Explicitly, the groups are divided by whether SERECI workers had at the specified time completed work in reviewing the transcription/image pair sent from the polling station. In turn, this depended on how quickly jurors successfully transmitted complete results and images of the acta,[2] and whether there was any conflict between the transcription and image that required resolution. Voter eligibility, cellular access, and juror training and skill (including Spanish literacy and prompt access to electoral notaries) all affected the transmission time and therefore whether a polling station was included in the announcement. Plausibly, areas of low connectivity may be able to transmit the few numbers comprising the transcript but might struggle to submit the much larger image, increasing the time between transmission and verification. Conflicts that delay verification are more likely if, say, juror training and/or skill is low (thereby delaying transmission), but also may be a sign of overly hasty work (premature transmission leading to delay in the approval of the transcription). Nickels before dimes.
This all complicates Escovari and Hoover’s interpretation of their work as a “natural experiment.” Imagine a soccer match between Sweden and Japan, with all the players lined up on the sideline before the game for a pre-match ceremony. We begin announcing the roster, reading off the names and heights of each player starting on Japan’s side and moving on to those of Sweden. Halfway through Sweden’s roster, with the average height announced so far being 180.4 cm, a vuvuzela randomly blares, interrupting the announcement. The roster list then proceeds, and the height of the remaining players averages 185.2 cm. Did the vuvuzela cause the late-announced players to grow nearly 5 cm? The question would be absurd. It’s obvious that the nonrandom announcement of Japanese players before Swedish imposes a bias in the selection of players into groups before and after the vuvuzela. Nickels before dimes.
Similarly, Escobari and Hoover impose important selection biases when they draw the line at the TSE announcement. The “shutdown” does not offer a true opportunity for a natural experiment because the “treatment” group is neither plausibly similar to the “control” nor known to be treated in any ordinary sense. In the next post, we will begin to explore how differences between Escobari and Hoover’s groups (apart from possible fraud) complicate interpretation of their results.
Coda:
Escobari and Hoover cite Roe and Just (2009), saying “natural experiments typically have greater external validity than field experiments.” Even if we accept the 2019 election to offer a natural experiment and we agree with Roe and Just, this is a confusing aside. External validity means the experiment’s conclusions apply outside its confines. In this case, the goal is to estimate fraud in 2019. External validity seems irrelevant and Escobari and Hoover do not expand on its possible importance elsewhere in the paper.
[1] The referendum sought to lift presidential term limits and is therefore recognized as an imperfect proxy for support for the incumbent president, Morales. More precisely, the results of the referendum may be a proxy for all kinds of geographic and demographic factors which then help to explain support for Morales.
[2] The image requirement is irritatingly important. Once jurors submitted a transcript, the polling station was flagged as “transmitted.” Yet until an image of the acta is received, SERECI workers cannot proceed. That is, although verifications generally proceeded in the order in which transcripts were transmitted, any transcripts lacking an image to match would not be entered into the verification pool.
