The Hand Count Question:
Is Hand Counting Better than Machine Counting?
rev 8 200911251200

Mitch Trachtenberg,

Many election integrity advocates feel that hand counting all ballots at the precinct is the best way to ensure everyone's vote is properly counted.  Many registrars disagree, preferring machine counting. In an attempt to discover what it is about hand counting that is considered superior, please follow me in some thought experiments. 

Let's think about which of the following scenarios are best, and why.

Scenario A:

Ballots are removed from the ballot box and taken by a single election judge. If any observers are present, they may watch the judge closely enough to see the ballots, their votes, and the judge's tally sheets. There is, however, no requirement for observers.

This judge, who has been selected from a list of persons approved by all candidates, counts the votes and seals all ballots in a container so they are available in the event a recount is declared.  She then signs her counts and turns them over to the elections office.  These become the official count.

Scenario A2:

As in scenario A, but no observers are allowed. (It could be argued that this is the current state of machine counting.)

Scenario B:

Ballots are removed from the ballot box and taken by a pair of election judges as in scenario A.

These judges, who have been selected from a list of persons approved by all candidates, work as follows: the first judge sits at one desk and the second at another.  They sit back to back.  The first judge picks up each ballot in turn and reads the name of the selected candidate in the first race.  The second judge makes a tally mark on a tally sheet next to this candidate's name.  The process is repeated for each ballot, and then again for each race until the judges are done.  The judges both sign the counts and turn them over to the elections office.  These become the official count.

Scenario C:

As with Scenario B, two judges are involved.  This time, however, both judges look at each ballot and both judges maintain tally sheets.  The tally sheets are checked at the end of every twenty ballots to make sure they are the same, and the judges redo the previous twenty ballots if they have found a disagreement in their tallies.  Otherwise, the process is as in Scenario B.

Scenario D:

Three judges take the ballots.  One faces one direction and reads each ballot, each of the other two face in the opposite direction from the first judge, and each marks tallies on a separate tally sheet.   Otherwise, as in Scenario C.

Scenario E:

Three judges take the ballots.  One faces one direction and reads the name of the selected candidate, a second watches over their shoulder to ensure they read the correct name.  The third faces the other direction and marks votes on a tally sheet.  Otherwise, as in Scenario B.

Scenario F:

Four judges take the ballots.  One reads the name of the selected candidate, a second watches over their shoulder to ensure they read the correct name. The third and fourth judges each maintain tally sheets, which are checked for differences every twenty ballots, as in scenario C.

Which Do You Like?

If you are like me, the only scenarios that you find acceptable are Scenario C and Scenario F.  Scenario A2 is the worst -- who wants to depend on one person's unchecked report of what the ballots said?  Unfortunately, in the absence of dedicated observers willing to stay past the close of polls, Scenarios A and A2 are identical.

Scenario B doesn't improve things at all -- there are two people splitting the labor of the count, but neither can check that the other is doing their part accurately. 

Scenario C is better: the two judges act as checks on one another for all stages of the process.  Scenario C is acceptable, but because it requires both judges to look at each ballot and then mark their tallies, it is slow.

Scenarios D and E are both as bad as Scenarios A and B.  Each seems to have been developed to protect a security risk in the process, but each leaves an alternative security risk unprotected.  Each is like blocking one door on a cage with two doors -- whatever you were trying to keep in the cage can still get out.

Scenario F closes both doors, just as Scenario C closed both doors.  Scenario F requires more labor, but will get things accomplished at a faster rate.  Unlike Scenario C, Scenario F allows each judge to focus on one task (reading the ballot, monitoring the reader, marking a tally sheet).

It's the Redundancy, not the Human

If you also selected Scenario F and Scenario C, we agree.  The benefit of hand counting is not solely that human beings are involved, but the fact that there are people checking one another's work at each stage of the count.  The checks may just be for accuracy, or they may be to prevent fraud.  It doesn't really matter whether fraud is suspected: by checking for accuracy one has made fraud impossible without the cooperation of two of the judges.

Other improvements might be possible.  For example, rather than selecting as judges persons who are acceptable to both candidates, one could have each candidate provide one of the judges for each stage of the process.  For a proposition vote, the "Yes" campaign could provide one "caller" and one "tally-er" and the "No" campaign could provide one "caller" and one "tally-er."  Now there is less chance that two judges will collude.  They might still defect from their side in exchange for a large enough bribe, but if we are willing to assume they truly want a particular outcome, we may be willing to live with this risk.

Another improvement would be to use three judges at every stage in the process.  Now, a single defector will be challenged by two other judges, and can be easily detected.  The bribery problem now involves convincing two judges to defect, and working out a strategy between them by which they both call false outcomes on the same ballots.  Even if such a strategy could be implemented, the third judge could complain to some other authority, and the panel could be replaced.

At this point, I think we have narrowed in on the key points that make human counting so appealing:  multiple people involved in each stage of the process act as natural checks on one another.  Three or more people involved in each stage of the process not only act as checks on one another, but can self-repair any inadvertent errors and allow any one honest judge in a set of three to sound an alarm. 

Can Computers Count Ballots (if they're honest)?

My question now is: if these characteristics could be duplicated by a counting system that used multiple computers, how much of a change would that be to the security of the process?  For the sake of simplicity, let's just introduce one new scenario, Scenario G. 

Scenario G looks like this.  At the opening of a polling place, three or more persons confirm to their satisfaction that all ballot boxes are empty.  The ballot boxes are then placed in a location from which three people can observe them.  Ballots are handed out to voters, who fill them out.  Each voter then takes their ballot, wrapped in a privacy sleeve, to a standalone station where, if they wish, they can run the ballot through a device that will detect and report to them any undervoted or overvoted contests.  They may then return the ballot and start again, or continue from this station to a "weigh and deposit" station.  At this station, the ballot and privacy sleeve are weighed to confirm that the privacy sleeve contains only one ballot.  If the weight is satisfactory, the ballot box opens and the ballot can be slid from the privacy sleeve into the ballot box.

At the conclusion of the voting session, three election judges open the ballot boxes and thoroughly sort the ballots.  They then run the complete set of sorted ballots through two or more different devices provided by different vendors -- each such device photographs or scans the ballots and generates vote counts for all races.  The judges then compare the results.  If they match, the judges sign the results and this becomes the result reported to election headquarters.  It is phoned or faxed in, with the judges keeping the results to make sure those received at election headquarters match the results they've just seen.  The ballots are sealed and transported to storage, so they can be further counted in the event of any problem.

In my opinion, because Scenario G continues to rely on multiple independent "counters" coming up with matching results, it does not introduce any security issues that are not also present in scenario F (multiple humans at each stage). 

Scenario G does, however, introduce several practical benefits over Scenario F. 

But Do the Computers Improve Anything?

First, the machine counts can be quicker.  I don't think anyone would argue this point. 

Second, the act of photographing or scanning the ballot documents creates an independent set of copies of the vote information, so that even if the ballots are intercepted and altered at any point after leaving the precinct, there are two independent images of each ballot that will, at the least, raise suspicion that the original has been subject to tampering.  (One independent image might give 50/50 odds that the tampering was with the image; two images from two independent machines, both of which appear identical but neither of which matches the physical, stored ballot would suggest that the ballot had been altered after leaving the precinct.)  Third, by providing both images and per-ballot vote information, each machine makes it possible to compare intermediate results: are the images different from one machine type to another? are the images identical but the votes different? These comparisons can be done without access to the physical ballots.

The programming of the machines should, of course, be public.  With multiple machine types, this is not as critical as if a single machine type were used, but it still makes sense.  The more "eyes" on the programming, the easier it will be to eliminate any potential problems. 

Ideally, the machines would not need to be reprogrammed or provided with any election specific information from one election to the next. This would be easy as long as all ballots conformed to a set of rules about candidate and contest separation, and about the appearance of a votable area.

Although problems will be detected, it is still important that the machines operate subject to the closest possible scrutiny.  If there were a common path by which both machine types could somehow be led to produce identical but wrong results, that would be a serious problem, so it is important that the machines be physically protected and "tamper evident."  This situation, where both machines, despite their different origins and designs, were simultaneously hacked to produce identical but wrong results, seems to me to be similar to the situation in which multiple elections judges from different parties collude to throw an election.  That is, there is an irreducible risk whether the human route or the machine route is chosen.

We rely on machines, as well as humans, to defend our democracy.  The F-15 pilot is critical, but without the F-15, her abilities are limited.   In defending our elections, we can rely on machines as well.  We just need to know they are on our side. (Hat tip: I'm loosely paraphrasing Andrew Pollack.)

How Can We Keep the Computers Honest?

Let's discuss what process would be desirable in determining whether a particular machine is suitable for counting votes.

The system that has been used to date is called "certification."  Certification means that a panel of experts is willing to certify that a machine meets various security requirements.  The assumption is that if a particular combination of hardware and software is certified, it can be trusted for use in elections, whether or not it shows intermediate results.

Certification has one major disadvantage: it creates a "barrier to entry" for new machine manufacturers.  I've heard estimates that the certification process can take a million dollars.  The effect of this hurdle is limited competition -- who wants to be down a million dollars before even entering a competition against well-entrenched competitors?  If certification resulted in reliable equipment, however, perhaps it would be worth putting up with this disadvantage.

But the certification process, and then even the experts' California Top-to-Bottom review of voting machines, did not prevent continued use of Diebold's system. In my county, Humboldt, our local Election Transparency Project found Diebold's GEMS had dropped a batch of ballots from its count. We found this only the second time we scanned ballots.  California's Secretary of State then looked into Diebold's equipment in more detail, and found a variety of problems that should have, but had not, prevented certification.  Alarmingly, these problems affected the reliability of the system's audit logs.

So perhaps that "panel of experts" approach is not really delivering reliable systems.

Here are some suggestions for implementing security in a way that anyone can understand and confirm.

First, the software that runs on the machine should be available for inspection by any citizen.  More eyes, more discoveries.

Second, the machine should be able to run a series of arbitrary test ballots and produce results that agree both with a hand count and with a set of other machines already validated.

Third, the machines should generate images of every ballot. 

Ideally, these images should also be available to all citizens as a matter of course.  If this raises unanswered privacy concerns, a first step would be to restrict access to these images.  Perhaps only elections office staff plus representatives of each candidate might be allowed access. 

My opinion is that there is no real risk in letting any citizen view every cast ballot.  As with any real life situation, there will be exceptions. For example, there may be too few ballots of a particular type to avoid revealing who voted a ballot. In this and other situations, common sense will be required. However, allowing the withholding of a tiny subset of ballots is very different than withholding all ballots.

If a ballot contains identifying marks, current law provides that the identifiable ballot should be disqualified anyway.  Experts can (and do) devise scenarios in which a person might mark their ballot in clever ways, so that it would be hard to determine they've created an identifying mark.  (As a simple example, the person could fill the voting spot in such a way that a pattern of unfilled specks remain in specified locations.)  However, in California and many other states, an increasing number of ballots are cast through the mail.  This opens so many easier paths to vote-selling that it seems foolish to focus on closing the hypothetical risk of "secret identifying marks" when very real risks remain: voters, for example, can fill out and mail their absentee ballots in front of a witness who will pay them for their votes.

One frequently cited concern is the use of "pattern voting" as a means of selling votes if all ballots are exposed to public view. One way to mitigate this would be to withhold those parts of ballot images that included contests where the vote was for a minor candidate (one with fewer than, perhaps, 5% of the total votes in the race). This would result in withholding a relatively small part of ballot data from the public, but might make the released images unsuitable for a pattern voting scheme. Also, released ballot images could be split by column, with all columns after the first re-shuffled. There would be no way to link a subpattern in column 1 with a corresponding subpattern in any subsequent column.

Another concern might be that voter handwriting can be recognized when a vote is cast for a write-in candidate.  If this is deemed a serious concern, it would only require keeping back the relatively small subset of ballots with write-in votes; in fact, only the lines with the write-in would need to be redacted from such ballots.

If the machines generate images, and these images are made available prior to the sealing of ballot containers, it becomes very straightforward for the machine operator to check ballots at random to ensure that the images match the ballots.  It would be important that this check be performed on images that had already been written out to physical media, so that the physical media could be loaded into any observer's laptop to confirm the images against the physical ballots.

Once the machine has generated images, it is really fairly simple to automate the process of checking the images for votes.  Programs like the one I've developed can check the brightness or darkness of each votable area to determine whether it contains a mark.  More sophisticated checking can examine whether any marks exist outside the votable areas, making it possible to refer unusual ballots to humans for further checking.  (For example, a voter might mark one candidate by mistake, then mark the desired candidate, surround the desired candidate with a circle, and draw an "X" over the mistaken mark.  The few times this happens in any given election, the software can kick the ballot "upstairs" for human inspection.  It is easier to hand-check 50 ballots than 100,000.

Each machine could provide a vote report based on an analysis of the votable areas.  Alternatively, the machines could be restricted to generating images, with other machines running analysis software.  The critical point is that, once images have been written to physical media like a DVD, and once these images have been spot-checked to confirm they reflect the paper ballots, it then becomes impossible to tamper with one of the paper ballots without the image DVD "tattling."  If the images are made in public, either as the ballots enter the ballot box or just after the ballots are removed from the ballot box, then many security risks are eliminated.  My preference is that the imaging be done at the conclusion of the voting day, so that the ballots can be shuffled prior to imaging.  This eliminates any privacy risks that might exist if ballot images in any way reflected the sequence in which the ballots were inserted into the ballot box, and also reduces the time window during which imaging will be taking place.

Machine analysis of the images can contribute to objectivity in handling borderline cases.  What happens if vote area is half-filled?  If the spot is for Candidate A, then Candidate A will feel it should count, and other candidates will feel it should not.  If both candidates agree in advance on an average lightness or darkness that will count as a vote, disagreements like this can be resolved objectively.   In truly unusual cases, again, the ballots can be sent to a panel of humans for final decisions.  By being able to resolve the vast majority of border cases, the machine can reduce the workload on these judges, and allow them to devote more time where it is needed.


The critical desirable aspect of hand counting is not that human eyes are used, but that the hand counting process is done with more than one set of eyes on the ballots and the tallies.  This part of the hand count process can be carried over to machine counting.  Ballots imaged by two different machines immediately after being removed from their ballot box, combined with multiple independent machine counts on these images, can give the same assurance as hand counting that an  inaccurate count will be immediately obvious.  Because the imaging and/or counting process can proceed quickly, it is easier to implement at the precinct level than hand counting.  By counting ballots at the precinct, the risk of ballot tampering can be reduced.

For a far more technical discussion of some of these ideas, please see the paper by Joseph A. Calandrino, J. Alex Halderman, and Edward W. Felten, in the Proceedings of the 2007 USENIX/ACCURATE Voting Technology Workshop, August 2007.