Our Votes are Too Important...

by Mitch Trachtenberg, 12/7/2008

Humboldt County, California is fortunate to have a County Clerk and Registrar of Voters, Carolyn Crnich, who is herself an election transparency advocate. There were probably groups in just about every jurisdiction in the country who would have liked to do an independent vote count, but in Humboldt County we were in the pleasant situation of having Humboldt's registrar not only neutral, but actively on our side. Through her election advisory committee and the Humboldt County Election Transparency Project, Crnich was able to provide access to the ballots -- something that in some other areas seems to require a lawsuit.

I met Crnich earlier this year, and remember excitedly telling people "she gets it!" Years ago, I'd spent some time trying to convince the previous elections manager of the dangers of DRE touch screens, but I could tell I was getting nowhere. This time, the person in charge of the vote counting was as indignant about secret counts and missing paper trails as I was!

Humboldt uses equipment produced by Diebold, a name which became so notorious in election circles that the company has changed its name to Premier. Our county hadn't switched to DRE touch screens. We were still using paper ballots and Diebold's optical scanners. Because Humboldt's system preserves paper ballots, it seemed to me highly unlikely that there'd be any discrepancies between our independent count and the count generated by Diebold. "They wouldn't dare," is what I'd been thinking. Still, you don't KNOW things are OK until you've done your independent count.

I figured that by demonstrating how paper ballots could be independently counted, we'd be providing a useful argument for those who were trying to prevent the touch screens from becoming more popular. I also thought we'd be verifying the vendor counts, and putting the vendors on notice that independent counts could be done, limiting any future impulse towards shenanigans.

Others in the community wanted complete manual counts. I've seen what manual counts look like, and they have their own problems. The more people that are in a room with the ballots, the more distractions. Every time ballots are moved, that's a potential opportunity for misplacement or worse.
The time commitment to get a solid manual count is astonishing. I watched the start of the mandated 1% count, and was impressed by the commitment of the citizens doing the counting. Panels of five. One person picks up a ballot and calls out the name of the winning candidate in the Presidential race. Person one hands the ballot to person two, who looks at it and puts it in a stack. Persons three and four both mark tally sheets with a pencil. Person five watches. After a bunch of ballots go by, person one calls "tally" and persons three and four both read their count. Repeat, seemingly forever. Then start again on the next race. It works, but it's very slow; unrealistically slow for full timely recounts

Kevin Collins had a better idea.

Collins, a commercial fisherman, who, together with Crnich, Tom Pinto of the D.A.s office, Green Party Presidential Candidate David Cobb, and Parke Bostrom, had initiated the transparency project, wanted to scan the ballots so people could do recounts on their own.

I'm a software developer, and I felt it was important to provide open source software that could count the votes on the scanned images. Otherwise, I thought, the scanned images might sit there with nobody really bothering to count their votes. So I wrote a program to count the votes.

To keep things as open as possible, I wrote the counting program completely separate from the software we used to do the scanning. We used Debian Linux, a completely open platform, to do the scanning. Linux uses an open scanning protocol called SANE (short for Scanner Access Now Easy). SANE didn't work with the serial number imprinter on our Fujitsu scanner, so I contacted M. Allan Noah in Virginia -- in open source lingo he's the author and "maintainer" of the SANE Fujitsu backend. He was incredible, getting us some initial changes the same day and guiding us in getting him enough information that, even though he doesn't have the very expensive scanner model Humboldt had purchased, he was able to decipher the USB dialog that took place between Windows and the scanner and reproduce it within SANE. I wrote a very simple script to allow our volunteers to drive the SANE program and get new serial numbers every time they started a scan batch.

We'd heard concerns that stray marks on the ballots might lead to allegations that we were facilitating vote buying by creating the ability for a person to prove which ballot was theirs. Kevin Collins, along with many others, has argued extensively that if someone wants to buy votes in California, they'd do it the easy way: by taking advantage of vote-by-mail, offering money to voters who let the high-bidder fill out their vote-by-mail ballots. In any event, I wrote software to spot stray marks on the ballot images and "scrub" the images clean. Then we all realized no one would want to see "scrubbed" ballots, they'd want to see what we originally scanned. So with a bit of a wink, wink, nudge, nudge from some high state officials, we figured we'd release the ballot images and see if anyone complained. And, under California law, any ballot that can be traced to an individual is automatically invalid.

I also wanted to try to replicate a manual count as closely as possible, so I set up the counting program to display each image as it counted it, optionally overlaying the ballot image with boxes around each vote oval, and a red box and label around each vote oval that, in the opinion of the program, had a vote contained within. I added a delay slider, so we could project each image in a public hall for a certain number of seconds so that people could count along with the program, or note their disagreements. Finally, I had the program write out each vote to a file that could be imported into a spreadsheet. This file shows the image number, information that can identify the precinct, the contest, the candidate, and some debugging information that can be used to locate ballots where the program's output should be checked.

Last Sunday [11/30/09], after the election department had wrapped up its certification work, the last ballots were made available to our project for scanning. Carolyn would unseal a bag of ballots and hand them to me, I'd feed the ballots into the scanner, and Carolyn would reseal the bag. With the stack of bags dwindling, Carolyn and I both began making nervous jokes about what would happen if we didn't have as many ballots as the elections office had certified. We looked at the bags and made estimates about how many remained.

When our numbers were approaching those of the official count, we breathed a sigh of relief. And then I joked that we weren't out of the woods yet, maybe the Project would have too many ballots. We both laughed. And then that's what happened -- we had 433 images too many.

We were rapidly able to determine that one of the images was a single-sided ender card that had been scanned by mistake... that left us with 432 extra images, or 216 extra ballots. I think both Carolyn and I assumed at that point that somehow we'd double-scanned a bag or two, though it was hard to imagine how that had happened. Every ballot side that went through the scanner got a serial number imprinted upon it, so that the scanned image could be tracked back to the individual ballot side if questions arose. How do you fail to notice that you're feeding the scanner a ballot that already has a serial number... 216 times?

I started a run of the counting software. The software can be run simultaneously on multiple computers, so that each computer only needs to run part of the vote. I wasn't yet confident that the merge would go smoothly and I didn't think this was a good time to experiment, so I just ran the run on one machine... it took 12 hours. At the end, it was clear that precinct 1E-45's vote count for both Obama and McCain was substantially off from the official results. We'd kept careful logs of the scanning process and went through them and, sure enough, there was a whole bag of ballots from precinct 1E45 that we'd scanned for the Project, but hadn't made the official count.

Here, I believe, the elections office began trying to figure out what went wrong. I think everyone in the office probably assumed that, if the Transparency Project hadn't made a mistake, they must have. No one thought we were both right. I wasn't there as the office went through their own logs, but I'm sure it was tense... the election results had already been certified before the Board of Supervisors. If I were Carolyn, I'd have been thinking "no good deed goes unpunished."

But then it turned out that the elections office HAD run the "missing" bag through the Diebold system, but the Diebold system had not included that bag in the detailed, final, official canvass results that it spits out. Calls to Diebold and the Secretary of State's office confirmed that we'd found a problem that had been in the Diebold counting software for the last four years. Diebold had told -- rather casually, it seems to me -- county election officials about the problem, but they never notified the office of the California Secretary of State. And Humboldt's previous elections manager never passed along news of the problem to his successor. It looks like Diebold released subsequent versions of their software without fixing the problem. It looks like some of those versions may have been used in states like Maryland and (you guessed it, didn't you) Florida. Hey, it's always Florida.

What about going forward? Well, we've demonstrated that independent counts are practical, and we've demonstrated that, without independent counts, unexpected and potentially undetectable errors can creep into official results. I think it's time for every jurisdiction in the country to eliminate use of DRE machines. I think we need to stop using secret, proprietary software to do our vote counting.

Some registrars may say that independent counting will only work in small counties like Humboldt. I disagree. I believe that the approach we've used can be scaled up to any size. Larger counties will have more ballots, but they'll also have more volunteers to scan, or more of a budget to pay people to scan. If you've got a million registered voters, just split your ballots into ten groups and buy ten scanners. If you've got five million, just split your ballots into fifty groups.

I'm hopeful that some inspired group may develop faster and less expensive ballot scanning hardware -- I have ideas about how to do it, but I'm all thumbs.

This was all made possible by a registrar, Carolyn Crnich, who was willing to work with the concerned citizens in her community, rather than dismiss them. I think Carolyn Crnich is now a national hero. And all California citizens are very fortunate that our Secretary of State, Debra Bowen, stood up to the companies that wanted to foist DRE touch screen voting on us. Touch screens don't solve any problem -- their use to help those with disabilities was, in my opinion, a phony argument from the start. Touch screens must go. In counties and states that have switched, paper ballots must return. Those with disabilities can be helped to vote with systems that assist them in filling out paper ballots.

Our votes are too important to be counted by secret software running on black-box vendor machines.

Our Votes Are Too Important... by Mitch Trachtenberg is licensed under a Creative Commons Attribution-No Derivative Works 3.0 United States License.