Instant Runoff Voting in San Francisco

San Francisco’s first Instant Runoff Voting (IRV) election occurred in November 2004. Prior to that, San Francisco used Top-Two Runoff (TTR), a system in which a Plurality Voting election is held, and if no candidate receives a majority of the votes, a later “runoff” is held between the two top vote getters. Here we attempt to summarize the effect IRV has had in SF.

First, it should be noted that IRV is also known by the somewhat ambiguous alias, “Alternative Vote” (AV), primarily in the UK. Plurality Voting is also known as “First Past the Post” (FPTP).

Why IRV?

The hype

IRV was promoted largely on the claims (e.g. by FairVote members such as Steven Hill) that it would:
  1. Save money, by eliminating the expense of holding a second (runoff) election
  2. Increase voter turnout, by eliminating supposedly low-turnout runoff elections

The reality

How have these claims panned out?
  1. From this page on IRV costs, election integrity activist Joyce McCloy explains:

    Maybe IRV saves money, but there isn’t a solid cost savings analysis using San Francisco’s actual election department’s net annual expenditures. From San Francisco’s Budget Reports:

    2000-2001 Actual 9,024,000
    2001-2002 Actual 13,872,000 includes the cost of $1,322,849 for a runoff election & $150,000 due to litigation costs
    2002-2003 Actual 8,610,553
    2003-2004 Actual 15,204,781
    2004-2005 Actual 10,400,868
    2005-2006 Actual 11,930,228
    2006-2007 Actual 10,062,052 (budget) 9,126,318
    2007-2008 Actual 14,839,686 (budget) 19,809,917

    Warren D. Smith, a Princeton math Ph.D. who has researched election issues for over a decade, also makes the following point:

    Yes, one round is cheaper and easier than two, but with IRV, that one round is more complicated and it cannot be done on ordinary “dumb totalizing” voting machines, whereas both rounds in delayed runoff can be done with such machines; and IRV is non-additive (no such thing as “precinct subtotals”) and non-monotonic; and the second round in delayed runoff often does not happen. (Top-two runoff also is non-monotonic, but each of its two rounds, in isolation, of course is monotonic.) In view of those facts, it is not at all clear to us that IRV actually saves money. And in any event, the money spent on elections is negligible compared to other government expenditures, so it is more important to get quality in elections, than to save money. Thus “saving” money would be a false economy that surely would actually cost more in bad government than it saved in election expenditures. For example (2006), a pro-IRV group was recently arguing that Oakland California should switch to IRV because each runoff election under the old delayed-runoff scheme cost Oakland “hundreds of thousands of dollars,” which was their way of saying $200,000. However, they did not mention that Oakland’s annual budget is over $1 billion so that the “cost savings” they were lobbying for was of order 0.02% fractionally. Surely there are superior ways to save Oakland’s money! Also they did not mention that it cost (neighboring, comparable size) San Francisco $1,600,000 to upgrade its voting machines to run IRV two years before. So the payback time required to justify this cost “savings,” as you can see, would be very large, perhaps 30-40 years assuming elections every 2 years and runoffs required half the time. Quite probably Oakland would be re-replacing its machines before that time, in which case the costs never would be repaid.

  2. Warren Smith created this analysis of San Francisco’s official election turnout data. It shows that, overall, election turnout has actually decreased, albeit by a statistically insignificant amount.
Moreover, it is important to note that these expected effects, even if they had panned out, would have had nothing directly to do with adopting IRV. They are both an expected result of eliminating runoffs. And that can be done without adopting IRV. IRV proponents would point out that this would present a problem that a candidate could then win without a “majority of the vote”. However, as we have noted, IRV can fail to elect a true majority winner.
 
For instance, in the November 2, 2010 election for District 10 supervisor, Malia Cohen won in the 20th round, with 4321 votes from 20550 ballots cast. That’s a mere 21% of the vote. And IRV can even elect candidate X, even when candidate Y is preferred to X by a sizable majority of the voters, and got more first-place rankings than X.

But isn’t it more democratic?

Bayesian Regret

The Center for Election Science supports a utilitarian (maximal group welfare) economic model (which we believe is the only logically tenable ethical framework). Here is a simple graph of Bayesian Regret figures which express the welfare-increasing effect for a host of voting systems, as a function of the amount of tactical voting.
 
Bayesian regret black and white
 
This graph excludes TTR, so we also cite some additional figures which include TTR:
 
Voting Method Regret (50-50) Regret (100% honest) #Agreements with (true-utility-based)
Condorcet Winner (when CW exists)
SociallyBest 0 0 12653
Range 0.16329 0.04802 11796
Approval 0.215 0.18983 10997
Borda 0.28234 0.093768 10821
CondorcetLR 0.35602 0.13781 10342
Top2Runoff 0.49909 0.23604 8823
IRV 0.50115 0.21684 8387
Top3IRV 0.53124 0.23683 7842
Plurality 0.64371 0.33137 6357
RandomBallot 0.72014 0.72305 5185
RandomWinner 1 1 2930
SociallyWorst 2.0024 2.0098 6
 

What we can see here is that, from a pure democratic-ness standpoint, IRV is a little better than Plurality Voting. And TTR is approximately equal in quality to IRV. However, IRV has a number of concerning side effects whose practical importance must be also taken into account.

Ballot spoilage

IRV typically results in about seven times as many spoiled ballots as Plurality. This is an empirical indication that, despite the “easy as 1-2-3″ claims of its proponents, IRV is in fact more complicated to voters. And this effect has been shown to correlate to demographics. Voters in poorer and more minority districts are more likely to spoil their ballots.
 
* We note that Score Voting (aka “Range Voting”), the simplest form of which is Approval Voting, both result in fewer spoiled ballots.

Precinct summability

IRV cannot be counted in precincts, because it is not “additive”. This is explained by a Princeton math Ph.D. here. A more layman-friendly description of the implication of this fact comes from the San Francisco city elections page:

“Due to the requirement that all ballots must be centrally tallied in City Hall and not at the polling places, the Department of Elections has not set a date for releasing any preliminary results using the ranked-choice voting method.”

 

This appeared on their website for several weeks after their 2008 RCV elections. The need to transport votes to a central location for tabulation is of great concern to election integrity activists. It means greater susceptibility to fraud.

Risk of ties/near-ties

IRV increases the probability of tie/near-tie scenarios (i.e. election disputes and recounts are more probable).

Complexity

The preceding points speak to the practical complexity increase introduced by IRV. But let’s get a better picture of the issue, by looking at a typical IRV voter’s take. I am a software engineer, and in early 2011, I had the following instant message conversation with a fellow co-worker of mine. This co-worker is also a professional software engineer, and has voted in San Francisco’s IRV elections for years.

me
hey man.
can you do me a favor?
 
coworker
sure
what’s up
 
me
okay, i need to use you as a guinea pig for a voting op-ed i’m writing.
it will just take a minute.
 
coworker
k
 
me
don’t look up anything.
i want to see how much a typical sf voter knows about our voting system.
so you know how we rank our choices on the ballot?
so like, X>Y>Z means the voter likes X first, then Y, then Z.
make sense?
 
coworker
yes
 
me
okay, so look at this list and tell me who wins.
% of voters – their ranking
35% W > Y > Z > X
17% X > Y > Z > W
32% Y > Z > X > W
16% Z > X > Y > W
 
coworker
well… i could do math here right?
do you want me to or not?
 
me
yes.
tell me who wins.
without looking anything up.
 
coworker
looks like Y
 
me
why do you think it’s Y?
how did you arrive at that answer?
 
coworker
because in the first row it’s next to the winning W, and in between row#2, and #3, it’s pretty high up
basically according to weighted average it’d roughly be a leader
 
me
nope.
the winner is X.

Conclusion

If we take both performance (Bayesian Regret) and practical cost and complexity into account, we find that TTR is better than IRV, and IRV is better than Plurality Voting. This means that San Francisco’s switch from TTR to IRV was probably for the worse. Whereas it would be beneficial for the UK to switch from Plurality to IRV.
 
That being said, there are much better and simpler reforms. Score Voting and Approval Voting are both simpler and much better than IRV.
 
For multi-winner elections, Reweighted Range Voting and Asset Voting are two proportional methods which have important advantages over the Single Transferable Vote (STV) and various other PR systems.
email