The Academy Awards: World’s Greatest Democracy?
[Topic: Reweighted Score Voting in Practice]
by Clay Shentrup
On February 24, 2013, the Academy of Motion Picture Arts and Sciences will hold their 85th annual Academy Awards ceremony. While tens of millions of viewers will tune in to celebrate the artistic brilliance and technical wizardry of the film industry, few will ever realize they’ve just witnessed the state of the art in democracy.
Plurality Voting Wins Worst Picture
The Academy is composed of over 6000 motion picture professionals who vote to determine the nominees for each category, and ultimately elect the winner. The stakes are high. A Best Picture nominee gets an average box office boost of $6.9 million, according to a 2001 analysis by Randy Nelson, professor of economics and finance at Colby College. If it goes on to win the Oscar, the film can expect a boost of $18.1 million.
With all those viewers and all that money on the line, you might expect the the Academy to have developed a sophisticated process for ensuring a good outcome. But prior to 2010, the best “technology” the Academy had to offer was Plurality Voting, aka First-past-the-post. This is the “vote for one” system used throughout the United States to elect single-winner offices like mayor or governor. Plurality Voting, while simple and familiar, can lead to terrible results whenever there are more than two options to choose from. For example, consider an audience with the following preferences (where “>” means “is preferred to).
35% 2001 > Apocalypse Now > Full Metal Jacket
33% Full Metal Jacket > Apocalypse Now > 2001
32% Apocalypse Now > Full Metal Jacket > 2001
A whopping 65% majority of the voters prefer either of the classic war movies to Stanley Kubrick’s abstract sci-fi piece, 2001: A Space Odyssey. But if we merely allow the judges to cast a single vote for their favorite, then 2001 wins with 35% of the vote. This is called the “vote splitting” or “spoiler” effect. Politically inclined folks know it well as a chaotic force that often produces undemocratic election outcomes. For example, many believe that Ross Perot was a spoiler who caused Bill Clinton to defeat George Bush Sr., or that Ralph Nader siphoned away enough votes to cause Gore to lose to George Bush Jr.
The spoiler effect likely accounts for some of the noteworthy upsets throughout Oscar history. Steve Brams, an NYU political science professor who promotes an alternative voting system known as Approval Voting, believes the spoiler effect was at work in 1950. That year, both Bette Davis and Anne Baxter received Best Actress nominations for All About Eve. When Judy Holliday became the surprise winner in Born Yesterday, it was widely believed that Davis and Baxter had split the vote.
Mathematicians have known for centuries that there are other ways to vote. In 2010, in an apparent effort to reduce the incidence of inexplicable outcomes that have drawn public outcry in the past, the Academy began using Instant Runoff Voting (IRV) for its Best Picture award. IRV, which has been used in Australia since the early 1900s, is a process where voters rank the candidates in order of preference; the system then sequentially eliminates the option with the fewest first-place votes, until a candidate has a majority of the remaining votes.
In our example above, IRV would start by eliminating Apocalypse Now. Those voters would then have their support transferred to their second choice, Full Metal Jacket. Full Metal Jacket would then win with 65% of the vote. This is an improvement, to be sure. But as statistician Nate Silver pointed out in a 2011 article in the New York Times, IRV is no panacea. Take a second look at our hypothetical example. While a 65% majority of the judges favor Full Metal Jacket to 2001, an even greater 67% majority prefer Apocalypse Now to Full Metal Jacket.
Moreover, the first row is in the perverse position of being “punished” for ranking their favorite movie in first place. If just a handful of them were to insincerely rank Apocalypse first, then it would win instead, giving them their second favorite instead of their least favorite. Election theorists describe this as a failure of a fundamental principle called the Favorite Betrayal Criterion (FBC). In short, a voting system should never give a voter a worse outcome for giving his full support to his sincere favorite option. Failure of the FBC not only makes for bizarre outcomes, it also encourages tactical game playing.
How then does the IRV algorithm draw the paradoxical conclusion that the voters favor Full Metal Jacket? The answer lies in the fact that IRV considers the rankings like the layers of an onion, only seeing one layer at a time. IRV eliminates Apocalypse because it is ranked first by a mere 32% of voters. But it does not even consider that Apocalypse is everyone else’s second choice—whereas both other movies are the least favorite of a significant percentage of the judges. IRV reduces the vote splitting problem, but doesn’t completely eliminate it.
This may explain why Ben Affleck’s critically acclaimed Argo failed to receive so much as a nomination for Best Director, despite the fact that Affleck won the best director category in both the Golden Globe awards and the Broadcast Film Critics’ Association cinema awards for that film. The nominees were picked using Single Transferable Vote (STV), a sequential multi-winner version of IRV, which unfortunately retains IRV’s vulnerability to the vote splitting pathology previously outlined. An article in the Guardian summarized it like this:
some are suggesting that Affleck came a cropper of the preferential voting system of the nomination process which rewards small but passionate groups of supporters – those for Michael Haneke [Armour] or Behn Zeitlin [Beasts of the Southern Wild], for example. Furthermore, voters may have felt they had to choose between Affleck’s CIA thriller and Kathryn Bigelow’s [Zero Dark Thirty] which may have had the effect of cancelling each other out. The result was an accidental snub rather than an intentional diss.
And the Nominee Is: Score Voting
But in selecting this year’s nominees for Best Visual Effects, the Academy has introduced an innovative scheme that may vanquish this mathematical quandary once and for all. The system, called Score Voting, is a simple and intuitive idea that’s been ubiquitous on the internet for years. Also known as Range Voting, it powers popular customer review sites like Yelp, Amazon.com, and appropriately enough, the Internet Movie Database. Did you love last night’s meal at The Palm? Give it five stars. Were you riveted by Argo? Give it 10 stars. While the range of allowed scores can be tailored to fit the purpose, the concept is the same.
Despite being radically simpler than ranked schemes like STV, Score Voting cleverly evades their pitfalls. It simultaneously registers a voter’s support for his favorite as well as all other options, precluding the kind of vote splitting exhibited in our hypothetical scenario above—that may have also bitten Argo. Score Voting also has the rare property that it satisfies the Favorite Betrayal Criterion: a voter can never get a worse result by giving the fullest support to his sincere favorite. If he doesn’t think that favorite has a chance, he’s free to support any number of more viable options. Thus, while Score Voting is not completely immune from tactical behavior (mathematicians long ago established that no deterministic voting system is), it manages to stave off the brunt of it.
There was a time when the election theory community mistakenly held quite the opposite view. While Score Voting was widely used in uncontentious polls, theorists presumed that its propensity for exaggeration made Score Voting unsuitable for use in heavily contested elections. But then along came Warren D. Smith, a Princeton math PhD whose mathematical insights turned centuries of prior work in the field on its head.
In the book Gaming the Vote, author William Poundstone describes how in 2000, Smith conducted an extensive suite of computerized election simulations aimed at determining how well various systems would behave in a variety of electoral circumstances. Smith explored what might happen if, for instance, 50% of voters behaved honestly while the other half gamed the system. Or if 20 candidates ran instead of three. To Smith’s surprise, Score Voting outperformed the classical (typically rank-based) schemes championed by his predecessors, regardless of how he tuned these “knobs”. Smith’s subsequent mathematical theorems and analysis drilled down into the inner workings of these systems, helping to shed light on exactly how Score Voting was able to counter-intuitively accomplish this feat.
Reweighted Score Voting Makes Its Debut
But things really get interesting when we look at the special “reweighted” variant of Score Voting used to select the five nominees for Best Visual Effects. First imagine what would happen if the Academy merely nominated the five highest rated movies. While that would produce a set of broadly appealing nominees, there would be little representation for less mainstream works. An obscure but brilliant film that might elicit passionate support from 20% of the judges would stand little chance of making it onto the list of nominees.
Consider a simplified case in which we must pick merely two nominees, using our previous hypothetical example.
35% 2001 = 5, Apocalypse Now = 0, Full Metal Jacket = 0
33% Full Metal Jacket = 5, Apocalypse Now = 4, 2001 = 0
32% Apocalypse Now = 5, Full Metal Jacket = 3, 2001 = 0
If we merely nominate the two movies with the best total score, we get:
2001 = 0.35 * 5 = 1.75
Full Metal Jacket = 0.33 * 5 + 0.32 * 3 = 2.61
Apocalypse Now = 0.33 * 4 + 0.32 * 5 = 2.92
Two highly popular war movies take both nominations. While this gives us confidence that our nomination slate includes the most popular movie, it provides for little diversity. Reweighted Score Voting (RSV) allows for diversity of viewpoints by “turning down the volume” on voters who’ve already had their say. Winners are chosen one round at a time. In every round, a voter’s ballot is divided by the amount of support he’s given to previous winners. Let’s see how it plays out with our previous example.
With RSV, we weight each ballot by 1 + p/5, where p is the number of points given to previous winners. In the first round, there haven’t yet been any previous winners. So p is 0 for all voters, and thus “1 + p/5” simplifies to 1—meaning that the first round results for RSV are always the same as with ordinary Score Voting. The only difference is we’re only selecting one winner at a time. Apocalypse Now takes the first round with 2.92. Since Apocalypse is the only winner so far, p becomes whatever score each faction gave to Apocalypse. So in the next round we have:
35% weighting = 1 + 0/5 = 1; effective ballot: 2001 = 5
33% weighting = 1 + 4/5 = 9/5; effective ballot: Full Metal Jacket = 25/9 = 2.78
32% weighting = 1 + 5/5 = 2; effective ballot: Full Metal Jacket = 4/2 = 2
2001 = 0.35 * 5 = 1.75
Full Metal Jacket = 0.33 * 2.78 + 0.32 * 2 = 1.5574
Just like with the STV system recently adopted to select Oscar nominees, RSV produces diversity. But unlike IRV/STV, it eliminates the vote splitting pathology, choosing the more broadly appealing of the two war movies. Had RSV been used instead of STV, Argo might not have been inadvertently snubbed.
Best Voting Method Goes to the Academy
The origins of RSV are far from the realm of entertainment. Warren Smith proposed the scheme in a 2004 essay which presented it as a means to ensure fair political representation. (He later discovered that it had been previously proposed in 1890, by a Danish statistician.) To understand the problem from a political perspective, consider that something like 10% of Americans identify with the Green Party, as well as the Libertarian Party. Given this, one could reasonably demand that 20% of the seats in Congress to be allocated to members of these two minor parties.
But due to our vote-for-one Plurality Voting system, these groups have zero representation. Of the 435 seats in Congress, 430 are occupied by Democrats and Republicans. Two are occupied by independents. Three are currently vacant. That’s right—our voting system means that there are more seats empty than are held by people outside the two-party system.
The use of Reweighted Score Voting means that the Academy Awards has a more advanced democracy than any government on the planet. That will be all the more so if, as we expect, RSV proves a success for the Best Visual Effects award, and subsequently supplants STV for picking the nominees across all categories. But while RSV may produce a great set of nominees, the Academy will still run into trouble if it continues to rely on the highly defective Plurality Voting method to pick the final winner. Much better would be a final Score Voting election from just that pool of nominees. We wish the Academy much success in their pursuit of a better voting system.
Please consider donating to support our work. [Donate Here]