By Kevin Standlee: In light of the revelation that modeling of E Pluribus Hugo does not result in quite the “magic bullet” that some may have hoped for, we may need to consider other changes to the Hugo Award voting system to deal with bad actors deliberately setting themselves against the wishes of the majority of the voting members of the World Science Fiction Society. Over the past few weeks, I have written up descriptions of three different proposals that attempt to deal with bad actors in different ways, and to make it more likely that the results of the Hugo Awards represent the wishes of the majority of the participating members, without the members having to resort to the 16-ton anvil that is No Award.
All of the proposals below are compatible with either of the proposals up for ratification this year (EPH and 4/6). They are not necessarily compatible with each other. None of them require a Very Strong Administrator picking and choosing individual members’ ballots or disqualifying individual finalists on what I call “ideological” grounds. All of them aim to give the majority of the members a strong voice in picking the Hugo Awards.
In this overview, try not to get too deeply bogged down in specific details. For example, all three proposal refer to the “Top 15.” This is a reference to the “long list” currently defined in the WSFS constitution thusly:
3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, . . . places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes.
Therefore, the “top 15” can be defined in different ways, and there’s a reasonable argument to be made for many of them. Don’t get too tangled up in specifics. These proposals have general principles, and if you’re agreeable to the general idea of any of them, then we can discuss the specific values for some of the blank spots in them.
In the list below, the links lead to my LiveJournal where each proposal is listed in more detail.
3-Stage Voting (3SV) adds a new round of voting to the Hugo Award process, called “semi-finals,” between the existing nominating ballot and the existing final ballot. In 3SV, the top 15 nominees (including any ties, and see the warning about specifics above) are listed in a way that doesn’t show how many nominations they received. The members (supporting and attending) of the current Worldcon (not the previous and following Worldcons) are presented with this list, with a question on each of the fifteen semi-finalists in each category: “Is this work worthy of being on the Final Hugo Award Ballot?” with the choices being YES, NO, and ABSTAIN.
If a sufficient quorum (which is why counting explicit abstentions is important) votes, and if a sufficient number vote NO, that semi-finalist is disqualified from further consideration. As currently proposed, the necessary NO vote is “more NO than YES votes,” but the exact amount needed to disqualify is negotiable. Remember, even if every vote is NO, if a quorum (minimum number of voters) doesn’t participate, the work cannot be disqualified. (This makes it difficult for a small group to campaign against a work.)
At the end of the semi-final round, any works disqualified by the members, withdrawn by the nominees, or disqualified by the Committee on technical grounds is out of the running. From among the remaining semi-finalists, the five that got the most votes in the nominating phase become the finalists, and the final ballot continues as it currently does. Note that in this case it doesn’t matter how many YES votes a semi-finalist got, only that it didn’t get a negative majority.
3SV effectively moves the votes on NO AWARD to the semi-finals, although it would remain a candidate on the final ballot. It allows the members of the current Worldcon (the ones who will be voting on the final ballot) to decide in advance which works they think deserve to be on the final ballot. It does this at a price, however, and that price is to carve 6-8 weeks out of an already relatively crowded schedule. The proposal moves the deadline by which you have to be a member in order to nominate up by a month, and in practice would require the nominating deadline to be earlier in the year.
Another drawback of 3SV is that it triples the number of nominees that the Worldcon Committee (the Hugo Award Administrators) need to vet and contact. However, as a trade-off, it gives the administrator roughly three times as much time to do this contact work, and makes the vetting and contacting process public. That is because the Administrator would not need to contact semi-finalists in advance of announcing the “longlist” of semi-finalists. While the semi-final ballot runs, the Administrator would be contacting semi-finalists to give them an opportunity to withdraw from consideration in the final ballot. Administrators could put out a public appeal if they are unable to contact a given semi-finalist. In addition, the Administrator can do eligibility confirmation in public, with the help of the many other people who will undoubtedly be checking over the list and asking questions. Should a semi-finalist be disqualified on technical grounds, the semi-finalist would not be replaced on the semi-final ballot. Ineligible works show up on the existing Top 15 lists now, and the semi-final round would be too short to allow for replacing longlist slots.
An incidental feature of 3SV is that the finalists would not know they’d made the final ballot until the shortlist was announced. They’d know they were semi-finalists, and during the semi-finals they would have had time to decline if they so choose, but it would be impossible for them to leak being a finalist because they would not know it themselves.
Additional drawbacks to 3SV include the fact that it is explicitly negative. It is a place where you vote against things. Some people are philosophically opposed to “down-voting” works. Furthermore, should the known bad actors stop trying to game the system, there is little need for this semi-final round, and you might find people not even bothering to participate in it, which might result in complaints that we’d added complexity and expense for no obvious reason.
3SV is a “vote against stuff” system. An alternative to it was the second proposal.
Double Nominations with Approval Voting (DN/AV, sometimes just DN) is similar to 3-Stage Voting, in that there would be a semi-final round with the Top 15 (see warning at the beginning of the article) nominees listed in an order that would not reveal how many nominations they received. Only members (attending and supporting) of the current Worldcon would vote at this stage. But instead of voting against semi-finalists as you do in 3SV, you would vote for those semi-finalists who you think deserve to be on the final ballot. You could vote for one or all of the semi-finalists. The version as currently proposed included a single write-in slot as well, primarily as a safety valve. In this case, the number of votes a semi-finalist gets here is critical, because only the top five would continue to the finals.
In 3SV, the number of YES votes doesn’t matter as long as there are fewer NO votes (or insufficient ballots cast to qualify the election at all). The relative number of YES votes doesn’t matter as long as the semi-finalist isn’t disqualified, because it’s the original nominating ballot count that sends works on to the final round if they survive the weeding-out process of 3SV’s downvotes. However, in DN/AV, the five works with the most votes in the semi-final round go on to the final ballot, and the number of nominating ballots cast to get the works onto the longlist is irrelevant. Some have called for nominating counts to be used as a tie-breaker, as they envision a 15-way tie for the final ballot. I personally think this unlikely, and there have been as many as eight finalists on a Hugo ballot due to ties for the final position, so I think we could live without a tie-breaker.
All of the extra administrative issues of 3SV are shared by DN/AV, so I won’t go over them again here.
DN/AV eliminates the “negative” aspect of 3SV, in that you vote for semi-finalists, not against them.
Some have suggested that there’s an implication that “you need to have read all of the semi-finalists in order to vote on this ballot,” although I don’t think that is true in either case. DN/AV is a second nominating round. Just as the current nominating ballot makes no pretense that you should have read everything published last year, DN/AV doesn’t expect voters to have read all fifteen semi-finalists, but instead to pick those that the voters think worth considering on the final ballot, knowing that only the top five will appear there.
The biggest advantage to either 3SV or DN/AV is that they put the decision of “what to have on the final ballot” in the hand of the members who will be voting on that final ballot, and that the majority will of those voters will prevail. Neither system is particularly susceptible to gaming by small minorities. The biggest drawback to either of the first two proposals is that they add an additional round of voting, with administrative overhead and complexity.
Some people have asserted, with various degrees of strength, that the Committee (Hugo Award Administrators) should simply ignore “slate voters” or disqualify “obvious slate-generated finalists.” In my opinion, such proposals are hugely problematical, in that they give Administrators authority they have never had in the entire history of the Hugo Awards. However, there is one historical precedent to which we can look when a group of bad actors appears to have forced a work onto the ballot, and the Worldcon Committee tried to ameliorate the action without actually disqualifying anyone. We’ll consider this in the final proposal.
In 1989, the Hugo Award Administrator noticed an odd situation, where a set of nominating ballots arrived in close order with a single nomination cast in a single category. This was enough to place a finalist on the ballot that seemed unusual to anyone who had been watching the kinds of things that had made the shortlist in recent years. The ballots all having included membership payments in the form of consecutively numbered money orders further raised suspicions. According to the coverage of the situation in File 770 at the time, the 1989 Worldcon committee said in a statement at the time that they didn’t do any investigation, but that one person wrote to the committee to inquire why he had a membership when he hadn’t paid for it. (After they made a public statement, some fans contacted the committee claiming to have cast these votes with innocent intent.) In any event, there has always been a strong ethos in the Hugo Awards to allow the membership to speak, not a small committee. On the other hand, this case seemed to be doing someone out of a Hugo Award finalist slot unfairly. The Committee took the unprecedented step of adding the sixth-place nominee to the shortlist. Not too long thereafter, one of the six finalists (the one that seemed to be odd compared to the others) withdrew. There is no evidence or suggestion that the finalist who withdrew had anything to do with the string of bullet-voted ballots. At most, this was a case of enthusiasts with more money than common sense or ethics.
The final proposal that I’ve written up would explicitly authorize the Worldcon Committee (in practice, the Hugo Award Administration Subcommittee set up by the Worldcon Committee) to take the action that the 1989 Worldcon did whenever they think that there was a pattern of unethical voting in the nominating round. The Committee would be authorized to add up to two additional finalists from among the Top 15. They would not reveal which finalists they added until after the Hugo Awards ceremony, when it would be included in the post-ceremony detailed results.
I have no expectation that the current people who have typically been part of the Hugo Administration Subcommittee would be the ones who would make the actual Plus 2 decisions. I expect that any sensible Worldcon Committee would recruit additional Administrators for their literary judgement, just as the World Fantasy Award does. Any Administrator, no matter whether they were recruited for computer skills, literary skills, or public relations skills, would be ineligible for that year’s Award just as the current Administrators are.
Should the Administrators determine that there was no need to do so, they could choose to not add additional finalists. Thus, if known Bad Actors desist from their wrecking ways, the Administrators could simply continue running things as we have done in the past, leaving it up to the five highest pluralities of nominations. This would simplify administration and require relatively little change.
This proposal allows the Committee to pull from the Top 15, rather than simply the next two finishers in the nominations, to minimize multiple-slating attacks on the system that would try to dominate the top seven rather than the top five positions. While 20% of the electorate has been able to dominate the first five positions relatively easily, it seems unlikely that such a small group could dominate the top fifteen unless their voting power grew to a majority of the entire electorate. Inasmuch as solutions that represent a majority of the electorate (even if you personally dislike the result) are not contrary to democratic process, there is nothing in this proposal that tries to “defend against it.” As I’ve said above and will continue to say, if you can command a majority of the voters, you get your way, even if I don’t like it. It is minorities dominating the process that troubles me.
Note that there’s nothing magic about adding two additional works. It could be one or more. The exact value is debatable. However, consider that we do hope that most people read all or most of the finalists, and therefore making the shortlist too long works against any good you might get from adding (say) five extra works to the ballot.
Broadly speaking, Plus Two is compatible with either (but not both) 3SV and DN/AV. That is, this proposal can be considered separately from either 3SV or DN/AV, whereas the first two proposals above are antithetical to each other, and only one of them could be reasonably considered at a time.
The biggest advantage of Plus Two is that it’s relatively simple, does not add a lot of administrative overhead, and allows the existing two-stage process to stay in place, with only a relatively minor change of adding works to the final ballot. It also adds the “human judgement element” that it appears that many people seem to think is necessary to combat bad-faith efforts to sabotage the Awards.
The biggest disadvantage is that it gives Administrators an authority that they’ve never actually had before, and that they have used only once before, without any explicit sanction. Administrators may be reluctant to serve on the Hugo Award Administration Subcommittee if they know that they may be responsible in some way for adding works to the Hugo Award ballot.
There are no “magic bullets” when it comes to tinkering with the Hugo Award rules. None of the proposals here is a perfect fix. Indeed, per Arrow’s Theorem, there is no such thing as a perfect voting system. However, we can try to move things around and minimize unfairness, usually at the expense of additional complexity. The political question then is how much complexity we can tolerate to improve perceived fairness.
The Instant Runoff Voting system that we use on the final ballot is a case of trading complexity (IRV boggles the minds of people who reject anything other than First Past the Post voting) for fairness (IRV usually returns the least-disliked candidate in an election, rather than the one with the largest plurality; in a field of more than two candidates like the Hugo Awards, it usually returns a consensus winner, not just a strong front-runner). Should we decide that the changes we’ve started with E Pluribus Hugo and 4/6 that are up for ratification this year are insufficient to tilt the field back toward perceived fairness, it behooves the members of WSFS meeting in Kansas City this summer to consider additional changes now, not later.
WSFS rules are intentionally complicated to change, in order to prevent concerns of a day or even of a single year to overwhelm the process. However, that doesn’t mean that we cannot start queuing up additional changes now while we continue to monitor how things proceed, in order to protect our own longer term interests.
I expect at least one of the proposals outlined here to be on the agenda of this year’s WSFS Business Meeting. We might even have all three of them.