Hugo Voting Idea Toolkit

At my request, Stoic Cynic updated his outline of the ideas attached to various Hugo voting reform proposals. He calls it “trying to define the solution domain.”

Guidelines:  Any solution should, to the greatest extent possible, maintain the pre-slating character of the awards.

Boundaries:  There are three stages we can act: Membership, Nomination, and Voting.

Actions:  Solutions would seem to fall into four general categories: Gatekeeping, Filtering, Addition, and Vetoing.

Proposed solutions, each with pros and cons, include –

1. Do Nothing

2. Membership

a) Restrict nominations to attending members.

b) Restrict nominations to a jury.

c) Remove second year nominating rights.

d) Increase supporting membership fees.

e) Ban bad actors.

f) Restrict nominating to members with consecutive years of membership.

3. Nomination

a) Implement slate detection algorithms (These were mentioned last year. What would they look like?).

b) Allow the membership to confirm nominations through a long list (DN).

c) Allow membership to deny nominations through a long list (3SV)

d) Empower the admins to remove ballots of bad actors.

e) Empower the admins to add a limited number of nominees in years with bad actors (A+2)

f) Empower a jury to add a limited number of overlooked nominees to the short list (Juried+2)

g) Algorithmically filter the nominations (EPH, EPH+, Diluted Nomination, Frozen Nomination, NOTE: Jameson Quinn is working on another algorithm variant, as yet unnamed, to incorporate a ‘satisfaction index’ and achieve A+2 type results in conjunction with EPH programmatically).

h) Restrict the nominations relative to the field (4/6)

i) Counter-slating through a third party collation. *

j) Empower the admins or a jury to remove works which tend to bring discredit on WorldCon

k) Allow the membership to vote on expanding just the number of nominees on a given year’s shortlist.

4. Voting

a) Filter the votes by preference ranking (IRV)

b) Veto bad actors (No Award)

* Removed by request

69 thoughts on “Hugo Voting Idea Toolkit

  1. I wasn’t claiming my suggestion to be ideal, but to get it on the list with all the other non ideal ones.

    To me the ability for puppies to reduce the likelihood of some few stories they dislike being nominees seems like not a problem. The wide array of options makes this prospect way less harmful than their poorer choices being in the nominee lists.

    And it avoids a third round so it is less of a hassle. And I sincerely doubt anyone could get hundreds of people organized to vote without information leaks.

  2. @snowcrash

    IIRC, Greg Hullender suggested something similar a while back. As SFK says, this is an explicit “Vote Your Hate” call, and nothing brings out the Usual Idiots like giving them the ability to paint a target on women/ minorities etc.. It will also almost certainly lead to various people (Scalzi, Jemisin, Swirsky, Sarkeesian etc etc) never getting a nomination.

    Yes, although, for the record, I also included an option for the fans to reverse the disqualification in the final vote. (I’m a big believer in checks and balances.) That was meant to defeat attempts by slates to target people like Scalzi and Jemisin. It had the drawback that things like “rape room” and “space raptor” would still end up in the Hugo Packet.

    I think 3SVs vetting process is far better, and has far less potential for abuse.

    I agree entirely.

  3. The obvious missing option is a longlist stage that’s treated the same way as the current nominations system (ie 5 votes, with or without EPH) rather than a strange negative voting or “vote for everything” round. This would also address the weaknesses of EPH, ie that normal nominations are still too diluted compared to slates.

  4. I think a second round of nominating based on the short list is problematic.
    The nominations are the nominations – people have made their choices.
    All that is needed is a tool for tossing out the absolute ringers.
    The high-bar negative voting option provides that.
    It’s like a pre-emptive No Award that allows the voting to proceed with the five best organic nominations.
    It’s the least disturbance possible.
    Re-sorting the nominations from the long-list is a little too much like a long slate for my tastes.

  5. Missing are a variety of proposals, including mixed proposals, passing the problem on to the voting membership. Broadly, these are a class of “write-in” proposals which allow the voters to choose to ignore what made the ballot and vote other choices. These may be combined with methods that allow the members to guess or even know the longlist, and thus work out for themselves what the “real” ballot would have been (absent slate effects) and choose their write in from that.

    I call this class of solutions “write-in” to draw the parallel to the use of write-in as the almost universal standard in most election systems (including WSFS site selection) for allowing the voters to correct flaws in the nomination system. In practice, voters would probably chose from some canonical list, possibly even a longlist published by the convention, to simplify the canonicalization of their ballots.

    These have some similarities to the 3-stage systems, effectively giving voters an opportunity to perform some of the functions of the nomination system after the traditional nomination.

    e,f,g and j bear similarities to other proposals floated, though they found many opponents. Though again, the reasoning is that in most of the world, attempted corruption of systems is usually dealt with by some sort of court system which, under rules, applies human judgment rather than algorithms to fix the problem. This is supported by the general principle that dynamic human interference with any system can rarely be repelled with algorithms, especially algorithms which take 2 years to change. Even cryptographers, who are the most algorithm-loving group in the security world, design so that their algorithms can be quickly replaced if flaws are found.

  6. The negative voting (or the as-many-as-you-like voting which is the same thing in a different hat) is already re-sorting the nominations though. You’re tossing out 10 of 15, not just the worst of the worst. And the thing it’s sorting for is the least controversial works, which is really not the best way to find a good SF story even without bad actors.

  7. Kevin P, the sorting has already been done. The proposal doesn’t change the rank of the long-listed works. In a world (in fact, in the world we have right now) where the longlist doesn’t exist, the top five get on the ballot, period. In a world with 3SV, the top five that the electorate think are valid nominations get on the ballot; the current consensus is that they are not re-ordered. You may have missed the thrashing-out of details in THIS THREAD; we’re up to page 8 as of the time of this writing.

    So any edgy or controversial works, assuming that a very large number of the electorate don’t think it’s unqualified, which end up in the top five of the original ballots (or get bumped into the top five by disqualifications of other works) will be on the ballot.

  8. How does any hypothetical +1 interact with the “the sorting has already been done” part? Surely it would end up re-ordering the original nominations….

  9. Cally, you might want to ask over in the “Three Possible” thread; that one seems to be where most of the thrashing-out-of-details is going on.

  10. Write-ins let a voter blow off steam but it’s not an effective way to change the roster. Write-ins won’t meaningfully aggregate enough to displace Griefer noms.

  11. Jim, but write-ins-from-the-longlist might conceivably bump a 6th place work into 5th place. (I’m not saying this is a bad thing. There are always things on the longlist that I wish I’d read in time to nominate them.)

  12. @Cassy B: Brad Templeton’s class of write-in proposals seems to keep the current two-round process (no 3SV, no DN), The writing-in happens at the finals stage. If it’s accompanied by an early release of the longlist, then it becomes a couple things, depending on circumstances:

    * If the Hugo packet itself only includes the Top Five, then it becomes a 15-item final ballot without sure access to the Bottom 10 items.
    * If the Hugo packet includes the Bottom 10s, then it becomes a 15-item final ballot with triple the reading/viewing load and no practical value to being in the Top Five.

    Either way, the Hugo packet goes out with Let’s Beat the Crap Out of Rachel Swirsky BUT HEY IT’S JUST A STORY and People Who Have Committed No Crime Totally Have LOL and This Time I Found a Smut Author Isn’t More Than a Match for Me on it.

    A two-step write-in process is inadequate to Griefer campaigns born of malice. And malicious griefing remains the problem before us.

  13. Jim Henley: Write-ins let a voter blow off steam but it’s not an effective way to change the roster. Write-ins won’t meaningfully aggregate enough to displace Griefer noms

    … and they would result in a significant addition of work in the already-shortened time period, because the Hugo Admins would have to normalize and count them, anyway.

  14. 3SV vs. 3SV+1?

    I’m thinking once there is 3SV, there is no reason to introduce any further tampering with the members’ nominations.
    It’s a LOT more work for the Admins, basically a whole new set of votes, potentially re-ordering the nomination ranks, instead of just confirming up or down on each item.
    And it isn’t a fix for the problem that is being actually being addressed: the presence of dreck on the ballot.
    There is no need to concentrate votes to achieve a votable ballot, because 3SV has already done that.
    If 3SV is set up correctly the resulting ballot will consist of the top five items nominated by the membership, with no slated crud remaining.
    There is no need to meddle any further with the voters’ choices, even if the prospect of second-guessing oneself is shiny.

  15. @Errhead:

    Highlander Slate Elimination is based on “there can be only one.” Any identical ballots are compressed into 1, and then the unique ballots are calculated with whatever method you choose. The more disciplined the slate, the more effective it is. Against the 1984 data with simulated slates it’s highly effective in removing the slate influence. The most popular works in the unslated data are also the most widely nominated so that the flattening of accidental small slates evens out and the results are virtually the same as as the original finalists. I’d love to see what effect it would have on the 2015 data, pretty sure the line on the graph would would be much flatter than the others. It would be more than sufficient to stop the Sad Puppies kind of slating, but it’s susceptible to active malicious collusion.

    I’m getting interested in HSE! But, in terms of malicious collusion, I’m thinking:

    200 Griefers. Each is instructed to pick any four of the five items on my slate randomly. For their fifth item, they are directed to find the most obscure candidate they can find, also randomly. I suspect that, without a special provision for ballot-checking, this fifth candidate could actually be ineligible: a story from ten years ago, a Buzzfeed listicle, a random string of letters I pretend is a title or person.

    Don’t I end up with something very close to 160 nominees for each candidate on my slate?

  16. @Jim Henley

    I’m interested in the answer too. Unfortunately I suspect all of the vote counting tweaks can be gamed. Trying to work around the Chicago cemetery vote calls for election judges more than tweaking the vote counting algorithm or adding stages (in which the cemetery vote is still allowed to participate!). The temperature of the electorate isn’t right for election judges yet but I think it’s the real solution.

    (Can’t mention cemetery voters without an obligatory Simpson’s quote: “Oh no, the dead have risen and they’re voting Republican!”)

  17. @Stoic Cynic:

    The temperature of the electorate isn’t right for election judges yet but I think it’s the real solution

    Agree. The movement toward 3SV is encouraging because it represents fandom edging toward a recognition that human judgment is an inescapable part of the health of the process. But I have my doubts that a “committee of the whole” can be an effective equivalent of a forum moderator.

  18. Pingback: To Say Nothing of the Dogs; or, How We Confound the Hugos’ Third Slump (Hugo voting proposal discussion 5) | File 770

Comments are closed.