Hugo Voting Rules Proposals Sponsored By Harris, Buff, Standlee, Others

Mini Hugo rocket carried into space and photgraphed by astronaut Kjell Lindgren in 2015.

Mini Hugo rocket carried into space and photgraphed by astronaut Kjell Lindgren in 2015.

Apart from the discussions Jameson Quinn has been leading here, another group of fans has been working on ideas for reforming the Hugo voting process. Yesterday they published the drafts of their three main motions and an amendment to EPH (given its first passage last year) as a Google document.

The three main motions do these things:

(1) Change the deadline you must be a Worldcon member to be eligible to nominate from January 31 to December 31 of the previous year.

(2) Restrict eligibility to nominate to members of the current and preceding Worldcon.

(3) Add a second round that allows members to vote out something that makes the initial long list (“Three Stage Voting”).

Colin Harris (co-chair of the 2005 Worldcon), Warren Buff, Kevin Standlee (co-chair of the 2002 Worldcon), Nicholas Whyte, and Colette Fozard each sponsor at least one of the several motions. Harris explains:

We plan to submit the motions officially in about a week; we are publishing them now to encourage discussion, rather than because we expect to change the text — but of course if people point out important things we’ve missed, we’ll take the opportunity to fix any issues.

Commenting specifically about the Three-Stage-Voting proposal, Harris says:

To be clear, my stance as the main mover on 3SV is simple. I wish this change was not necessary, but I believe that EPH and the other proposals already in hand will not achieve the necessary outcomes. In particular, I believe that guaranteeing a couple of broadly acceptable finalists per category is simply not a high enough bar for “success” in restoring the integrity, reputation and stability of the awards. I do not know if 3SV will pass, but I believe that the Business Meeting should have the opportunity to discuss this more direct option for tackling manipulation of the nomination process.

The text of the proposals follows the jump.

PROPOSALS TO REVISE THE HUGO VOTING PROCESS

This document provides an advance look at a set of related motions which are being submitted to the 2016 WSFS Business Meeting in relation to the Hugo voting process. The motions have been developed by a small discussion group including Worldcon and NASFiC Chairs and Hugo Award Administrators.

Three separate proposals are presented, plus an associated amendment to the EPH ratification provision.  The motions are essentially independent in effect although they all relate to the Hugo voting process.

  • December is Good Enough (move the Nomination Ballot eligibility cut-off date from January 31 to December 31 of the previous year)
  • Two Years are Enough (limit Nomination to members of the current and preceding Worldcon).
  • Three Stage Voting (3SV), or “The Only Winning Move is Not to Play”
  • Amendment to EPH

SHORT TITLE: DECEMBER IS GOOD ENOUGH

Moved, to amend the WSFS Constitution by striking out and inserting text as follows:

3.7.1: The Worldcon Committee shall conduct a poll to select the finalists for the Award voting. Each member of the administering Worldcon, the immediately preceding Worldcon, or the immediately following Worldcon as of January 31 the end of the current previous calendar year shall be allowed to make up to five (5) equally weighted nominations in every category.

  • Proposed by: Colette Fozard, Warren Buff, Nicholas Whyte

Discussion:

The administrative strain on Hugo Administrators has increased substantially in recent years, due to:

(a) extension of nominating rights to include members of the year N+1 Worldcon – requiring the Administrator to merge three sets of membership data, two of which are continually changing as new members join the respective conventions.

(b) a substantial rise in the number of Supporting members due to increased interest in the Awards.

Members also expect to receive their nominating rights (and PINs) rapidly after the ballot opens – typically in early January – while the Administrator has to continue merging membership data from that point until the eligibility deadline, with no control of the timeliness of the data feed from Worldcon N+1.

Changing the eligibility deadline will reduce this strain and ease the administration process. In addition,  in the event that 3 Stage Voting passes, an earlier eligibility cut-off will ensure that the overall schedule remains manageable (e.g. for a nomination phase of 8  weeks starting on January 1, then a December 31 cut-off is significantly more robust and easier to handle than January 31).

SHORT TITLE: TWO YEARS ARE ENOUGH

Moved, to amend the WSFS Constitution by striking out and inserting text as follows:

3.7.1: The Worldcon Committee shall conduct a poll to select the finalists for the Award voting. Each member of the administering Worldcon, or the immediately preceding Worldcon, or the immediately following Worldcon as of January 31 of the current calendar year shall be allowed to make up to five (5) equally weighted nominations in every category.

Provided that members of the 2019 Worldcon will retain their nominating rights in the 2018 Hugo Awards.

  • Proposed by: Warren Buff, Colin Harris

Discussion:

The extension of the nominating franchise from two Worldcons (N and N-1) to three by including year N+1 was implemented as part of the drive to expand the voting population at a time when member participation in the Hugos was substantially lower than it is now.  This change has created some undesirable side effects, and we believe the most appropriate response is to return to the previous arrangement.

The inclusion of year N+1 creates a very high burden on the Administrators since (a) they have to merge three sets of membership data rather than two (b) two of these are in motion with new members joining up to the eligibility cut-off date (c) the most recently seated Worldcon is often still establishing robust data and membership processes at the point where the current Worldcon needs a reliable flow of quality data.

The removal of year N+1 will remove this burden while only marginally reducing the size of the voting pool. In a typical recent year, Worldcon N-1 might have 6,000 members at the eligibility cut-off, Worldcon N might have 3,500, and Worldcon N+1 might have 2,000 – but in the early months after being seated, Worldcon N+1’s membership will be dominated by people who participated in Site Selection and hence are also members of Worldcon N.

This change would also mean that individuals who only wish to join to participate in the nomination process would have to purchase memberships every two years rather than every three.

SHORT TITLE: THREE STAGE VOTING (3SV), OR “THE ONLY WINNING MOVE IS NOT TO PLAY”

Moved, to amend Section 3.7 (Nominations) and Section 3.8 (Tallying of Nominations) for the purpose of creating an intermediate stage in the Hugo Award selection process by striking out and inserting text as follows.

Section 3.7: Nominations.

3.7.1: The Worldcon Committee shall conduct a two-stage poll to select the finalists for the Award voting. Each In the Nominating stage, each member of the administering Worldcon, the immediately preceding Worldcon, or the immediately following Worldcon as of January 31 of the current calendar year shall be allowed to make up to five (5) equally weighted nominations in every category.

3.7.2: The Committee shall include with each nomination ballot a copy of Article 3 of the WSFS Constitution and any applicable extensions of eligibility under Sections 3.4.

3.7.3: Nominations shall be solicited only for the Hugo Awards and the John W. Campbell Award for Best New Writer.

3.7.4 3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.

3.7.5 3.8.3: Any nominations for “No Award” shall be disregarded.

3.7.6 3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.

3.7.7 3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.

3.7.8 3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.7 or 3.2.8, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category

Section 3.8: Tallying of Nominations. Qualification Stage.

3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

[3.8.2, 3.8.3, 3.8.4, 3.8.6, and 3.8.7 moved to Section 3.7.]

3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.

3.8.1: The Qualification stage of the process shall be based on a long list of the top fifteen Qualifiers (including ties that include fifteenth place) from the nomination process in each category. Only WSFS members may vote in this stage.

3.8.2: The purpose of the Qualification Stage is to allow the membership to confirm their willingness to see each Qualifier taken forward as a potential Hugo Award Finalist.

3.8.3: In the Qualification Stage ballot, each voter may choose between the options “Accept”, “Reject”, and “Abstain” for each Qualifier in each category.

3.8.4: A Qualifier shall be eliminated from consideration for the Final Ballot if it meets the following two criteria:

(1) the number of “Reject” votes is at least 60% of the combined total of “Accept” and “Reject” votes;

(2) the number of “Reject” votes is at least the higher of 600 or 20% of the number of eligible voters.

3.8.5: The final Award ballots shall list in each category the five eligible Qualifiers who received the most nominations in the first stage Nominating Ballot and were not eliminated from consideration in the Qualification Stage. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

Provided that unless this amendment is re-ratified by the 2023 Business Meeting, Sections 3.7 and 3.8 shall revert to their wording prior to the initial ratification of this amendment, and

Provided that the question of re-ratification shall be automatically be placed on the agenda of the 2023 Business Meeting with any constitutional amendments awaiting ratification.

  • Proposed By: Colin Harris, Kevin Standlee, Nicholas Whyte, Colette Fozard, Warren Buff

Discussion:

The essential argument for this change is that it enables us to directly address slates and bad actors in a direct way – and a way that statistical solutions such as EPH and 4+6 cannot.  We have put it forward in recognition of the fact that the reputation and integrity of the Hugo Awards is under sustained attack, and in the belief that a response is needed to this reality.  We would of course have preferred it if such a response was not necessary. The rationale for the proposal is as follows.

  • Statistical solutions may reduce the number of slate nominees; but not sufficiently to act as a deterrent to slate campaigners or bad actors. Ensuring 1-2 non-slate candidates per category is ultimately a pyrrhic victory in terms of the reputation of the Awards; securing 2-3 finalists per category offers a strong incentive for slate campaigners to continue their efforts indefinitely, and continues to exclude nominees that would otherwise have made the Final Ballot from being recognised.
  • We have seen that the membership is willing to use No Award to reject finalists that they consider are not appropriate winners of the Hugo Award. However this test is applied when the damage is done. The proposal essentially moves the No Award test to an earlier stage of the process.  The Qualification Ballot is not a ranking ballot, but specifically enables members to reject candidates that they believe have benefited from inappropriate promotion. (It is important that the Qualification Ballot is presented clearly as one in which “Reject” should be used only for candidates that members feel are not suitable for inclusion on the Final Ballot – typically due to abuse of process – and not as an opportunity to express preference among the acceptable candidates. The wording of the Qualification Ballot rubric will therefore need to be clearly presented.
  • It is similarly important to note that the Qualification Ballot is not used to re-rank the candidates for inclusion on the Final Ballot. Once any rejected candidates have been eliminated, it is the ranking from the original Nomination Ballot that determines the ranking and hence which finalists should be taken forward.

Why now? We have been asked why we have brought this proposal forward now, when EPH has yet to be ratified and tested. Our concern is that the Awards are being progressively tainted, and that we cannot afford to wait until EPH has been tried before having a “Plan B” under way. (Assuming EPH is ratified in 2016 and is used in 2017, and does not resolve concerns satisfactorily, a new approach passed in 2017 and ratified in 2018 would only be in effect in 2019.)

Our suggestion is that 3SV should be passed now to ensure that there is a strong option in hand for protecting the future of the Awards.  If this option is found to be unnecessary once the 2017 nomination process has been run, then 3SV need not be ratified.

If 3SV is felt to be necessary – to send a message that WSFS decisively rejects attempts to game the awards and will single out and reject candidates that gain from such tactics – then there will be a separate decision on whether EPH should continue to be retained as well. While EPH is a sophisticated algorithm, it would arguably be redundant in rejecting slates if 3SV is in place (and indeed, 3SV makes it easier to separate tactical slates from genuinely correlated votes such as fans of a TV series nominating multiple episodes of that show). Moreover, it imposes a significant administrative burden and is a less transparent algorithm for members to understand.

Our view is that the other measures which are up for ratification i.e. 4+6 and Nominee Diversity – would combine with 3SV to provide a robust way forward for the Awards, and that EPH should potentially be dropped if it does not materially resolve the slate issue. As such we have introduced an amendment for additional re-ratification of EPH in 2017, such that the Business Meeting could choose to drop EPH if it proceeds with 3SV. The BM may of course choose to continue with both solutions – they are complementary – but we feel that this option should be available to the meeting in Helsinki.

AMENDMENT TO EPH

Moved, to substitute the following new enacting clauses for the existing clauses in the E Pluribus Hugo Proposal to require annual re-ratification until 2022:

Moved, that unless this amendment is re-ratified by the 2022 Business Meeting, Section 3.A shall be repealed, and

Provided that the question of re-ratification shall be automatically be placed on the agenda of the 2022 Business Meeting with any constitutional amendments awaiting ratification.

Provided that unless this amendment is re-ratified by every Business Meeting between the initial ratification of this amendment and the 2022 Business Meeting, Section 3.A shall be repealed, and

Provided that the question of re-ratification shall be automatically placed on the agenda of each Business Meeting between now and the 2022 Business Meeting with any constitutional amendments awaiting ratification, unless any of those meetings should fail to re-ratify the amendment, in which case no further re-ratification votes shall be held.

  • Proposed By: Colin Harris, Kevin Standlee

Discussion:

This is an adjunct to the 3SV motion.  It provides additional opportunities to repeal EPH in the event that the Business Meeting decides to adopt 3SV (or an alternative solution) in place of EPH as a means to address the issues experienced in recent Hugo ballots. EPH can of course be retained alongside such additional measures, but the Business Meeting may not wish to do so in view of the administrative load and the loss of voter understanding associated with the complex algorithm.

99 thoughts on “Hugo Voting Rules Proposals Sponsored By Harris, Buff, Standlee, Others

  1. I hope commenting here is useful to the authors.

    In “Two years are enough” you say “– but in the early months after being seated, Worldcon N+1’s membership will be dominated by people who participated in Site Selection and hence are also members of Worldcon N.” Shouldn’t that Worldcon N be Worldcon N-1?

    It’s something I’d been looking at with the data I could find and my rough and ready impression is that there are the origional site selection members, then a slow trickle of new members until just after Worldcon N when there is a sudden leap in memberships of N+1. I strongly suspect that these are from N members deciding to attend next year too, and that there are very few “new” people buying memberships at this stage.

  2. Thanks to the group for all their work!

    The points about relieving administrative strain are well-made. With regards to 3SV, I like the ‘Qualifier’ and ‘Qualification Stage’ terminology. And I agree that the Qualification Rubric needs to be carefully worded.

    However, this part:

    While EPH is a sophisticated algorithm, it would arguably be redundant in rejecting slates if 3SV is in place

    is untrue. As Greg Hullender explains here, if EPH is absent, a group of 300 slate voters is sufficient to sweep most of the top 15t. This is not a minor issue – there are six categories where they would sweep it completely. I’ll emphasize his conclusion:

    With EPH, 3SV seems to do exactly what we need. Without EPH, 3SV is a dismal failure.

    I want to restate that I agree with all the other parts, and thanks for all the work that went in! But I think the EPH amendment would be a bad mistake.

  3. It seems to me that 3SV addresses a different problem from EPH, namely that of particularly egregious nominees.

    The proposers say:
    to send a message that WSFS decisively rejects attempts to game the awards and will single out and reject candidates that gain from such tactics;
    they clearly envisage voters excluding anything that benefits from a slate. But I think this is unlikely to happen. If people vote down anything that is on a slate, VD can prevent Scalzi or Jemisin getting a Hugo by putting them on a slate. If people attempt to vote down only things that have reached the final stage by being on a slate, they won’t have enough information to do this effectively, absent full voting figures. There are, I would think, five slated finalists this year which would certainly have reached the final without a slate, but there are several others that might have done. How do we decide which to vote down?

    What most people will do, I think, is vote down clearly unworthy works. This year, that would include ‘If You Were an Award, My Love’, the Eness and Greyland works, perhaps Chuck Tingle (but perhaps not), perhaps everything published by Castalia House. I don’t think any of this year’s nominees for Novella, for instance, would be voted down.

    This means that there will still be an advantage in being on a slate – slate works that are not voted down will keep their lead. To the extent that EPH fails to solve the general problem posed by slates, this proposal also fails. If we want a ballot that is actually chosen by the members of Worldcon, rather than partly by the members of Worldcon and partly by Vox Day – and I think the proposers make a good case that this is what we want – we have not yet found a way of achieving that.

  4. To answer Andrew M and expand on Bartimaeus’s comment, the problem is that the top-15 list that 3SV uses needs to be computed using EPH (or, better, EPH+). If it is computed using just the simple number of votes, then the slates will dominate the whole 15-item list in every category. If EPH were used, every category but one would have at least 5 organic nominees in it. EPH+ would have at least five organic results in all categories.

    Without any form of EPH, only three categories would have five or more organic nominees, and only six would have more than two. Even in lists of 15.

    That fails the stated goal of the measure.

    I would suggest adding to Section 3.8.1 “The top 15 list shall be computed using the EPH+ algorithm.”

  5. “While EPH is a sophisticated algorithm, it would arguably be redundant in rejecting slates if 3SV is in place (and indeed, 3SV makes it easier to separate tactical slates from genuinely correlated votes such as fans of a TV series nominating multiple episodes of that show). “

    I absolutely believe that this part is very much wrong and think it best would be removed. It serves no other purpose than creating a very unnecessary fight and cause a split where people otherwise would be in agreement.

  6. @Greg Hullender: can you provide numbers to back up this assertion? ISTM that you \might/ be correct for short story due to the large number of potential nominees, but that in most categories there haven’t been enough slaters that dividing them into 3 equal groups (each group nominating a unique slate) would force others. Note that I am asking for provable numbers; I have seen your estimates of the number of slaters and thought they had some large assumptions.

    @Doire: my reaction also, from being on 3 Worldcon steering committees — although the last was 12 years ago. Certainly N+1 starts with a huge bulge from N-1; it would be interesting to see how many memberships recent Worldcons have sold between coming home from their selection and the end of that year, but I doubt it’s a large number or that many of those were not members of one of the two previous Worldcons, since AFAICT people not already in the network don’t hear about a Worldcon they might go to until well after its selection.

    @Hampus: I don’t think “belief” is called for here; I’d like to see real numbers. This shouldn’t involve the kind of confidential work that showed EPH’s limitations; how many nominations did the top 15 works in each category receive in each of the last two years? (Or at least last year; IIRC, this year’s numbers are withheld for some time to avoid affecting the final voting.)

  7. “would force others off the ballot“. The things one see only on the Nth review….

  8. Are these proposals intended to replace or enhance EPH? Assuming EPH recieves final approval at MAC this year, it seems to me that we would need a year or two to assess if EPH is doing what it was supposed to do before further changing the system. Or is the intent to reject EPH and replace it with these proposals?

  9. @Chip Hitchcock

    Nomination data for last year is here (beginning on page 18):

    http://www.thehugoawards.org/content/pdf/2015HugoStatistics.pdf

    I’m not going to dig up my analysis right now (It’s on another computer). In a lot of categories though the number of nominations for non-slate works are low enough that a combination of two and three way slate splitting would allow anywhere from absolute control of that category’s long list to significant dominance of the long list.

    Of course we don’t know how many actual slate voters there are but I think a reasonable estimate is 200 to 300. I believe I used 250 in my analysis (though without digging the files out I won’t swear to it). The estimates above are based on that range anyways.

    Coordinating a multi-slate approach and maintaining slate discipline would be difficult but not unachievable. 3SV by itself is leaving an exploit on the table.

    Can I guarantee it will successfully be exploited? No. But why leave the exploit in place?

  10. @T. Wright Barnes

    Are these proposals intended to replace or enhance EPH? Assuming EPH recieves final approval at MAC this year, it seems to me that we would need a year or two to assess if EPH is doing what it was supposed to do before further changing the system. Or is the intent to reject EPH and replace it with these proposals?

    It had been believed that the slaters would lose interest if they couldn’t sweep entire categories, since it that would mean that they could neither get awards for their own favorites (since fans would No Award them) nor “burn down” the awards, since fans would have at least a couple of organic works to give awards to. No one foresaw the “griefing” strategy of nominating works whose mere presence on the finalist list would cast the awards into disrepute.

    All of the available data suggests that EPH isn’t going to work well enough to do that. It will prevent a slate sweep, for the most part, but many categories will only have one or two organic candidates. EPH+ (a modification to EPH) would change that to two or three, but even so the slates could still force about half of all nominees to be garbage like Space Raptor.

    So something additional is required.

  11. The biggest problem I’ve always seen with EPH and its variants is that it is too complex for most people to understand. I suspect that a significant number of those voting for it last year were doing so on faith alone.

    3SV does indeed add complexity to the system; however, each step of the process can be explained relatively simply (even IRV, although even that makes a lot of people’s eyes glaze over). One of the more simple explanations of 3SV is that “we’re adding the main No Award vote up to an intermediate round.” You could still vote No Award on the final ballot, but it seems much less likely that it would ever win if the finalists all had to pass a straight up-down vote at the Qualifying Stage.

    In fact, although it is not something I’d want to touch right now, assuming 3SV ever gets ratified, I’d consider removing the “No Award Showdown” rule, which complicates final ballot counting to little purpose in my opinion.

    If our only defense against Griefers becomes making the vote counting system impossible for most people to understand, we’ll eventually lose credibility even with those people who want to support the Hugo Awards. EPH is, in my opinion, perceived as a “black box” into which nominations go in and finalists come out, with “and then a miracle happens” written on the box.

  12. @Stoic Cynic: thank you for disproving Greg. I had not realized that the number of nominations to get on the ballot (absent obvious slating) was so low in so many categories — although I find the spread in slate-looking nominations interesting. (Possibly explainable by differences between Sad and Rabid Puppies’ lists, perhaps.)

  13. “There is no such thing as a fool-proof plan,” and there are some high powered fools out there.

  14. I’m not thrilled with the suggestion that EPH wouldn’t be necessary with 3SV, but otherwise I like it on initial read-through.

  15. Kevin Standlee: EPH is, in my opinion, perceived as a “black box” into which nominations go in and finalists come out, with “and then a miracle happens” written on the box.

    OK, so that’s what you think. Just the same, if you’re going to persuade Business Meeting voters, many of whom spent long hours last year and this studying the rules language and pushing people to work up simulations, you better not start by telling them “You just think EPH is magic.”

    Let’s not allow that to become the SMOF narrative.

  16. @Kevin Standlee

    Cubist and I wrote a song* that explains EPH in 3 minutes and 30 seconds or so, so it can’t be *that* complicated.

    And my estimate of the system is that if we made it worth their while the griefers could hold the top 15 slots in a lot of categories in the absence of EPH (or better, EPH+).

    Let’s have both. Suspenders and belt.

    *I wrote the lyrics; Cubist wrote the tune.

  17. “Let’s have both. Suspenders and belt.”

    This. Both EPH and 3SV. There is no magic in either. There is no blackbox in either. EPH is what lessens the impact of griefing in the more flat categories and also what lessens the impact of kingmaking.

  18. @Kevin Standlee: EPH is not that hard. Do you really intend to claim that most members of Worldcon are stupid?

  19. 1. EPH does have the appearance of black boxing the problem, if only because it “divides a vote”. This is not usual, nor customary, nor familiar. If one were discussing it in the abstract, no, it’s not that complicated. But when one discusses it in the context of how things have always been done, it becomes a complicated thing.
    My reading of reactions to EPH since it has been introduced included a lot of questions about its complexity and a lot of attempts to simplify explanations. Had this been done with anything but the generally way above average Worldcon audience, I think it would have died a-borning. If you go back over the discussion, you will see numerous statements that “EPH is too complicated”. I think it is pretty obvious that that is the prevailing perception. Voters (after lots of explanation) were willing to give it a try as the only seeming viable option, but not necessarily because they completely understood it or were comfortable with it.
    And insofar as explaining “How the Hugo Awards Voting Works” to new voters, well, the original “Australian System” was complicated enough for most American voters who have it drilled into them that the way voting works is, you tick a box, all the ticks are added up and whoever has the most ticks wins (usually Lymes, or maybe Rocky Mountain Spotted Fever). Having to explain “after tallying, the nominee with the least votes is dropped and everything is tallied again….) takes a while to penetrate. Now we’re adding: your initial vote is split up among the nominees you choose…just sounds completely foreign to most voter’s experience.
    I fear that new voters’ reaction will be “I don’t understand how my vote is counted, so I won’t bother to particpate” (or, translated into puppy speak: they’ve stolen my ‘right’ to vote for who I want to by hiding things behind a complicated algorithm).
    2. 3SV offers voters the opportunity to actively address bad actors. This is an important aspect of this proposal.
    3. Gaming likely popular nominees by putting them on a slate is easily addressed by the following:
    anyone who is not a bad actor and might conceivably be nominated for a Hugo Award sometime in future should make it abundantly and publicly clear that they do not want to be included in such activities, disavow such activities and wish to be removed from any slates they are placed on (whether those wishes are honored or not).
    I have a list of such statements, though it is far from complete. A resource containing all such statements should be developed that is easily accessible to all.
    Gaming is one thing, common sense, another. Most of us know when a slate nomination is a ploy – but absent statements by the nominee, we leave an opening for puppies to claim victory, which, however ridiculous and transparent it may be to non-puppies, can be used successfully by puppies for recruitment and further BS.

    Closing this loophole will strengthen 3SV; Worldcon and voters should strongly encourage those they may nominate in future to make such a statement.

  20. “Gaming likely popular nominees by putting them on a slate is easily addressed by the following:
    anyone who is not a bad actor and might conceivably be nominated for a Hugo Award sometime in future should make it abundantly and publicly clear that they do not want to be included in such activities, disavow such activities and wish to be removed from any slates they are placed on (whether those wishes are honored or not).”

    What if they disavow slates and still are placed on it? Still are voted on it? Are not aware of being on a slate? Live in another country with no knowledge of ameriacan slaters?

    This is a totally unworkable idea.

  21. Unless someone is actively enthusiastic about being on a slate, I am giving them the benefit of the doubt.

  22. Hampus: it’s not unworkable – numerous authors have already done it.
    If they’ve asked to be removed and are ignored, they still get a pass, and that pass has even more weight.

    Voters can ask those they wish to vote for; the community is concerned and already last year there were several instances of nominees being informed of their presence on slate(s) by concerned readers.

    All we need is a little bit more pro-active engagement with this and it will work: more folks stating that they simply will not play the game is the only real way to end this whole thing

  23. I’m in favor of not playing the game, but I have a different take on doing that.

  24. The problem with gaming the current system, as it stands, is not the presence of slates in itself. The problem is that it only take a small percent of the nominators (10% to 15%) to dominate the top 5 nominees in (almost) every category, a tactic slates have been using to dictate nominees to the members. With the top 15, yes it is theoretically possible for a combination of slates to dominate all 15 slots but at that point those slates represent a super majority of the WSFS membership, and in any fair system their choices are going to be what appears. It’s also a big enough number that a couple of smaller slates working together can’t do it, which is an issue with “4 and 6” by itself.

    Speaking of “4 and 6”: If the 4/6 is ratified this year, and the meeting ultimately passes 3SV as written then each nominator would receive 4 nominations in the initial round (the language dictating that is untouched by 3SV), and the new second round will place 5 nominees on the final ballot (because it completely strikes and replaces the relevant text). This would create a 4 and 5 solution. The meeting can, of course, amend the 3SV proposal to place 6 nominees on the final ballot (restoring 4 and 6), but its worth mentioning before hand.

  25. As things stand, I think we need both EPH and 3SV… going forward, I suspect the one that’s worth keeping, as an ongoing thing, is EPH. It’s a step that’s not particularly noticeable to the voters and nominators – it doesn’t entail any extra effort on their part – but it throws up an ongoing barrier against future slating, and that’s worth keeping in itself. (I don’t think the actual process of EPH is that hard to follow, although the process by which it gets its results – how it drives identical ballots to compete against themselves – is slightly more recondite. But even a non-statistician like myself can work it out.)

    3SV is extra hassle for everyone, though – certainly, it’s necessary now, and for the foreseeable future, since we have a jabbering primate on the sidelines who is willing to fling as much of his own excrement as possible, simply in order to disrupt the process. It’s a step that we might be better off omitting, though, in primate-free years.

  26. Christopher Hensley said:

    The problem is that it only take a small percent of the nominators (10% to 15%) to dominate the top 5 nominees in (almost) every category, a tactic slates have been using to dictate nominees to the members. With the top 15, yes it is theoretically possible for a combination of slates to dominate all 15 slots but at that point those slates represent a super majority of the WSFS membership, and in any fair system their choices are going to be what appears.

    Using the numbers you cite, it would only take 30% to 45% of nominators to dominate 15 slots. Is 30-45% what we’re calling a “super majority” now?

    And in some categories it would actually take far less than that. Look at last year’s Best Short Story nominations, for instance. The lowest ranked slate nominee had almost twice as many votes as the highest ranked organic nominee. A slate could have dominated all 15 slots in that category with only 231 disciplined voters. There were 2122 valid nominating ballots last year. That means that less than 11% of nominators could have dominated all 15 Short Story slots.

    Voter turnout for nominations almost doubled between last year and this one, and we won’t have solid numbers on what effect this has had on the longlist until August. But considering the data from previous years, I think it’s almost certain that “long tail” categories like Short Story, Novelette, Fan Artist, etc. would be dangerously vulnerable under 3SV if we didn’t also have EPH to moderate the effects of a slate on the longlist.

  27. 1. I’m sure Greg is right, but I can’t for the life of me see how what he said is an answer to me. My comment was addressed to a completely different point.

    2. Christopher Hensley:

    With the top 15, yes it is theoretically possible for a combination of slates to dominate all 15 slots but at that point those slates represent a super majority of the WSFS membership.

    People keep saying this, but I don’t see it. Three slates, each with 200 voters: 2000 voters overall. A perfectly possible scenario, and the slaters don’t have a majority. They can still take the top fifteen places.

    3. Further to what I said earlier: I think that the idea of ‘bringing the No Award forward’ won’t work perfectly. Last year, a lot of us were only prepared to No Award the slated works after reading them (or in the case of longer works at least sampling them). No one will be able to read all the slated works in the brief time allowed for a preliminary vote.

    4. And moreover, last year almost all the slated works were rubbish, so once we had read them, No Awarding them was easy. This year that isn’t so: in the bits that aren’t self-promotion or shock-causing, Mr Day seems to have taken his responsibilities as a kingmaker seriously, and chosen some decent works. If he continues to focus on this part of his mission, we may in future get list of works that are perfectly acceptable, and not the sort of thing you would reject out of hand, but still skew the ballot in a particular direction. 3SV does nothing to stop that. EPH does do something (though not enough).

  28. By the way, when we talk about three slates, do we actually mean one group of slaters dividing itself into three for strategic purposes, or do we mean three independent slates (Sad Puppies, Happy Kittens and Indifferent Piglets)? Because if the RP slate remains an effective presence for the next few years, I don’t think we can stop counterslates developing. (And EPH actually works very well with counterslates, sharing out places between them proportionately).

  29. @Christopher Hensley

    With the top 15, yes it is theoretically possible for a combination of slates to dominate all 15 slots but at that point those slates represent a super majority of the WSFS membership, and in any fair system their choices are going to be what appears.

    This is very much not true. It only takes 300 slaters to do it. It is debatable whether there really are 300 hard-core slate voters, but it’s not debatable that they could dominate lists of 15. To see this, just look at the historical nominee counts and see how few organic nominees got more than 100 votes. With 300 slate voters divided into three teams, every one of their nominees would have exactly 100 votes.

    Three hundred slate voters is a very small minority of the membership–not a super majority by any stretch of the imagination.

  30. @Andrew M

    I’m sure Greg is right, but I can’t for the life of me see how what he said is an answer to me. My comment was addressed to a completely different point.

    Looking back at it, neither can I. Oops. 🙂 Sorry about that!

    And EPH actually works very well with counterslates, sharing out places between them proportionately.

    In fact, it arguably works better with counterslates, since it (and EPH+) are based on voting systems designed to fairly allocate seats in multi-party democracies. The Rabid Puppies are a small minority, but there is no other party. If we had three or four other “parties” (groups proposing their own slates) then EPH and EPH+ would do a fine job of allocating nomination slots between them.

    I can’t see counter slates working very well, though, since slate discipline implies voting for things without reading them, and I don’t believe very many fans are really up for that, It sort of defeats the whole purpose.

  31. I think some folks don’t realize how small the nomination counts in some categories are. The field is big enough and the voting pool diverse enough you don’t see huge nominating numbers. This means a very small group acting in concert can dominate the field. Yes, even the long list. It’s far from a super majority. Sorry for the post length but for a sampling here are the top 5 nomination counts for 2013 and 2014 (excluding withdrawals and disqualifications):

    Novel
    2013: 193, 138, 135, 133, 118
    2014: 368, 184, 160, 120, 98

    Novella
    2013: 101, 91, 90, 74, 69
    2014: 143, 127, 111, 106, 86

    Novelette
    2013: 89, 62, 54, 45, 38
    2014: 118, 92, 79, 75, 69

    Short Story
    2013: 107, 38, 34, 30, 28
    2014: 79, 73, 65, 43, 38

    Related Work
    2013: 90, 88, 65, 40, 39
    2014: 89, 83, 80, 77, 52

    Graphic Story
    2013: 127, 98, 43, 32, 26
    2014: 89, 83, 80, 77, 52

    Dramatic Long
    2013: 383, 210, 179, 158, 141
    2014: 401, 278, 183, 174, 138

    Dramatic Short
    2013: 95, 93, 93, 91, 89
    2014: 312, 193, 127, 47, 38

    Editor Short
    2013: 154, 119, 116, 114, 108
    2014: 182, 115, 96, 94, 86

    Editor Long
    2013: 75, 74, 71, 63, 50
    2014: 169, 118, 63, 54, 48

    Pro Artist
    2013: 124, 62, 62, 54, 53
    2014: 129, 109, 51, 50, 49

    Semi-Prozine
    2013: 172, 123, 118, 113, 75
    2014: 162, 108, 97, 82, 60

    Fanzine
    2013: 93, 72, 57, 50, 45
    2014: 107, 83, 61, 61, 53

    Fancast
    2013: 91, 66, 63, 62, 35
    2014: 81, 63, 48, 41, 35

    Fanwriter
    2013: 116, 61, 50, 47, 44
    2014: 94, 80, 73, 68, 62

    Fan Artist
    2013: 49, 40, 32, 29, 28
    2014: 46, 39, 38, 35, 31

    Campbell
    2013: 89, 85, 43, 37, 34
    2014: 156, 146, 77, 73, 70

  32. Steve Davidson: What would you say to the following scenario?

    a. VD names five works as nominees for Best Novella.

    b. The authors of these works all say, clearly and publicly, that they do not wish to be on the slate and do not wish to derive any benefit from it.

    c. Accordingly, no one votes them down.

    d. Hence, VD’s five picks end up on the ballot. He has succeeded in excluding from the ballot works he does not like, or likes even less.

    If we vote down anything that is on a slate, he can use the ‘poisoned cup’ strategy. If we draw distinctions, the works we favour will still be getting an advantage from being on the slate. So he wins either way.

  33. @Andrew M

    If we vote down anything that is on a slate, he can use the ‘poisoned cup’ strategy. If we draw distinctions, the works we favour will still be getting an advantage from being on the slate. So he wins either way.

    Maybe we’re looking at it the wrong way. If the works that actually win the award are works that really deserved to get awards, then maybe that’s good enough. If someone manipulated the awards, maybe we should only care if that either resulted in no award-worthy works on the ballot or else resulted in an unworthy work winning an award. If that’s all we’re trying to achieve, I think EPH as it stands probably accomplishes it.

    It would be nice to do more (and I think it’s possible to do at least a little better), but maybe it’s not as urgent as we’ve been thinking.

  34. @Stoic Cynic

    I think some folks don’t realize how small the nomination counts in some categories are.

    Thanks for posting that. Everyone should look at how few three-digit numbers there are in those lists. Those are the only organic results that would survive in the top-15 list in the face of a disciplined 300-person slating operation. This clearly shows why 3SV as described above just doesn’t meet its own goal of guaranteeing more than one or two acceptable finalists per category.

    As I think about it, what I believe would work would be to use EPH+ to construct the top-15 list, and then make the middle round a simple vote. That is, let everyone vote for up to 5 of the 15 and then just count the total votes. It makes the process of picking the original top 15 a bit obscure to people, but still better than a purely juried selection, whereas the process from that point forward is as easy to understand as the old system was.

    If 3000 voters averaging 3 nominations each spread their votes evenly across 15 organic candidates, that would come to 600 votes each, which would swamp 300 disciplined slate voters. And “evenly” is the worst case.

  35. This is very much not true. It only takes 300 slaters to do it. It is debatable whether there really are 300 hard-core slate voters, but it’s not debatable that they could dominate lists of 15. To see this, just look at the historical nominee counts and see how few organic nominees got more than 100 votes. With 300 slate voters divided into three teams, every one of their nominees would have exactly 100 votes.

    Combining this with 4/6 (which is my personally preferred solution at this time) or even the default 4/5 further dilutes their effect. I will admit it is not a perfect solution, but I have yet to see a perfect solution.

  36. Greg Hullender: If the works that actually win the award are works that really deserved to get awards, then maybe that’s good enough.

    No, really not. For one thing, this allows the slate to skew the results in a particular direction; there can be a set of finalists which are all, as individuals, perfectly worthy, but do not give a fair representation of the field (or that part of the field which is of interest to Worldcon members). This year’s ballots in Novella and Graphic Story may well be examples.

    But also, the Hugo results are meant to be useful to the wider reading public, as a a recommendation. If we just want to find a worthy work, well, the Nebulas can point one out (or the Clarke, or the WFA, or…). The Hugos should supply a distinctive perspective. For that I think we really need results that are decided on by the members of Worldcon, not partly by the members of Worldcon and partly by VD. An awards process that is partly controlled by VD would be very hard to justify to the wider world.

  37. As I think about it, what I believe would work would be to use EPH+ to construct the top-15 list, and then make the middle round a simple vote. That is, let everyone vote for up to 5 of the 15 and then just count the total votes.

    This has been discussed, and it has some clear advantages over the downvote option; in particular, it would eliminate any advantage the slate had, beyond bringing the works to light in the first place, whereas on the downvote option slated works, if not downvoted, carry their slate advantage into the next round. But it foundered on the problem that we would have to read all the works we were upvoting, which would not be possible in the time allowed.

  38. @Greg

    ….make the middle round a simple vote. That is, let everyone vote for up to 5 of the 15 and then just count the total votes.

    Which would be like the “Double Nominations” proposal. And it was generally concluded here that it would make participating in the second round difficult due to lack of time to become familar with the long list.

  39. @Stoic Cynic

    The numbers you show are why I believe we need EPH or better EPH+ as well as 3SV. With 300 RP split 3 ways they can come close to dominating the top 15 spots in all but novel, dramatic presentation, and possibly novella. If we go back looking at numbers for 2010-2012 it’s even more obvious how small a group can dominate the top 15 spots.

    —-

    Right now we are on an upswing in member participation. New proposals need to harness good actors participation. Proposals can’t overlook bad actors ability to do much harm with small numbers concentrating nominations and their different win conditions.

    Unfortunately I believe we need 3SV. But I don’t believe it can work without EPH/EPH+.

  40. As for the small voting pool issue, that is a known issue. It’s one reason why there isn’t a perfect solution. You have high participation categories, like Novel, and low participation categories like the Editor awards. You also have awards with the nominations spread over a small field (Dramatic Presentation, Long) and awards with nominations spread thinly over a wide number of works (Short Story).

  41. @Christopher Hensley

    That’s why you need both EPH and 3SV. Either by itself is a poor solution. Together though: EPH ensures enough organic works make it to the qualifying stage to have a viable pool of works for voters to filter. It’s about as effective an algorithmic solution as we’re going to find. Personally I think we should be looking at human judgment at the nominating stage but that is outside most folks comfort zone. Since politics is the art of the possible EPH and 3SV together are the current best option. Dropping EPH in favor of 3SV on the the other hand is simply reckless and likely a mistake we’ll regret down the line.

  42. @Anreww M and Laura

    Which would be like the “Double Nominations” proposal. And it was generally concluded here that it would make participating in the second round difficult due to lack of time to become familar with the long list.

    Yeah, I remember thinking that too at the time. A few things have changed my mind.

    First, given that it starts in February, we could allow three months for this phase followed by three months for the final vote. That’s more than enough time, even for people who really wanted to read everything

    But there’s no reason to ask or expect people to read everything. No more than the current system expects people to read thousands of stories before nominating. Most folks would find a few things in the list that they’d already read, and we’d mainly be asking them to see if they could find enough award-worthy works in the top 15 to get that up to 5. People would easily find lots of guidance from different reviewers and pundits as to which of the fifteen were most worth considering.

    I think we were too quick to give up on this idea.

  43. Double Nomination was never about reading everything. It was still only a nomination round. The problem was mainly that it was not effective enough in keeping pure abuse off the ballot.

    At least that was the reason I changed my mind.

  44. I didn’t really think it was necessary to read everything for any long-list variation, but some people expressed that concern with DN.

  45. @Hampus Eckerman

    The problem was mainly that it was not effective enough in keeping pure abuse off the ballot.

    Okay, so what was your reasoning? The way I’m thinking, provided you used EPH+ to generate the top-15 list, there would always be at least 5 organic nominees there. Then, even doing simple approval voting (i.e. “the old way”) on the top-15, the membership’s superior numbers should be enough to prevent any slate nominees (other than “hostages”) from making it to the 5 finalists.

    The only hole I can see is if most members didn’t bother to pick 5 nominees from the top-15 list. What am I missing?

  46. I don’t think anyone suggested you have to read everything. What you do have to do, surely, is read the things you vote for. That’s five things in each category, in what up to this point was always envisaged as two weeks – and without the help of a voter packet.

    (And don’t say ‘but surely you’ll have read a lot of them already’. In many cases we won’t. A lot of us don’t regularly read short fiction, and don’t read novels when they are new. The only eligible things we will have read at this point are things we deliberately checked out in order to nominate; if our choices did not fit those of the majority, we will be stuck.)

  47. For what it’s worth, my opinion: I think some means of vetoing obviously abusive nominations is needed, and 3SV looks like a reasonable effort in that direction. It appears to set a pretty high bar, in terms of the number of votes required to “reject” a nomination, which might have disadvantages. Anyway I’d probably support this as well as EPH.

  48. @Laura

    I didn’t really think it was necessary to read everything for any long-list variation, but some people expressed that concern with DN.

    Sigh. I might have even been one of them. If so, I apologize.

Comments are closed.