By Kevin Standlee: In light of the revelation that modeling of E Pluribus Hugo does not result in quite the “magic bullet” that some may have hoped for, we may need to consider other changes to the Hugo Award voting system to deal with bad actors deliberately setting themselves against the wishes of the majority of the voting members of the World Science Fiction Society. Over the past few weeks, I have written up descriptions of three different proposals that attempt to deal with bad actors in different ways, and to make it more likely that the results of the Hugo Awards represent the wishes of the majority of the participating members, without the members having to resort to the 16-ton anvil that is No Award.
All of the proposals below are compatible with either of the proposals up for ratification this year (EPH and 4/6). They are not necessarily compatible with each other. None of them require a Very Strong Administrator picking and choosing individual members’ ballots or disqualifying individual finalists on what I call “ideological” grounds. All of them aim to give the majority of the members a strong voice in picking the Hugo Awards.
In this overview, try not to get too deeply bogged down in specific details. For example, all three proposal refer to the “Top 15.” This is a reference to the “long list” currently defined in the WSFS constitution thusly:
3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, . . . places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes.
Therefore, the “top 15” can be defined in different ways, and there’s a reasonable argument to be made for many of them. Don’t get too tangled up in specifics. These proposals have general principles, and if you’re agreeable to the general idea of any of them, then we can discuss the specific values for some of the blank spots in them.
In the list below, the links lead to my LiveJournal where each proposal is listed in more detail.
3-Stage Voting (3SV) adds a new round of voting to the Hugo Award process, called “semi-finals,” between the existing nominating ballot and the existing final ballot. In 3SV, the top 15 nominees (including any ties, and see the warning about specifics above) are listed in a way that doesn’t show how many nominations they received. The members (supporting and attending) of the current Worldcon (not the previous and following Worldcons) are presented with this list, with a question on each of the fifteen semi-finalists in each category: “Is this work worthy of being on the Final Hugo Award Ballot?” with the choices being YES, NO, and ABSTAIN.
If a sufficient quorum (which is why counting explicit abstentions is important) votes, and if a sufficient number vote NO, that semi-finalist is disqualified from further consideration. As currently proposed, the necessary NO vote is “more NO than YES votes,” but the exact amount needed to disqualify is negotiable. Remember, even if every vote is NO, if a quorum (minimum number of voters) doesn’t participate, the work cannot be disqualified. (This makes it difficult for a small group to campaign against a work.)
At the end of the semi-final round, any works disqualified by the members, withdrawn by the nominees, or disqualified by the Committee on technical grounds is out of the running. From among the remaining semi-finalists, the five that got the most votes in the nominating phase become the finalists, and the final ballot continues as it currently does. Note that in this case it doesn’t matter how many YES votes a semi-finalist got, only that it didn’t get a negative majority.
3SV effectively moves the votes on NO AWARD to the semi-finals, although it would remain a candidate on the final ballot. It allows the members of the current Worldcon (the ones who will be voting on the final ballot) to decide in advance which works they think deserve to be on the final ballot. It does this at a price, however, and that price is to carve 6-8 weeks out of an already relatively crowded schedule. The proposal moves the deadline by which you have to be a member in order to nominate up by a month, and in practice would require the nominating deadline to be earlier in the year.
Another drawback of 3SV is that it triples the number of nominees that the Worldcon Committee (the Hugo Award Administrators) need to vet and contact. However, as a trade-off, it gives the administrator roughly three times as much time to do this contact work, and makes the vetting and contacting process public. That is because the Administrator would not need to contact semi-finalists in advance of announcing the “longlist” of semi-finalists. While the semi-final ballot runs, the Administrator would be contacting semi-finalists to give them an opportunity to withdraw from consideration in the final ballot. Administrators could put out a public appeal if they are unable to contact a given semi-finalist. In addition, the Administrator can do eligibility confirmation in public, with the help of the many other people who will undoubtedly be checking over the list and asking questions. Should a semi-finalist be disqualified on technical grounds, the semi-finalist would not be replaced on the semi-final ballot. Ineligible works show up on the existing Top 15 lists now, and the semi-final round would be too short to allow for replacing longlist slots.
An incidental feature of 3SV is that the finalists would not know they’d made the final ballot until the shortlist was announced. They’d know they were semi-finalists, and during the semi-finals they would have had time to decline if they so choose, but it would be impossible for them to leak being a finalist because they would not know it themselves.
Additional drawbacks to 3SV include the fact that it is explicitly negative. It is a place where you vote against things. Some people are philosophically opposed to “down-voting” works. Furthermore, should the known bad actors stop trying to game the system, there is little need for this semi-final round, and you might find people not even bothering to participate in it, which might result in complaints that we’d added complexity and expense for no obvious reason.
3SV is a “vote against stuff” system. An alternative to it was the second proposal.
Proposal 2: Double Nominations with Approval Voting
Double Nominations with Approval Voting (DN/AV, sometimes just DN) is similar to 3-Stage Voting, in that there would be a semi-final round with the Top 15 (see warning at the beginning of the article) nominees listed in an order that would not reveal how many nominations they received. Only members (attending and supporting) of the current Worldcon would vote at this stage. But instead of voting against semi-finalists as you do in 3SV, you would vote for those semi-finalists who you think deserve to be on the final ballot. You could vote for one or all of the semi-finalists. The version as currently proposed included a single write-in slot as well, primarily as a safety valve. In this case, the number of votes a semi-finalist gets here is critical, because only the top five would continue to the finals.
In 3SV, the number of YES votes doesn’t matter as long as there are fewer NO votes (or insufficient ballots cast to qualify the election at all). The relative number of YES votes doesn’t matter as long as the semi-finalist isn’t disqualified, because it’s the original nominating ballot count that sends works on to the final round if they survive the weeding-out process of 3SV’s downvotes. However, in DN/AV, the five works with the most votes in the semi-final round go on to the final ballot, and the number of nominating ballots cast to get the works onto the longlist is irrelevant. Some have called for nominating counts to be used as a tie-breaker, as they envision a 15-way tie for the final ballot. I personally think this unlikely, and there have been as many as eight finalists on a Hugo ballot due to ties for the final position, so I think we could live without a tie-breaker.
All of the extra administrative issues of 3SV are shared by DN/AV, so I won’t go over them again here.
DN/AV eliminates the “negative” aspect of 3SV, in that you vote for semi-finalists, not against them.
Some have suggested that there’s an implication that “you need to have read all of the semi-finalists in order to vote on this ballot,” although I don’t think that is true in either case. DN/AV is a second nominating round. Just as the current nominating ballot makes no pretense that you should have read everything published last year, DN/AV doesn’t expect voters to have read all fifteen semi-finalists, but instead to pick those that the voters think worth considering on the final ballot, knowing that only the top five will appear there.
The biggest advantage to either 3SV or DN/AV is that they put the decision of “what to have on the final ballot” in the hand of the members who will be voting on that final ballot, and that the majority will of those voters will prevail. Neither system is particularly susceptible to gaming by small minorities. The biggest drawback to either of the first two proposals is that they add an additional round of voting, with administrative overhead and complexity.
Some people have asserted, with various degrees of strength, that the Committee (Hugo Award Administrators) should simply ignore “slate voters” or disqualify “obvious slate-generated finalists.” In my opinion, such proposals are hugely problematical, in that they give Administrators authority they have never had in the entire history of the Hugo Awards. However, there is one historical precedent to which we can look when a group of bad actors appears to have forced a work onto the ballot, and the Worldcon Committee tried to ameliorate the action without actually disqualifying anyone. We’ll consider this in the final proposal.
In 1989, the Hugo Award Administrator noticed an odd situation, where a set of nominating ballots arrived in close order with a single nomination cast in a single category. This was enough to place a finalist on the ballot that seemed unusual to anyone who had been watching the kinds of things that had made the shortlist in recent years. The ballots all having included membership payments in the form of consecutively numbered money orders further raised suspicions. According to the coverage of the situation in File 770 at the time, the 1989 Worldcon committee said in a statement at the time that they didn’t do any investigation, but that one person wrote to the committee to inquire why he had a membership when he hadn’t paid for it. (After they made a public statement, some fans contacted the committee claiming to have cast these votes with innocent intent.) In any event, there has always been a strong ethos in the Hugo Awards to allow the membership to speak, not a small committee. On the other hand, this case seemed to be doing someone out of a Hugo Award finalist slot unfairly. The Committee took the unprecedented step of adding the sixth-place nominee to the shortlist. Not too long thereafter, one of the six finalists (the one that seemed to be odd compared to the others) withdrew. There is no evidence or suggestion that the finalist who withdrew had anything to do with the string of bullet-voted ballots. At most, this was a case of enthusiasts with more money than common sense or ethics.
The final proposal that I’ve written up would explicitly authorize the Worldcon Committee (in practice, the Hugo Award Administration Subcommittee set up by the Worldcon Committee) to take the action that the 1989 Worldcon did whenever they think that there was a pattern of unethical voting in the nominating round. The Committee would be authorized to add up to two additional finalists from among the Top 15. They would not reveal which finalists they added until after the Hugo Awards ceremony, when it would be included in the post-ceremony detailed results.
I have no expectation that the current people who have typically been part of the Hugo Administration Subcommittee would be the ones who would make the actual Plus 2 decisions. I expect that any sensible Worldcon Committee would recruit additional Administrators for their literary judgement, just as the World Fantasy Award does. Any Administrator, no matter whether they were recruited for computer skills, literary skills, or public relations skills, would be ineligible for that year’s Award just as the current Administrators are.
Should the Administrators determine that there was no need to do so, they could choose to not add additional finalists. Thus, if known Bad Actors desist from their wrecking ways, the Administrators could simply continue running things as we have done in the past, leaving it up to the five highest pluralities of nominations. This would simplify administration and require relatively little change.
This proposal allows the Committee to pull from the Top 15, rather than simply the next two finishers in the nominations, to minimize multiple-slating attacks on the system that would try to dominate the top seven rather than the top five positions. While 20% of the electorate has been able to dominate the first five positions relatively easily, it seems unlikely that such a small group could dominate the top fifteen unless their voting power grew to a majority of the entire electorate. Inasmuch as solutions that represent a majority of the electorate (even if you personally dislike the result) are not contrary to democratic process, there is nothing in this proposal that tries to “defend against it.” As I’ve said above and will continue to say, if you can command a majority of the voters, you get your way, even if I don’t like it. It is minorities dominating the process that troubles me.
Note that there’s nothing magic about adding two additional works. It could be one or more. The exact value is debatable. However, consider that we do hope that most people read all or most of the finalists, and therefore making the shortlist too long works against any good you might get from adding (say) five extra works to the ballot.
Broadly speaking, Plus Two is compatible with either (but not both) 3SV and DN/AV. That is, this proposal can be considered separately from either 3SV or DN/AV, whereas the first two proposals above are antithetical to each other, and only one of them could be reasonably considered at a time.
The biggest advantage of Plus Two is that it’s relatively simple, does not add a lot of administrative overhead, and allows the existing two-stage process to stay in place, with only a relatively minor change of adding works to the final ballot. It also adds the “human judgement element” that it appears that many people seem to think is necessary to combat bad-faith efforts to sabotage the Awards.
The biggest disadvantage is that it gives Administrators an authority that they’ve never actually had before, and that they have used only once before, without any explicit sanction. Administrators may be reluctant to serve on the Hugo Award Administration Subcommittee if they know that they may be responsible in some way for adding works to the Hugo Award ballot.
Conclusions
There are no “magic bullets” when it comes to tinkering with the Hugo Award rules. None of the proposals here is a perfect fix. Indeed, per Arrow’s Theorem, there is no such thing as a perfect voting system. However, we can try to move things around and minimize unfairness, usually at the expense of additional complexity. The political question then is how much complexity we can tolerate to improve perceived fairness.
The Instant Runoff Voting system that we use on the final ballot is a case of trading complexity (IRV boggles the minds of people who reject anything other than First Past the Post voting) for fairness (IRV usually returns the least-disliked candidate in an election, rather than the one with the largest plurality; in a field of more than two candidates like the Hugo Awards, it usually returns a consensus winner, not just a strong front-runner). Should we decide that the changes we’ve started with E Pluribus Hugo and 4/6 that are up for ratification this year are insufficient to tilt the field back toward perceived fairness, it behooves the members of WSFS meeting in Kansas City this summer to consider additional changes now, not later.
WSFS rules are intentionally complicated to change, in order to prevent concerns of a day or even of a single year to overwhelm the process. However, that doesn’t mean that we cannot start queuing up additional changes now while we continue to monitor how things proceed, in order to protect our own longer term interests.
I expect at least one of the proposals outlined here to be on the agenda of this year’s WSFS Business Meeting. We might even have all three of them.
Discover more from File 770
Subscribe to get the latest posts to your email.
Toward a more formalized version of GTFO-A/B:
1) The current year’s WorldCon shall appoint a moderation panel (consisting of?) to consider the nominations of the following WorldCon.
2) Prior to publication the Hugo administrators shall provide the moderation panel the results of the nominating phase.
3) The moderation panel may recommend removal of any works that, in their sole judgment, might tend to bring discredit on WorldCon or its members.
4) Removed works shall be replaced by the next highest finisher on the nomination list.
5) The moderation panel may recommend addition of up to two works that, in their sole judgement, are worthy works that may have been overlooked by nominators.
6) If the moderators recommend any additional works or removal of works they shall provide the moderated list to the administrators within (a timeframe)
7) If the moderators present a moderated list to the administrators, the administrators shall present the members of this year’s WorldCon with memberships as of (a cutoff date) with both the moderated and unmoderated short lists.
8) The administrators shall conduct a vote of those members within (a timeframe) to accept or reject the moderated list. The majority of votes shall determine which list, in its totality, to use for determining the Hugo awards for this WorldCon.
9) The decision of which list to use shall be binding on the administrators.
10) The administrators shall announce the Hugo finalists within (a timeframe) of this vote.
This might or might not need a quorum. Items 3 and 4 deal with the RP libel probem. Item 5 is designed to deal with the SP / Hubbard weak works problem. I think there are checks and balances throughout.
I know I’m in the minority here in preferring something like this. Thanks for reading and giving reasonable consideration in any case.
David Goldfarb: Ah, fair enough.
Greg Hullender: OK. I was not quite clear whether you were still looking at revoking memberships, and the word “ban” seems a bit loaded and confusing. I suggest “declare ineligible.”
Greg: I’ll explain what I think is the (minor) flaw with what you said after I’ve posted graphs I can point to.
And to be fair GTFO-A/B is pure snark as a title. As would be my other alternate Troll-Be-Gone (TBG). For serious purposes, until something better is proposed, let’s call it MSMC for Moderated Shortlist for Member Consideration.
Brian Z: The constitution says the Worldcon committee is required to do two things: hold the business meeting and give the Hugos. Trying to destroy the Hugos is an act of opposition to what the committee must do. The authority to oust members engaged in that is implicit, comparable to Lincoln’s measures to save the Union.
@Stoic Cynic
I think you can dispense with the quorum because in the worst case, you just end up with what we’ve got this year; it doesn’t really make things worse. It definitely has the virtue of being really simple for voters: a single up or down vote in every category.
On the downside, it has all the costs of creating a committee (both real and imaginary). If you’re going to spend that effort, it seems to me like you’d want a more powerful committee that could tweak parameters and do other useful things. Also, the effort to compare the lists and make that yes/no call seems like it might be about the same as the effort to pick items to downvote with 3SV.
Proposing ideas here is definitely more painful than it ought to be. Some people seem to see any proposal as an opportunity to try to hurt the feelings of the person making it.
What’s worse is that almost no one views flaws as something to try to fix. If someone made a proposal that said “this would apply to all nominated works” then a nice response might say “you need to say ‘nominated works, individuals, or companies'” but rarely does anyone do that. Instead someone will post “that fails because it doesn’t include companies!”
It’s as though almost everyone sees this as a competitive, not a cooperative effort. That’s a little disappointing. Even a really bad idea can contain the seed for a really good one, and the person proposing it deserves to be treated with respect regardless.
Destroying the Hugos is so 2015. Reynolds, King, Hao Jingfang? A book about Gene Wolfe? Jerry Pournelle? I’m not too tingled about Tingle, but some people like it. I’m on board for DN if all nominators participate. But banning people who wrote down the “wrong” things on their ballots?
@Jameson Quinn: If only one “plus” will be attempted, I agree that it should be EPH+, because it does more. This is from your earlier comment that the “+” in EPH+ would contribute 1 organic nominee, whereas the “+1” in 3SV+1 contributes 0.5 to 1 organic nominees.
(I do realize these numbers were guesstimates and we should wait to see the graphs for EPH+).
But this reasoning:
seems iffy to me. I saw several comments (early on) saying that one of the reasons they liked 3SV+1 was because it sounded more positive than 3SV. If the “+1 ” makes 3SV sound more appealing to a lot of people, it could make it easier to pass, not harder.
(Of course there’s a big “if” there and my personal bias could be blinding me. Just saying this “more positive” thing shouldn’t be dismissed out of hand.)
Even if there isn’t a sufficient groundswell for some sort of firewall-like panel at this point, it is worth having the conversation now anyway.
The fear is such a body having unrestricted power but without discretionary ability it can’t respond to new attacks.
I’d suggest three specific categories of works it can remove (but with broad ability to determine what does or doesn’t fall in each)
1. Ineligible works – a power the admins already have.
2. Works intended to harass or defame individuals or groups. The kind of code-of-conduct issues that we’ve been discussing. We got “If You Were An Award My Love” this year – it is a genuine issue.
3. Works intended to disrupt the PROCESS of the awards. I don’t want to catalogue the various ways that a griefer can do this but one example obvious enough that it I don’t think I’d be giving VD ideas: imagine if VD either via Castalia (or even more surreptitiously) put out a book in late 2015 called “The Fifth Saeson” by “N.Jemsin” (typos intentional for once). It isn’t going to win anything but by getting his cronies to vote (perhaps secretly) for it they get to make everybody’s life harder.**
There is another issue that is already within admin powers but which they may lack the right tools to act on easily:
4. Cheating. Apart from one famous example, it hasn’t been a major issue but if we assume griefers aren’t just going away it could be a bigger issue.
Constrained to these areas then I would suggest that the ratification step you include would not be necessary (and perhaps inappropriate.
Another possible check/balance step (not one I endorse just throwing it out there) is that a removed work would be re-eligible for a bonus year, so that it’s defenders can make a case for it.
*[Yes, somebody could make an argument that Mixon’s report on Requires Hate might fit within that criteria (I disagree but lets go with that for arguments sake) but 1. it wasn’t nominated anyway (Mixon won best Fan Writer not BRW) 2. it wouldn’t have been the end of the world if it had been nominated and then removed by a panel. Nominating a work for a Hugo isn’t the only way fandom can express themselves on a topic.]
**[Yes, “There Will Be Walrus” is also a spoof title – but I’d have zero issues with a panel chucking that out if there was a “Best Anthology” category or for point 2 reasons – or actually for any reason at all 🙂 ]
What I am for is:
A) Let administration throw out ballots for blockvoters.
B) Not paying back their membership dues.
Because there should be a written expectation that voters are expected to vote as individuals.
And I do not think there should be a hard definition for what a block is, that would only lead to plays of wack-a-mole.
But I do not think there’s possibility to get support for this.
@Jameson Quinn
I am all for EPH+ going through stage 1 this year. I have mixed feelings on +1 now that I understand it better.
@Stoic Cynic MSMC for Moderated Shortlist for Member Consideration
I like this with @Camestros Felapton’s additions. It’s a good compromise.
I think Stooc Cynics idea absolutely is worth considering. Because the administration could see what works only could game their way to the ballot using blockvoting. And also see what works would have made it anyhow.
Camestros Felaptons addendum is good, but needs to have addendum that works who only could be gamed onto the ballot by blockvoting can removed. In an ordinary year, the lists would be identical with no reason to vote.
Only two questions:
1) Would it raise the toxicity or ego of narcissists to single out their works?
2) Should voting be on list as a whole or on individual works?
Anyhow, this mix of administration and membership ratification removed the fear that griefers may try to organize No Award campaigns.
@Hampus Eckerman
Supposedly Vox Day did attempt a No-Award campaign against Best Graphic Novel last year. the puppies only had a single nominee in that category, and it did indeed have roughly 500 “excess” No Award votes beyond what I’d have expected it to have, given the historic No Award rate in the category and the per-year increase in No Awarding based on the Best Fan Artist category (which had zero puppy nominees in 2015).
They won’t attempt No-Award campaigns because they simply lack enough members to pull one off.
@Hampus Eckermann
I like Camestros’ modifications to MSMC as well. I think they strengthen it significantly.
Voting the lists as a unit I think is important. It removes gaming or campaigning toxicity that voting on individual noms might create. It also helps minimize any works creator feeling like they were singled out. It also simplifies the question for nominaters a lot.
To be honest, I wasn’t considering block voting as a criteria at all. Yes, practically that is what we are looking at but harassing or disruptive works can get on a ballot without identifiable block voting. Griefers don’t need to vote for a whole set of works or collaborate across categories. They only need one turd in the punch bowl.
The disruptive works don’t even need a lot of votes to make life difficult for any process that needs a data cleaning step.
I think MSMC is not a bad idea, but I do worry about having too many proposals at the BM. If we can get a rough consensus which is better, we should not propose both MSMC and 3SV/3SV+1.
Personally, I have a weak preference for 3SV. But then again, I’m not a regular worldcon member, and I probably won’t be this year unless EPH+ is on the agenda.
What do other people think? (Try to give the benefit of the doubt to both proposals; that is, assume that the wrinkles will be worked out relatively well and the final version of the proposal does not have superficial defects.)
I’ve been thinking of whether you could make the argument that EPH+ amounts to reducing the impact of the rule (allowing it to be approved in Kansas City and take effect in Helsinki), but, unfortunately, I think I’ve proved the opposite.
EPH uses weights of 1, 1/2, 1/3, 1/4, and 1/5, whereas EPH+ uses weights of 1, 1/3, 1/5, 1/7, and 1/9. What would “EPH-0” with weights of 1,1,1,1,1 look like? The scores and vote totals would be the same, and winnowing a nominee would not change the sort order of the list. The result is that EPH-0 would produce the same result as the system we have today. You could therefore make a good argument that increasing the weights would amount to a lesser change, but decreasing the weights amounts to a greater change, which would require two years for adoption.
So I don’t see how EPH+ could be approved in Kansas City as a modification to what was passed at Sasquan. It’ll need to be a new proposal.
If anyone can poke holes in the argument above, I’d love to hear it. Rarely would I be more happy to be proved wrong . . .
@Camestros Felapton
So does your modification to MSMC mean that the committee would simply disqualify works according to your criteria and there would be no vote by the members? I agree that that would be much simpler, but it seems to radically change the nature of the proposal. Or am I misreading you? Perhaps it would help if someone would restate MSMC with your modifications included.
@greg: That’s well put. And it raises an interesting (though irrelevant) point. EPH-?, using weights of 1,0,0,0…, would be IRV (with a strange tiebreaker). So arguably, if we’d passed a resolution to use IRV in the first round, we could change it this year to either EPH+ or EPH.
Cute.
@Greg Hullender
That was a possible part of his changes. That part I think can’t go in. The members don’t want to surrender authority and I think a vote has to be part. I’m out of pocket. If no one else offers a consolidated revised version I’ll look at it when I get home.
@Jameson Quinn
I’m thinking that the proposals before the business committee should be simple 3SV and EPH+ and that’s it. I’d certainly vote for MSMC if it came to a vote, but I fear that the committee aspect makes it too heavy to win acceptance and that it might drag 3SV down in the process, since the two are incompatible.
I still think the Nomination Blacklist would be a good idea, but not enough to try to get it onto the ballot without some evidence that someone besides me thinks so too. 🙂
I think it is better without a member vote for the same reason we don’t ratify questions of eligibility. The Martian wasn’t eligible – that was sad but that’s just how things are. I’m thinking of this as extending what can be classified as ineligible but on the underlying principle of allowing the awards to work. For ineligibility it is best to act swiftly and categorically – that way people get over it quicker* That is another reason why I didn’t list slates as a reason for disqualifying works and also why it is works/nominees rather than ballots or memberships. It is clarifying and extending an existing function rather than granting a complete new power.
Having said that…any such group with such powers will be regarded rightfully with concern. Hence people may prefer some kind of accountability step. But…then we are heading back to longlist type proposals. I’m dithering and Stoic Cynic made a good case.
*[OK, yes, I know people are still cross about The Martian, but it illustrates that the Hugos already take works out of the running for other reasons. Malice seems like a more ethical and reasonable case for disqualifying something that because of an unconventional publishing history]
Greg Hullender:
Sorry, I was unclear. I meant No-Award campaign in the second phase of 3SV.
Jameson Quinn:
I as a person would prefer MSMC, because there is less risk. Less possibility to try to game the system. And still puts everything for the members to decide. For me it is not about if a work is about harassment in this case, but about if the work has been slated on to the ballot through block voting. It is my belief that works of harassment would not be put on the ballot without slates.
I see this as the system with best possibilities to block out slates without given griefers new ways to exploit it. It is also my belief that this system would stop the need for trimming or addendums that have been discussed for all other systems.
But my preference is very much dependent on what the administrators themselves would think about this.
I should also note that with the changes I suggested the panel would not have had the power to do much about Sad Puppies 3 (except maybe Wisdom From…)
I’m assuming No Award, EPH and the ire of outraged fandom is a sufficient detterent for Sads.
@Greg Hullender:
What about my idea of requiring a vote to eliminate a work? A reasonably low threshold to trigger a vote, but a high threshold for the work to actually be eliminated (eg 2/3 majority of people who take part in the elimination vote).
@Camestros Felapton
I think that’s where the +2 comes in. It provides a bypass to work around weak works and there’s still EPH and No Award as needed. Though hopefully with a bypass No Award becomes a rarely used option again because there would be alternate choices.
@Stoic Cynic
Yes! +2 is a power that admins have used before in response to suspected cheating. Also it can be codifying as a way of making the set of nominees more representative of most voters (i.e ethical issues aside, the problem with slated nominees is that they collectively represent fewer voters than non-slates)
@Bartimaeus:
Where is SV3+1 actually defined? As I understand it, it could contribute up to five organic nominations (eg if 1000 people divided their plus ones equally between five different longlist works, it would should give those five enough nominations to outrank all the slate nominations).
@Jameson Quinn
I would like to see EPH+ and 3SV proposed.
I think that this will be a good year for EPH+ because people at the Business Meeting will have EPH and the report on the 2015 data study fresh in their minds.
I had originally wanted DN and then 3SV+1, but I have decided I’d rather stick with just 3SV. We can look at possibly doing more in the semi-final stage later. I think there are still ways to frame it in a positive light. Concentrate on how removing the unworthy gets the real top favorites from the 1st round on the final ballot. And that it greatly reduces the possibility of not giving out Hugos on award night.
@Greg
Kevin has made it pretty clear that trying to add EPH+ this year would be seen as a greater change. And I can imagine (just barely :)) that someone who was resigned to letting others decide on EPH might think EPH+ was going too far.
I guess some forms of disenfranchisement are ok, so long as the whim of the guy who hates us throws up a few readable works each year.
@Hampus Eckerman:
YOU MEAN EXCEPT FOR ME RITE HAMPUS??????
😀
Some comments on MSMC:
First, is it really necessary to have a “Moderation panel” that’s separate from the admins? Spreading the power over more people is good, but this seems to just move the power from one small group to another. And it’s not that much power anyway, as the decision is vetted by members.
As for the wording:
The way this is worded it sounds like the panel can choose the two extra works at their own discretion. I assume that is not the intention? It is also unclear if it is meant as two works total or pr. category. Revision: The moderation panel may recommend to expand the list of finalists, in one or more categories, with up to two slots.
I am unsure about this. Presenting two lists is in many ways the easiest. But I think it may be better to present the proposal in a slightly abstract form, without naming names. In particular, I think a vote for removing a work should be about the removal and not what we get instead. Publishing two lists also takes away much of the excitement of announcing the finalists. I suggest something like: The published proposal from the moderation panel may or may not include the names of candidates affected. If the panel recommends to expand the list, the panel is encouraged to describe their reasons for this recommendation without naming the candidates involved. If the panel recommends to remove works, the recommendation may include the name of the removed candidates but not the replacement.
Is one single take-it-or-leave-it always the best? Suggestion: The administrators shall conduct a vote among the members to accept or reject the moderation panel’s proposal. If the recommendation involves several distinct actions, the administrators and moderation panel may choose to conduct separate votes for each action.
And as a more fundamental change, I would have liked to give the moderation panel a more hard-hitting tool. Something like:
“The moderation panel can recommend to disqualify the ballots of voters belonging to specific voting blocks. If the moderation panel recommends this, the administrators shall find a way to identify the block voters – for example more than n% overlap with a published slate – and recount the nominations without the disqualified ballots.”
But I think MSMC can be a good solution without that.
@Brian Z: You do realize that what’s happening now is that everyone except the slate voters is being disenfranchised? That’s why I compared it to voter fraud earlier; the effect of the slate is to make the results as if the rest of the ballots were thrown way uncounted.
@Johan P.
I think having a moderation panel accomplishes two things:
1) A number of admins have already stated they’d quit or stop volunteering if asked to take the hot seat.
2) It allows us to split the authority across WorldCons. This potentially takes heat off the current World Con. It also adds a check and balance where the current World Con couldn’t rig the panel. I don’t think they ever would but if we’re looking for limits on authority it helps.
It is just a kind of job demarcation and who gets to shoulder the responsibility.
And what Stoic said 🙂
@Rail: Of course he realizes that. Responding to Brian Z as though he were arguing in good faith is a waste of your time.
I do not agree on this. The admin panel would need to have administrator rights in that they had to be able to detect blockvoting among the voters.
I really like MSMC with some of the additions. Panel selection could cause a furore, but otherwise I think it’s the best proposal I’ve seen.
@Johan P:
People have pointed out that it is easy to slate works outside the top 5, so the expanded list could include malicious works. I think someone even pointed out a scenario where positions #6-15 could be slated.
So maybe add a line: If the new works are non-conformant, they may be removed, and this step repeated.
If this sounds complicated maybe just stick with the original wording 🙂
@the little mole: That’s right, but I think Jameson’s comment assumes low participation in the addition round.
Messing around with the beginnings of some nomination-modeling this afternoon, I discovered the treasure-trove that is the 2009 nominations detail. Someone on the Montreal committee found this stuff interesting. You get a lot more data than other cons seem to have provided, including:
* A longer list of nominees, that apparently includes anything that got at least 5 votes in many categories
* Figures not just for total ballots for each category, but also total nominations and total unique nominees
This is gold, since it lets us see at least one view of the long tail. Consider:
Novel: 639 ballots, 335 unique nominees
Novella: 337, 122
Novelette: 373, 233
Short Story: 448, 470
Avoiding spurious precision, the ratios give an interesting sense of what the dispersion looks like.
Ratio of ballots to uniques:
Novel 2:1
Novella 3:1
Novelette 3:2
Short Story 1:1
Now consider the ratio of ballots to uniques for the approximately 150 party-line slate nominators in 2015:
Pure slate nominators 20:1 (150:7)
We have to start making some assumptions to reckon the power of the bloc of majority-slate nominators that Schneier and Quinn tell us was about equal in size to the party-line group, but we are swagging here anyway, so given the graininess of the ballot (you have either 1, 2, 3, 4 or 5 nominees), let’s say 40% of their ballots are open, so:
Non-slate short-story uniques = 150 (base 1:1) * .40 = 60
+ slate candidates = 7
TOTAL 67 candidates
Majority-slate ratio (short story) 2:1 (150:67)
Combined-slate ballots:unique ratio (short story) 5:1 (300:67).
This looks an awful lot like our theorized “5x magnification factor” for slate voters. That’s just short story, but I think I can find it for novel too:
150 majority-slate voters
x 1/2 (uniques/ballots ratio for novel)
x 0.4 (the truncation factor from most of their ballot going to slate candidates)
= 30 non-slate uniques
+ 6 slate candidates for novel (across SP and RP slates)
= 36 total uniques
Combined party-line and majority-slate ratio 10:1 (300:36)
Organic ratio (novel) 2:1
10/2 = 5!
Novellas? 150 x 1/3 (see above) x 0.4 + 5 = 25.
Combined slate ratio 12:1 (300:25)
Organic ratio 3:1
Here we only have a 4x magnification factor. Of course, at these levels of precision 4 ~= 5, so I wouldn’t get too worked up about the difference.
@David Goldfarb: Just venting.
@Jim Henley: Fascinating. A shame we don’t have those numbers for other years.
Your best argument is it’s OK to become like them? That’s your argument?
I agreed to go ahead and use a modified DN.
The objection is to taking the vote away from a huge number of the voters eligible to choose the Hugo final ballot. Whether by stripping them of the right to vote for it, or by a fraction of the eligible voters discarding what they didn’t like.
Go ahead and massage the rules to do that. Then give them several months to calculate just how many memberships they need to no-award YOUR nominees if they can’t to do that already and figure out how to screw up your awards worse next time. You’ll get to mock and gloat when they failed or wail and gnash when they succeeded.
Then try to remember what the purpose of doing this was in the first place.
Destroy the Hugos indeed. They ought to give you a medal.
@Jim Henley
Interesting! I’ve been using a grouping of publically available nomination ballots for this year to try and model attacks on 3SV etc.
Based on the Montreal figures I may have to drop them though. I’d guess insufficient sample size and scale (which I’d kind of worried about but without real data I was making do…). Playing mostly with Novelette as a vulnerable category I have 43 organic nominators and 68 unique works.
Henley, that’s either completely incorrect or at best highly misleading.
That is the selectively cited number of 2015 Best Novelette ballots for which all of the stories appeared on either Brad’s recommendation list (of four stories) or Vox’s recommendation list (of five).
We have not been told the number of those in other categories or how many of those ballots had one, two, three, four or five items.
There is another possible reason why the authors might have selected Best Novelette as very “slate-like.”
Brad encouraged his fans (and Larry’s and Sarah’s etc etc) to fill out the entire ballot. It is observably the case that Mad Genius fans read mostly online stuff and novels rather than print magazines and couldn’t recommend many novelettes in Sad Puppies 4. It stands to reason that even if they were trying to pick their own favorites in other categories, they might have fallen back on the recommendations in novelette. (With the possible exception of devoted Analog readers among them, who also would have responded favorably to choosing those three Analog stories.)
I’m going to guess that if most of those 150 ballots did have the so-called “full slate” of four or five stories, the authors would have said so. They are reading these threads and would be able to clear this up this right now if they wanted to and were allowed to.
(By the way, a comparison of the impact of EPH on ballots with five items to the impact on ballots with the usual average two or three items is something I’d be interested to see.)
I realize I must be sounding very harsh. Sorry.
I disagree with making that sort of highly selective reveal of controversial private information. But now that it has been done, it would be fairly simple to clear this up. The authors could tell us what percentage of those 150 ballots contained the full list (Brad’s or Vox’s) and what the average number of items per ballot was.
It would also be good to have the same information for other categories for comparison.
And to know, if those puppy-recommended nominees were removed, what the numbers would be for the remaining top finalists, for comparison.
Remember that if someone put all of somebody else’s recommendations on their ballot, and nothing else, that does not mean they did not express any their true preferences. They may have loved those stories.
I will not respond to Brian Z’s latest allegations in full, but I can say that Best Novelette was selected as a more-or-less typical category in 2015, not one which was particularly “slate-like”.
(Responding to every half-assed FUD would be a bad precedent, but I can re-state things that were already made clear in the paper.)
Note: I did not see BZ’s 7:36 before I made the above two responses, so “latest” above refers to his 7:23.
(sorry for the multiple posts; my “edit” button is acting up.)
Jameson, I do apologize for the harsh tone. However, I said it was possible the authors might have selected it as slate like and I didn’t know. The way we can know what is “slate-like” is to know how many of those ballots included all of the items on a recommendation list, as well as how many ballots included only some and on average how many. That’s all I’m asking at the moment.