By Jameson Quinn
1. Intro and disclaimers
This is a summary of the discussion thread on the post by Kevin Standlee discussing three proposals for expanding the Hugo nominations process in order to help avoid problems like the current list of Hugo finalists.
In summarizing a thread that’s over 500 comments long, and that refers extensively to two other threads with hundreds of comments, I am of course going to simplify matters. In particular, I’m going to overemphasize the points of agreement, without listing every qualification, caveat, quibble, or outright objection that was brought up. Obviously you can read the thread yourself, but if you don’t, please imagine it to have all the disagreement and misunderstandings (as well as off-topic filk, execrable puns, cute references, hastily-constructed codes, etc.) that you’d expect.
2. Initial options: A+2, DN, and 3SV
Of the three initial proposals by Kevin, one of them (nicknamed “+2” or “A+2”) relied on giving new discretionary powers to Hugo administrators. In the discussion thread this idea encountered significant opposition, so it will not be discussed further here.
The other two proposals would both create a new intermediate round of voting, in which the initial votes have been used to create a “longlist” of 15 works which is publicized and voted on in some fashion in order to get the list of 5 finalists. In the Double Nomination with Approval Voting (“DN”) proposal, nominators would vote for (“approve”) the longlist items they liked the best; and in the 3-Stage Voting (“3SV”) proposal, a majority of nominators would be able to disqualify some of the 15 works, without affecting the ordering of the remaining works from the nominations phase.
3. Consensus: 3SV (+1?)
Many of the participants in the discussion began with a preference for DN; they felt better about voting for the longlist works they liked than about voting to reject the ones they felt were illegitimate. However, as the thread progressed, and the strengths and weaknesses of the two proposals were analyzed in more depth, the consensus shifted, and by the end of the thread proposals based primarily on 3SV were the clear winners. Many supported a proposal nicknamed 3SV+1, described below, which integrated some aspects of DN onto the base of 3SV.
4. Why 3SV and not DN?
Why did people’s preferences shift from DN to 3SV? Several reasons:
A. Under DN, voters would have just weeks to assimilate and vote on a list of 255 longlisted items. Many of the most careful nominators would barely vote for any; while the most prolific voters would probably be going mostly by kneejerk reactions. This is true for 3SV, too, but it is less of an issue as explained below.
B. DN conflates two questions: “Do I like this work and feel it may deserve a Hugo?” with “Do I feel that this work’s presence on the longlist or in the list of finalists would be a legitimate result of honest fan preference?” In 3SV, those questions are separate, and votes to disqualify a work are based on the second question alone — one which does not require fully reading/reviewing every longlist work.
C. Unlike DN, 3SV would deal decisively with the issue of “troll finalists”: that is, works promoted by slates explicitly in order that their shocking and/or offensive nature might cast discredit on the awards.
D. 3SV would be similar in spirit to the “no award” option, except that works thus eliminated would not take up space on the list of finalists, and awkward moments at the awards themselves would be minimized.
E. DN would open up new kinds of attacks on the list of finalists, such as actually increasing slate voter’s capacity to act as “kingmakers” and/or perform “area defense” against certain kinds of works. All they’d have to do was to have enough voting power to reverse the gap between two works which both have significant organic (non-slate) support. But under 3SV, actually eliminating a work would not be possible without a relatively high “quorum”* of voters, and we hope that community pressure would lead to a low background level of organic rejection votes, so a minority of slate voters would be unable to use rejection as a weapon.
So, tell me more about how 3SV would work
The details are still up for discussion, but the basic idea is as follows:
3-Stage Voting (3SV) adds a new round of voting to the Hugo Award process, called “semi-finals,” between the existing nominating ballot and the existing final ballot. In 3SV, the “longlist” of top 15 nominees (as selected by the same process as the finalists will be selected; that is, EPH or EPH+ if those have passed) are listed in a way that doesn’t show how many nominations they received. Eligibility for this voting is being debated (see below), but the original proposal is that it would be restricted to members (supporting and attending) of the current Worldcon (not the previous and following Worldcons). Eligible voters presented with this list, with a question on each of the fifteen semi-finalists in each category: “Is this work worthy of being on the Final Hugo Award Ballot?” with the choices being YES, NO, and ABSTAIN.
If a work gets more than a “quorum” of no votes, it is not eligible to become a finalist. There are several proposals for how to calculate the quorum. The formula may involve such things as the number of eligible voters, the number of “YES” and/or “ABSTAIN” votes, the turnout in round 1 or in previous years, etc. The idea is that the quorum should be high enough so that a minority of slate voters will be unable to reach it, but low enough that a clear majority of fans can pass it, even given reasonable turnout assumptions.
During the “semi-final” voting period, admins would also be checking the eligibility of the works on the longlist, and accepting withdrawals from the author (or other responsible party; henceforth, we’ll just say “author”) of those works. The admins would make a good-faith attempt to contact authors, but note that since the longlist is public, the admins may assume that non-responsive authors have heard of their presence on the longlist, and thus that any authors who do not explicitly withdraw their works would accept becoming finalists.
The finalists will be the top 5 works from the longlist that have not been declared ineligible by vote, found ineligible by the admins, or withdrawn by the authors. EPH or EPH+ would not be re-run after ineligible works or withdrawals.
After the Hugos are awarded, admins would publish the usual statistics (that is, for EPH, the votes and points-when-eliminated for each of the members of the longlist). They would also publish the reason for ineligiblity (voted out, ineligible, or withdrawn) for any work that otherwise would have been a finalist. They would also publish the anonymized set of vote totals for each longlist item in each category, where “anonymized” means that they would not indicate which title was associated with which vote total.
What’s this “3SV+1” you mentioned earlier?
The “+1” means that, in addition to 3SV, all voters who were eligible for round 1 may add a single nomination per category, for something from the longlist they did not already nominate, to their existing ballot, during the same “semi-final” voting period. These combined ballots would then be counted by the usual process (that is, the current system, EPH, or EPH+) to find the finalists.
How would all of this interact with EPH, EPH+, 4/6, and/or 5/6?
The short version is that without EPH a realistically-sized, well managed slate could hope to entirely take over the longlist in many categories; with EPH, it could take about 2/3 (around 10 slots); and with EPH+, it could take over about half (6-8 slots), or possibly a bit more with cleverer strategies (but not as much as EPH, even then). 4/6 or 5/6 don’t change that story by much, though they help a bit in keeping organic slots among the finalists in spite of “kingmaker” slates. And +1 helps push slates towards around 1-2 finalist slots; hurting them in the common case that they had been going to get more, but actually helping them if they had miscalculated and were heading for less than that.
Here’s a graph of how many slots slate nominators could have gotten in 2014, as a function of number of slate nominators, if they’d split 3 ways and had the same level of coordination as they did in 2015. Note that 2016 had many more nominators than 2014 so it would have taken more slate nominators to get the same effect.
Here’s a similar graph, but assuming the slate nominators are better coordinated:
Here’s yet another graph, but assuming the slate voters split only 2 ways. In theory, this is better for them if there are fewer of them, but worse if there are more, because they max out at 10 slots. However, as you can see from the graph, it’s really not that much better even for small numbers; the random “bootstrap sampling” effects almost overwhelm any advantage:
Have you thought of any downsides? What about…
Yes, kinda. We have people (both honest supporters and honest opponents) thinking of attacks. And there’s always room for more on this “red team”. So far, here are the criticisms we’ve come up with. First, for “3SV” (we’ll talk about “+1” below):
Couldn’t slate voters take over the shortlist? As you can see in the graphs “take over 6-10 slots of the longlist” is the only one that we think is a concern if (as we expect) EPH or EPH+ is in place.
Wouldn’t this just increase negativity? There are several safeguards against this becoming merely an excuse for people to campaign against works they happen not to like. First and most important is social pressure; it should be clear from the outset that this is a just safeguard against outright bad faith, not a chance to express differences in taste, and I believe that any Worldcon members who promote disqualifying a work just because they don’t like it will not get much support. Second, there’s eligibility. Various rules, discussed below in “open issues”, have been proposed to prevent a campaign to bring in Worldcon outsiders after the longlist is public. Third, there’s the quorum; if participation in the second-round voting is low, it will not be enough to pass the threshold to eliminate any work. Fourth, there’s the relatively short period of the semifinals, also discussed below. And fifth, there’s the fact that elimination votes for a specific work would never be publicized; only anonymized distributions of votes for each category. (In some cases, of course, the identity of which work got a certain vote total would be easy to guess, but that would still be just a guess.)
Wouldn’t this fundamentally change the nature of the Hugos? They have already been changed by the slate. Many of the people in the discussion felt that this change, though it would not go back to exactly as before, would still be a change in the right direction.
Would this be more work for the administrators? In some ways, yes, of course. However, in at least one important way, it would actually simplify their lives. Since the longlist would be public, it would be much easier for them to contact authors. On a related note, authors could not leak their status as finalists, because until the list of finalists came out, they would know no more than the public at large.
Would this allow some unanticipated downside? Obviously, we can never rule that out 100%. However, we do think we’ve been pretty thorough at exploring all the angles. Again, you can read the thread and decide for yourself.
And, downsides for the +1 addition:
Would this be tough to administrate? Not if EPH or EPH+ were in place, since any program capable of doing either of these would already be able to associate multiple nominations with the same nominator and make sure that invalid votes, such as a single nominator nominating a given work multiple times, were not counted. It has been suggested that a proposal to institute +1 should say that this change will sunset (require re-approval) if EPH or EPH+ ever does.
What are the open questions/issues?
Eligibility: who should be eligible to vote on the semifinal round, and who should be able to add +1? The former question is more fraught. Several people said that they would want eligibility to be restricted enough that outsiders can’t come in and get memberships after the longlist is published. Others said that it is important to let the community respond and that if membership spikes on seeing the longlist that could be a healthy thing. One compromise proposal (suggested by yours truly) was that in order to be eligible, you would need to be a current member (attending or supporting) of this year’s Worldcon, AND also have been eligible to nominate (whether or not you actually did so). So if you were a member of the prior or following year’s worldcon, you could sign up after seeing the longlist; but not if you weren’t.
Quorum size/formula: There’s been various discussion of this issue.
+1 as attached or separate: The overall consensus seems to be that, if we propose +1, it should be in a separate proposal from 3SV, even though they share certain aspects (such as the concept of a semi-final round). However, there are varying opinions on whether +1 is a good idea, either in general or as a proposal for this year in particular.
Whether to try EPH+ this year: I’ll talk about this more in comments.
The admin discretion 14-18 longlist thing: I was thinking that, if the time between closing round 1 nominations and announcing the longlist is short, admins might have a hard time cleaning the data perfectly. In that case, it helps them be more certain of the list they publish if the next work below the list was not a near-tie with the lowest work on the list. To allow them to avoid such near-ties, especially in cases where they aren’t 100% sure they’ve cleaned the data perfectly, I suggested allowing them discretion to decide how long the longlist for each category would be, between 14 and 18 works.
https://file770.com/?p=29020&cpage=14#comment-436327: Define “nominating membership” in a way that legally allows for the possibility of a code of conduct that could lead to one year’s worldcon revoking voting privileges from a member of the prior year’s; in other words, closes the loophole whereby prior-year members are immune from any consequences for their actions.
Moderated Shortlist for Member Consideration: https://file770.com/?p=29020&cpage=16#comment-436437 This is an alternative to 3SV, where an administrative committee would tentatively suggest eliminating or adding certain works, and that suggestion would have to be ratified by an up-or-down vote of the members of the current Worldcon.
Where do we go from here?
A group led by Colin Harris that includes Kevin Standlee is writing up a proposal are writing up a proposal and are surely watching these threads. When that proposal is ready, we could write up +1 as an amendment or as a standalone proposal. Decide about EPH+ and deal with that. Anything else?
Discover more from File 770
Subscribe to get the latest posts to your email.
I was a DN fan and very against 3SV before, but I’ve warmed up to 3SV a lot, and I believe it’s a reasonable improvement to the existing voting system. Reading through 800+ comments on the other thread is what brought me to this point (well, some I skimmed, and unlike some folks I don’t waste time reading BrianZ). I like Kevin’s version – simply check reject (or whatever the phrase is) or don’t; there’s no accept/reject/abstain complication needed.
I am very much in favor of EPH, so I’m happy with EPH+ as well. I’m not in favor of most of the other stuff, though.
@Jameson Quinn & @Various: Some time last week, in another thread, someone suggested giving fewer nomination slots to previous/next Worldcon members, or to supporting members – I forget which. I tweaked it a little, suggesting a certain # of slots for previous/next members, a little more for supporting members of the current Worldcon, and a bit more for attending members of the current Worldcon. Then possibly the total finalist slots being either the same as the attending slots, or one more (in the mode of 4/6 or 5/6).
For example, 3 nomination slots for past/next Worldcon members, 4 slots for current Worldcon supporting members, and 5 slots for current Worldcon attending members. I guess the total # of finalists would be a moot point with 3SV, so maybe ignore that part.
Anyway, any thoughts on this? Since people have mentioned the possibility – though I think maybe now it’s off the table ??? – of removing next/previous con nomination rights, this seems like a way to keep them but lessen their effect. If you think (as I do) “griefers” (autocorrect sez: briefers!) would simply pay every year anyway, then giving fewer nom slots to current Worldcon Attending would reduce their effect on the ballot.
I’m not sure how this would interact with EPH(+); I guess folks with fewer slots would get fewer points, based on how many slots they had (e.g., 4 slots = .8 points, if 5 slots = 1 point).
Since no one commented on the original guy’s comment except me, and no one commented on my comment, probably this has no interest to anyone. But since Jameson’s here – and in case anyone else just missed it – I figured one last mention hopefully won’t annoy folks.
@Kevin Standlee:
This, I like.
It’s simple and is clear what it is for.
I couldn’t find current membership numbers – anyone else know how many 20% would be if hypothetically it were in place this year?
Kendall, the thing is, the argument you make against making an additional nomination in the second round is the same as the argument against making more than one nomination in the first round: if you’d rather have A on the ballot than B, why vote for both?
The answer, of course, is that your vote for B will most likely either make no difference (if B doesn’t get on the ballot, or if it would have made it onto the ballot without your help), or else it will displace something besides A from the ballot. Bullet voting, making a single nomination and leaving the rest of your ballot blank, is certainly a viable strategy if you feel very strongly about your top choice, but if you wouldn’t use that tactic under the existing system, then it’s unlikely to be a sensible tactic under +1.
Pingback: Griefer Matrix 3: Griefer Revolutions | Camestros Felapton
@Kendall:
I was not saying that people who have A on their initial ballot and B as their +1 like B better. I was saying that if B is in a position to win because this happens, there must be more people who had B on their initial ballot and didn’t add A, than there are people who had A and didn’t add B. That’s pretty weak evidence for the amount of general fan love for the two works, but insofar as it is evidence, it is saying that there’s more fan love for B than for A. It’s a tough call, but I think that B is the right winner in this scenario.
+1 can, in theory, “shoot yourself in the foot”. I think most people are able to make choices about what they value more. Do you want to nominate everything you think is worthy? Then +1 gives you some extra reading time, and a reading list with some great choices on it. Do you want to make sure you do everything possible to support that one amazing work? Then nominate it to start out with, and don’t use +1. Do you want to make sure you do everything possible to make the finalist list as overall awesome as possible? Then +1 gives you an extra chance for your vote to matter.
Do you want all those things at once? Well, sorry. Consider the tradeoffs and make a choice.
You can either say “+1 is a bad system because it increases my potential regrets”, or you can say “+1 is a good system because it improves the quality of both the best-case scenario and the expected (average) scenario”. If you care more about avoiding regret than maximizing expected value, then I guess you’d rather I didn’t give you a dollar because you might mistakenly spend it on the wrong thing.
MTV also has rules letting them throw away ballots:
http://www.mtv.com/content/ontv/movieawards/2016/categories/popUpRules.html
I do think those rules are good and there is a reason they exist. It is to let an Award survive the trolling age. They have been targets for longer time, thats why they have had these rules since before. People are talking about Boaty McBoatface, but I remember when Hank The Angry Drunken Dwarf won Peoples Magazines poll of most beautiful people in 1998.
This is nothing new, the strange thing is really how so many proposed solution continues to refuse to acknowledge that the problem is about assholes and block voting. Instead the solutions say we should treat all nominees as equal.
I vastly prefer the standard clause that I keep finding on award after award after award to throw away ballots for obvious assholes. See the example in the link above.
I think that, for completeness’s sake, the threshold for removing a work should be 20% AND a majority of the participants in the semifinal voting. It almost certainly doesn’t matter, because the chances of a second-round election with over 40% turnout and a close margin are basically nil. But if it ever did happen that 21% voted to eliminate something and 25% voted against elimination, then it clearly should not be eliminated.
@Steven desJardins: I don’t see that as being the same. For one thing, you’re changing your ballot – thinking you’re helping against slaters or thinking “hey I should nominate something else” – but surprise, you may undo your own preference. This doesn’t seem the same as picking anywhere from two to five things in your original ballot, but maybe I’m not thinking it through. BTW I didn’t say anything about bullet voting just for A.
@Jameson Quinn: Sorry for misunderstanding (and thus, misrepresenting) what you said re. people liking B better or not. And I’m not trying to have all those things at once (good points, though), but I’m just thinking people won’t necessarily use this in patterns you expect. But good point about B; this relates to the “least disliked/broadest support” that IRV traditionally gives us, I guess, so when I look at it in that light, hmm, maybe B would be the right winner, or at least right enough. 😉
Clearly I need to think about this some more, thanks, you two. I’m a bit more on the fence now about +1 (but still more interested in EPH+!).
@Jameson
Ah, that makes more sense, thanks.
I added split griefer slates and a bullet slate to the Simulator, and changed the default size of the puppy slates based on their released size cut down relative to the much smaller organic ’84 dataset.
With several competing slates, straight 3SV can have problems with the less popular categories, but the addition of EPH+ really seems to increase organic noms bubbling into the top 15.
Kendall, it really is pretty much the same thing. There were a bunch of categories where I had two or three items that I was super-enthusiastic about, that were auto-included on my ballot, and then three or four that I considered for the remaining slots. In each case, if I added a fourth or fifth nomination, there was a chance that I might “undo [my] own preference”, but there was a larger chance that I might help something that I thought was Hugo-worthy but not my top choice get on the ballot. This year, when I was concerned about the Puppy campaign, I generally chose to fill all five nomination slots, but there have been years where I decided, “Eh, I’ll just nominate these two.” It’s exactly the same calculation with +1 as it’s always been with 5/5: do you fill every slot you can, or do you leave some of them blank? And it’s exactly the same risk, the chance that by voting for your fifth choice instead of leaving it blank you cause your #5 to beat out your #1 for the fifth and last finalist slot.
You seem to be hung up on the idea of “changing” your ballot, but there’s no practical difference between changing your ballot from “A, B, C” to “A, B, C, D” and deciding “I think I’ll nominate my favorite 4 (or 5, or 6) things, even though I could nominate fewer.” If you would do the latter, why wouldn’t you do the former? (And, of course, if you wouldn’t do the latter, you certainly shouldn’t do the former. Nobody’s going to force you to add a nomination in the second round, if you’re happy with your existing choices.)
And remember, there’s a good chance that none of your first-round nominations will be on the longlist. In 2015, for instance, we know that only 24% of non-Puppies who voted in the novelette category picked even a single one of the top five non-puppy works. If you nominated in a category, and none of your choices made the longlist—or if you skipped nominating in that category, but you see something on the longlist that you think deserves to be on the ballot—would you consider adding a nomination in that category? Those are the voters, the ones who didn’t nominate anything that made the longlist, who have the biggest impact in +1, because they’re adding a full point to their new choice.
@Kevin Standlee:
Thanks for the clarification 🙂
I definitely understand that 20% of eligible members is a lot; as I said, I don’t think the scenario of 20% rejecting being met by a greater group of active non-rejecters is a plausible one ; I just wanted to understand the mechanic.
I think the overall goal of any voting system amendment is to preserve as much as possible the voice of the voters, while preventing ballot vandalism.
Basically, the least change that can be effective is the best choice.
As for setting up any kind of committee to act on the ballots, just heck no.
I don’t want to let an organized cabal of thugs force the Hugo administrators to become in turn a mirroring cabal, however well-intentioned.
So for me, setting up a panel of administrators to deal with vetting nominations is right out.
And if it weren’t, if you have your shadowy behind-the-scenes group cleaning up the ballot in this way, you wouldn’t even need 3SV since that would be just duplicating effort.
But dealing with questionable nominations is what we would create 3SV to do, and it leaves the decisions up to the membership, which to me seems much better.
I’m thinking of 3SV as a sort of pre-emptive No Award.
No Award has been used perfectly effectively to deal with crud on the ballot; this is just a way to ensure that we have a ballot left afterwards.
As such, it would be the current year’s eligible Hugo voters – not the earlier nominators – who examine the Long List and, if appropriate, vote No Award.
It is these voters who decide if No Award is appropriate now.
3SV simply allows them to do so at an earlier stage, but with the same standards.
And if, for example, there is a concern that voters might use 3SV punish a writer like Leckie for being provocative or innovative, it is worth considering that they already could have done so, through No Award.
The heavy hammer of No Award does not have a history of being used randomly or lightly, and there is no reason to believe it would be in future.
It is important not to let the behavior of a small percentage of bad actors drive us into paranoia about the whole process.
3SV is a change in timing, not in fundamentals.
However, I don’t see a need for any further fine tuning beyond this, and I really don’t like the +1 option at all.
Nominators put a lot of thought and effort into producing five, or – if 4/6 were passed – even six bites at the apple.
That seems an ample opportunity for their voice to be heard.
People will always have second thoughts, but I’m unconvinced that there is a need to act on them.
The nominating members have done their job by presenting their top choices out of everything eligible in that year, and their voices have come together to produce the Long List.
At this point the members have spoken, and the ballot is implicit in their choices.
There is, sadly, good cause to intervene, though, because of shenanigans
The nominators’ job is done, though – they have given us the Long List.
Now it is up to the voting members.
They have always been able to defend the ballot through the choice to No Award egregious crap, and they have done so when necessary without employing it frivolously or unfairly in the past.
3SV allows them to produce a ballot giving the organically-nominated works in the order in which they were originally proposed by the nominators, despite the effort of vandals to prevent this.
Doing anything more than simply removing egregious crap is a fundamental change, not housekeeping.
Adding +1 changes a ballot produced by people’s favorite works out of everything they read – what remains after the crap is removed – into a ballot produced off of a short curated list.
That’s a different thing.
For one thing, if 3SV allows the current voters to +1 a new nomination, surely all nominators should have this same opportunity, which rather complicates things in terms of voting.
In fact, if 3SV+1 is adopted, the burden appropriately falls only on nominators, not on Hugo voters at all, since now it is a question of widely editing nominations.
If someone was not eligible to nominate before – if their membership purchase came after the nominating deadline – why should they be allowed to add a nomination before voting?
And is it fair, if they can nominate at all, that they only get one nomination?
If they can nominate at all, how can they be limited to only choosing from the Long List…. and Xeno’s nominations and clean-up votes ensue.
This way lies madness.
Nope, the nominators are finished when they send in their five (or 4/6) six choices.
Cleaning up the ballot belongs to the folks who were always able to No Award, but now they just do it a little sooner.
And really, the question of whether people want to produce a final ballot from off a short list is, in fact, unrelated to making changes to deal with the Griefers.
It is a fundamental innovation sneaking in the back way while people are distracted with other more important matters.
Do all nominators, in fact, want a second chance to amend their ballots, given a chance to look the preliminary results of everyone’s choices?
Do we actually think it is more appropriate to select the Hugo ballot based on everyone’s initial decisions, or to present a two-stage nomination cycle in which the final ballot is selected from the Long List?
This a completely different conversation having little to do with deciding on a way to eliminate malicious nominations.
Perhaps we should concentrate on getting the crap off the rug before we consider moving the furniture around.
Nitpicky correction: “In 2015, for instance, we know that only 24% of nominators in the novelette category were non-Puppies who picked even a single one of the top five non-puppy works.”
OK, third time’s the charm.
I think that proposing 3SV and MSMC simultaneously this year is a bad idea. We have limited time and attention in the Business Meeting, and even if we didn’t splitting support between two similar proposals would be (as a voting theorist) my nightmare.
So, if we want the ability to get either 3SV or MSMC for 2018, we have to propose and pass a single amendment this year which does one of them, but next year can be turned into the other one with a “lesser change”.
I believe that the following would work:
-A clean 3SV proposal.
-An MSMC amendment to the 3SV proposal, which states that a panel (chosen by the previous WorldCon?) has the power to cancel the 3SV election for any or all works if it determines that there is no legitimate doubt that they were nominated in good faith and {whatever language we want to include about being free of grossly offensive/libelous/whatever content}.
If both of these passed, then next year’s Worldcon could clearly choose to ratify 3SV alone, as a “lesser change” than the amended MSMC version of 3SV actually passed.
I also believe that this version of MSMC is better. I’d rather be on a panel ruling that works A through W are manifestly innocent, than on one ruling that X, Y, and Z are proven guilty. The panel does not have to take a stand that some works should be eliminated; it merely has to allow the elimination election to go forward because there is legitimate doubt. Obviously, any 3SV election or MSMC decision could lead to hard feelings, but I think this is the least-negative version I can imagine.
A final note: I think that +1 should be a separate proposal, but it would mesh well with the above. In years when all the 3SV elections were cancelled (which would ideally be the norm in the long term), +1 would mean that the calendar time carved out for 3SV would not be a waste. In fact, I’d argue that even in such a year, +1 would arguably be doing a better job refining the finalist decision than if that time were just given to extending round 1.
@Hampus Eckerman: The MTV Movie Award nominations are selected by producers and executives at MTV; the public only gets to vote on the winners. The situation is fairly different from the Hugo voting, given that the MTV Movie Awards could be at risk of receiving floods of automated votes, but would not be at risk of inappropriate works being selected as nominees.
If the nominations for the MTV Movie Awards were wide open, I could imagine people trolling by, say, nominating something from a drama for Best Comedic Performance. But since the nominations are selected internally at MTV, that’s not an issue for them.
Joshua K:
But still, they have a clause saying that inappropriate works are not accepted:
My point is that this is a standard text used by a lot of awards. At least three I’ve found as yet and I haven’t spent that much time searching. The Hugos aren’t the first award under attack. There is a reason why so many awards have texts like this.
I guess you also saw the texts from the Shorty Awards:
Kendall on May 23, 2016 at 8:38 pm said:
I expect to see a proposal to do that on this year’s agenda, although I am not involved in it in any way. Doing so has some complexity. If I were drafting such a proposal, I would include language so that anyone who had prior/subsequent Worldcon nominating rights would not lose them during a transitional period. Thus nobody could plausibly claim that they bought a membership with what they expected would include certain rights that were then withdrawn after they bought the membership.
Hampus Eckerman on May 23, 2016 at 10:17 pm said:
WSFS has on the whole been extremely reluctant to vest anyone but the membership as a whole with the right to determine whether a nominee is award-worthy. If you allow the Committee to throw out ballots for any non-technical reason, you effectively allow them to decide who wins the award, regardless of how people voted. It’s a “trust no one” form of governance. Even the limited amount of trust we give Administrators is tempered by requiring certain statistics be published so we can check up on how they did after the fact.
This is a typical example of talking principles instead of practice. If people really believed administrators would do stuff like that, you shouldn’t trust them with numbers or counting votes anyhow.
So have a separate board that verifies that votes are removed because of slating. Or, as in Rocket Review, give membership as a whole a chance to confirm that they accept the choices of the administration.
As I said, I find 3SV to be a solution to vote for only because the common solutions that are most effectual and have least risk have a hard time finding acceptance. We’ll see if we get back to them in two years. After all, last year it was hard to even get acceptance for EPH or to do anything at all.
@Jameson Quinn
How do you vote against elimination? The proposal only allows for a vote to eliminate. I don’t think the fact that someone did not check the REJECT box should count as a vote against elimination. It may just mean they’re unsure or don’t know enough about that nominee.
@Jameson Quinn
Merging 3SV and Rocket Review might be a step too far. Whatever goes to the Business Meeting needs to be as straightforward as possible. A merger would likely get messy enough to get a down vote.
I’ve also been racking my brain to find a merger position. The problem is they just aren’t all that compatible.
I think there are two core viewpoints at play:
1) Let’s empower admins to handle the problem. As an out of band solution it can’t be gamed, should have minimal side effects, and ends the problem NOW(!). By not changing the algorithms or voting procedures we’ll retain the character of the awards. (Admin+2, Trust but Verify, etc)
2) More admin power?!? Oh, hell no!!! Changing the voting procedures should allow the members, as a committee of the whole, to address the problem. We expect gaming and side effects will be minimal, By leaving the decisions to the voters we retain the character of the award. (3SV, DN)
Rocket Review is already an attempt to compromise between those positions. I’m not sure how to edge it closer to either camp without breaking the compromise it’s already reaching for.
The perfect should not be the enemy of the good (though everyone’s definition of perfect varies). We should probably just focus down on whichever solution has traction so we get something passable this year.
Absent someone proposing a brilliant plan to merge them (without losing their essential characteristics) let’s just figure out what’s got the most support. If Rocket Review gets a groundswell (come on apocryphal e-mail lurkers! 😛 ), or Trust but Verify, or 3SV then let’s focus work on that one and move it forward to the Business Meeting.
Greg Hullender on May 24, 2016 at 7:19 am said:
By not casting a vote. (If you want to vote against eliminating all candidates, don’t cast a ballot at all.)
I think that the default state should be to accept all candidates. That’s what it is right now. The only reason for the intermediate stage of 3SV is to give members a chance to reject specific candidates. If the default is “accept,” then an explicit “accept” vote and an abstention are the same thing.
@Hampus Eckerman
What did you think of the proposal that the panel would “pre-check” some of the rejection boxes for 3SV? Those works would be rejected unless 20% of eligible members voted to uncheck them.
If nothing else, that would make like a lot easier for the members, who wouldn’t have to go looking up all the slate nominees in all the categories. They’d only need to read the justifications the panel wrote for each rejection.
@Stoic Cynic
If you define Rocket Review as “a panel that prechecks the boxes for 3SV” then I think you can factor it into a different proposal, with a proviso to the effect of “If 3SV does not pass, this proposal is withdrawn.”
Ditto with the 3SV+1. It should be a separate proposal, and, strictly speaking, it doesn’t even require 3SV to pass.
@Kevin Standlee
Yes, but Jameson is proposing that a nominee not be rejected unless the REJECT votes exceed the APPROVE votes. That makes no sense if there are no APPROVE votes.
Well, it was my proposal so. 😉 I find it as a good solution, if it also looks at works that wouldn’t be put on the ballot without blockvoting. It is not as riskfree as the solution I would have preferred, but I believe the risk is low enough.
Greg Hullender: Since I don’t envision the business meeting letting this discussion pick a solution for it, if there are two mutually exclusive solutions wouldn’t it be better to refine each and offer both?
@Hampus Eckerman: I did see the clause you highlighted in the MTV Movie Awards rules, but I had a hard time thinking of how someone might be able to violate it, given that the votes are cast by “clicking the button adjacent to or below the name of the Nominee of your choice.”
I guess that a voter could try to engage in annoying, abusing, threatening, or harassing by entering something of that nature in one of the other fields that they have to fill out (like a fake e-mail address containing a harassing message), but the MTV Movie Awards don’t seem as susceptible to attacks like that as other awards with a more open process would be.
@Kevin Standlee
Thank you for explaining the mechanism of 3SV more thoroughly; I like the fact that is simple, and can be set to only kick at a level that will prevent quiet mischief as well as public mischief.
Additionally, it adds some legitimacy to the awards if we know that the kicking function requires a large number of people. Additionally, 3SV effectively makes the moderating function a vote on the fitness of a nominee. It has a legitimacy to it that a select committee does not have. That being said, if there’s language out there for a sort of “break glass in case of trolls”, heck, why not.
@Mike Glyer
Well, Stoic did say that he was looking for a way to redefine his proposal so that it wasn’t incompatible with 3SV. Hampus’s proposal (3SV Precheck) seems to me to be a very elegant solution to that problem.
And Jameson also expressed concern about too many conflicting proposals. One solution is to make them non-conflicting. I think it’s possible to define everything that’s still on the table (here in File770) in such a way that they don’t conflict with each other.
Joshua K:
My guess is that it is a copy-paste clause several different awards use, regardless of if all parts are applicable to them or not.
@Hampus Eckerman:
I agree with you on that.
I think the biggest problem with 3SV is that the form is too long, and people will balk after taking one look at it. 255 entries! That’s triple the length of the current nomination form.
So I’m very much in support of the admin merger proposals – Jameson’s “removal of innocent entries” or Hampus’s “pre-check of guilty entries”. They effectively reduce the form length and make it easier for people to winnow out the chaff.
I like the removal of innocent entries more, since it seems the least “negative” but actually does the same work as the other proposal. I think it’s likely to have more support, perhaps even from people who don’t like strong admins.
Re “what is APPROVE?”: If you submit a 3SV ballot that does not reject a work, you are voting to accept that work as valid. If more people accept a work than reject it, it should not be rejected, even if there are more than 20% for “reject”.
Re: combining 3SV and Rocket Review: here’s the opinions I’ve seen so far, in a spectrum from “trust admins” to “voters only”.
1. Admins should be able to disqualify troll works (and/or add finalists that would otherwise be pushed off by kingmaker sock puppets).
2. Admins should make a recommendation as in 1, which is then ratified by the voters (basic MSMC).
2a. Up or down vote on admin recommendation by category.
2b. Individual vote on admin recommendation by work.
3. Voters should be able to disqualify questionable works, but there is an an admin panel which can (if unanimous) certify a work as non-questionable.
4. Voters should be able to disqualify any work, but there is an admin panel which can set the “default check state” of voter ballots.
5. Voters should be able to disqualify any works (pure 3SV).
(Note that for 4 and 5, voters would still be socially discouraged from disqualifying on questions of taste alone; disqualification is intended for seriously offensive stuff, not just “I don’t like it”.)
Naturally, I favor option 3, my own proposal. It makes the admin role into “good cop” by default, but it only takes one “bad cop” on the panel to throw something to the voters. However, in practice, I think that most works would not require a 3SV vote. The admins are encouraged to come to consensus, but if there is not consensus, that’s not seen as a broken system, but simple as something that the voters decide. This is, I think, the best way to minimize the negativity.
Another way to look at it is, what’s the worst case scenario? People might reasonably disagree as to which extreme would be worse, but obviously it’s one ot the extremes. So:
Say the admin panel allows a 3SV vote on every single work. In that case, this proposal is just the same as 3SV, which seems acceptable.
Say the admin panel doesn’t allow a 3SV vote on anything, even though some of the stuff really shouldn’t be there. First off, I think this is unlikely, given that it would require the panel to be unanimous. And second, even in that case, we would still have the option of “no award” (and possibly +1 and/or EPH+ would also help clean up the list of finalists).
@Jameson Quinn
Thinking about +1, I’m warming up to an earlier proposal that it shouldn’t be just +1. Why not let people “top up” their nominations back to 5? That is, your nomination form would still be active, but the things that made it to the top 15 would be grayed out (you can’t delete them now), and the ones that didn’t make it to the top 15 would be empty, giving you room to make new nominations. The new nominations would have to come from the top 15–no writing in things that didn’t make the top 15.
The programming isn’t much harder than for +1, and the limit seems more rational. Top-up would let people who didn’t nominate at all in round 1 (because it’s hard) to pick things they recognize from a fairly small list (a much easier problem). Slate voters, on the other hand, wouldn’t get any new votes because (presumably) their 5 would already be on the list.
We might actually rename the phases “prenomination” “nomination” and “final vote.” That might encourage more people to participate in the nomination phase (i.e. the 3SV/+1/top-up phase) as well as making it harder for slates to make hay from their inclusion on the list. “Hugo Prenominee” doesn’t sound very good, I don’t think.
Greg, I feel like “+ up to 5” might discourage people from voting in the first round. We don’t want large numbers of people saying, “It’s too much trouble to read everything, let’s just wait and see what other people come up with.”
Some form of +1/+2 might be okay: for example, if none of your nominations make the longlist, then you can pick up to two items from the longlist; if 1-4 make the longlist, you can add one; if 5 or more make the longlist, you get no additional nominations. (It’s possible for more than five of your choices to make the longlist if the administrators relocate a work from one category to another.)
But on the whole, I like +1 for its simplicity.
Why do I favor +1 over +5? After all, as a board member for the Center for Election Science (electology.org), which very strongly favors approval voting over plurality, shouldn’t I be opposed to any system which limits people to one vote at a time?
The problem is that this is not a normal voting situation. It’s not a situation of people-who-like-X versus people-who-like-Y; it’s more like people-who-like-X-or-Y-or-Z versus people-who-hate-people-who-like-X-or-Y-or-Z. We’re calling that latter group “griefers” because their openly-stated goal is to ƒ#¢? with the system in any way possible. (ETA: drat, my unicode grawlix was censored.)
+5 gives them a better chance to use round 1 to mess with the longlist and still also act as kingmakers in round 2. +1 is more in line with the idea of “vote for only things you’ve read and think are worthy”; assuming your idea of what’s worthy doesn’t change drastically between rounds 1 and 2, you’re likely to have a chance to read only a few more works per category, of which you probably won’t find more than 1 worthy. (on this principle, I think +2 would be fine for Short Story).
I would also prefer removal of innocent entries as on Jameson Quinns third alternative.
You acknowledge that 3SV, EPH, and EPH + also help them act as kingmakers? The new concern is that they will be able to do both?
They seem perfectly capable of using 3SV to field credible finalists with broad support to crown during the final voting, while also putting offensive things on the long list to provoke as much outrage as they desire and you’re willing to give them.
One reason the current system is better is it limits their moves. Do they provoke the SJWs and laugh at them, yet again, and eventually it must start to get old, or do they actually want to support the SFF they like for an award? They want to do both – you saw it in the rabid rec list this year. Under the old rules it’s harder to do both.
With all of these magic bullets it becomes easy.
Trolls gonna troll.
Brian Z: I am in fact unable to tell whether you’re saying this, this, or both.
Jameson,
It was specifically stated that they’re looking at a kingmaker gambit. These guys even have a Tor boycott. With EPH/EPH+ it is in their interest to nominate all the best, most popular things from other publishers and crown the one they like best. Every time you dream up something to stop them, you are putting another tool in their tool box.
The Sad Puppies sweep of 2015 was an accident. Back at the beginning, before all the hate, they even felt really embarrassed about it. Don’t expect a vindictive, well-organized group to do the same thing under new rules. This year Vox already switched it up by picking lots of stuff that both he likes and vast swaths of Worldcon fandom like too. Next time they’ll get even more creative. This is not a political group trying to get its slate of candidates into power. You’re looking at all of this through the wrong lens.
Mostly argle bargle.
Hampus, I expect that from you, but don’t encourage Jameson.
Sigh.
When you have something to say besides “EPH bad, sad puppies misunderstood“, I’ll be here reading it and responding if appropriate.
@Jameson Quinn on May 24, 2016 at 9:49 am said:
But I did not propose +5. I proposed letting people “top up” their nominations up to 5. Slate voters would get +0 because they would already have 5 nominees in the list.
@Hampus Eckerman
I’m not sure how well that would work in practice, though. It amounts to yet another small group voting, and it’s a vote that cannot be overriden.
Something to think about a bit.
Jameson, I have reservations about EPH, but I’d gladly take it over what you guys are cooking up now.
Try to think about what I said OK? You’re the election systems guy, and it’s understandable you are invested making this work. But you need to move to a different frame of reference to crack this one.
Try to think about what I said OK?
Why? You’re a broken record that does nothing but spout idiocy over and over again.