55 thoughts on “Re-Entering The Lists: Measuring The Slates’ Impact on the Final Hugo Ballot”

1. Compare this 2001-2005, when a single nominee out of 28 (3% overall, 0% 3 years out of 4!) was not from the two dominant recommendation lists of the time.

   In 2015, 60% of the Best Novels come from something other than the dominant rec lists.

   The argument that this year is not inclusive and diverse is complete bunk.

   2015 has greater diversity from rec lists than previously. By a blowout.

2. You left translator Lia Belt off of “The Day the World Turned Upside Down” by Thomas Olde Heuvelt.

3. Congratulations ULTRAGOTHA! You are awarded the title “File 770’s Savior Proofreader of the Day” with all the rights and privileges appurtaining thereto. Thanks!

4. Thank you. I’d like to thank the Academy and my parents.

5. So Puppies are now claiming there were “dominant recommendation lists” in the past? How did they dominate? Did they choose five works in each category, like Puppies did? Did they ask people to vote for every nominee in lock step, the way Vox Day did?

   “In 2015, 60% of the Best Novels come from something other than the dominant rec lists.”

   That’s only because Correia and Kloss declined their nominations. It would be 20% otherwise.

6. xdpaul: I’m honestly (truly!) curious. What were the two dominant recommended lists of 2001-2005? Are you referring to the Nebula shortlist, or something similar?

7. rcade: We don’t know that. It could be 40% because we don’t know how many votes Ancillary Justice or The Goblin Emperor received. It could be 20%.

8. Answering my own question, it’s definitely not the Nebula shortlist. So says wikipedia.

9. It could also be 0%; it’s not inconceiveable that Gannon declined a nomination without making any public announcement.

10. The Locus list is probably one. Who knows what the other would be.

11. I wouldn’t think the Locus Awards would be it as they have a gazillion novels on their “shortlists”.

12. Nevermind. Correction. I forgot about their tiered nomination process. But doesn’t the locus shortlist come out after the Hugo shortlist?

13. Mike Kerpan: Due to the 2001-2005 reference I believe he is referring to the two lists Frank Wu was quoted as naming in a Castalia House blog post, which were Emerald City and NESFA.

    While the NESFA list is doubtless an important focus of conversation among club members, and in an environment where 40 nominating votes is a big deal that matters, I never saw it treated as an opinion leader in online discussions the way Emerald City’s list was.

14. MG: Some interesting overlap in that Venn. I’d like to see the data extended beyond those years. One interpretation of that diagram is “wow those guys sure did agree on a lot of fiction.” Another would be, “A recommended reading list comprising 20 or so books would be a great idea to get our readers jazzed up for nominations.”

15. The NESFA list for 2005 recommended 20 novels:

    http://web.archive.org/web/20071012083232/http://www.nesfa.org/recommends/hugos05.html

    A recommended list that contains a large number of works is likely to contain a lot of overlap with Hugo nominees.

16. Okay, I’ve found the 2003 Hugo recommendation list from Emerald City, which also contains links to earlier lists. And here’s the NESFA archive of recommendation lists.

    A quick crunch of the numbers for Best Novel for 2003, being in the middle of the previously cited interval of 2001-2005:

    22 recommended by Emerald City.

    34 in the main recommendation list from NESFA, plus 18 more later in the page under “Other Good Reads”, for a total of 52.

    10 novels are on both the Emerald City and main NESFA lists, for a total of 46 novels between them.

    If you add the “Other Good Reads” list, the overlap is 13 and the total is 61.

    If you get 61 guesses about what’s going to be nominated in one category, your odds for hitting all 5 somewhere in there should be pretty good.

17. I could wish that the Hugos were nominated and awarded *two* years after publication. That would allow people to discover and read novels that get lost in the shuffle. Speaking for myself, I depend on a network of friends and on the recommended reading lists to find novels worth reading. Eric Flint spoke truth on his essay when he noted that no one has the time to keep up with the genre anymore. I read 43 new novels in 2014 and only thought two of them were worthy of Hugo nominations. I would like that ratio to be much higher.

18. 4-time Hugo winner Wu (7 time nominee, I believe) simplified it for everyone. If you weren’t on one of the two rec lists from 2001-2005, you would not win, and were among the 3% who even got nominated.

    Kevin Standlee endorsed Wu’s research: it seems to have been fairly accepted back then recommended lists by interested fans was how the nomination process worked.

    rcade is right: if you count the declined novel nominations, this year’s two dominant rec lists account for 80% of the Best Novel. That is still significantly less than the 4-plus-year run that the previous twin lists accomplished. A whopping 97% lockout rate. (That is, if you weren’t on their lists, you had very little chance of making it through. Period. Furthermore, if you were the lone outsider (Humans) who got nominated, you would not win.)

    So for everyone upset by the small selection of SP and RP and begging for Rabid Puppies to increase their number of suggestions…I think you should recommend that they do so for next year. I see no reason why they would not consider such a crafty plan…

19. While the NESFA list is doubtless an important focus of conversation among club members, and in an environment where 40 nominating votes is a big deal that matters, I never saw it treated as an opinion leader in online discussions the way Emerald City’s list was.

    Wu’s figures indicate that there was something other than online discussions driving the nomination, then, because NESFA had five nominees and two winners in that time frame that were not on the other list. Independent of the list EC had one more nominee (six) but only one winner.

    Combined, the lists jointly (lockstep?) advocated sixteen nominees and two winners.

    You can’t dismiss “common overlap” on those old dominant lists without dismissing “common overlap” on the new dominant lists.

20. If Sad Puppies put 20 works in each fiction category and treated all 20 equally, I don’t think anybody would be calling their efforts bloc voting.

    “If you weren’t on one of the two rec lists from 2001-2005, you would not win, and were among the 3% who even got nominated.”

    As several people have said, it’s apples and oranges. Absent bloc voting shenanigans, I could put together a list of 30 novels before the Hugo nominations vote and get 4 or 5 on each year’s ballot. Pros, reviewers and fans publicly discuss the books they most liked as Hugo season gets started. Watching their comments indicates what’s getting the most award buzz.

    But this wouldn’t mean I was exerting influence on the vote, because I was just observing the process.

21. xdpaul: “there was something other than online discussions driving the nomination, then”

    Is it something you are leading up to, or do you mean a literal x-factor? In 2001-2005, discussion in paper fanzines was still a common channel of influence.

22. Before you go very much further down that road, rcade, I suggest that you do what many others have done, including myself, and do the figuring, because mathematics are not on your side in this fight.

23. Xdpaul: comparing 3% to 60% is faulty analysis.

    1) for the 3% from prior lists you count a source that combined 4 categories, and only count 1 category in 60%. Since rp/sp dominated the novella/novelette/short story categories, including these numbers will work against your argument, because 100% or close to it of nominees in these categories needed to be on the slate to win. In any case a more accurate comparison would be 3% vs 40% (likelihood of being nominated if not on slate) before you even count for the adjustment above

    2) as others have said the much larger number of recommendations in the other reading lists makes it more likely that well-received work would make it to the top, and less likely that simple presence on the list guarantees a nomination. This would decrease the correlation between being recommended and being nominated, as compared to the puppy slates

24. Is it something you are leading up to, or do you mean a literal x-factor?

    No, lead-up I meant a literal x-factor. My point is that the NEFSA list was hugely influential during that time, even if they were offline. The clear message was that if you wanted to be invited to the dance, you needed the attention of one or the other, but preferably both lists.

25. You read Wu’s chart wrong, Learned. We are looking at the Best Novel category only.

26. Re: 3% vs. 40% scratch that point. It would only be true if the Liu book hadn’t been added to the list due to kloos’ withdrawal

    My other points stand

27. So, just so I’m clear, everyone here is pretty much in favor of Rabid Puppies recommending 10-15 candidates per category next year? The error of the rec list is not in presenting a list, but in not presenting enough of one?

28. “The clear message was that if you wanted to be invited to the dance, you needed the attention of one or the other, but preferably both lists.”

    You’re inventing this.

29. Xdpaul you are correct that I read the chart incorrectly. Thank you for correcting me

    The best criticism that comes to mind is that the correlation between being put on a slate and being nominated for a Hugo is much lower for these other recommendation lists. I.e., even if they have a higher % of including a nominee on their list, they have a much lower % likelihood of getting any individual work on the list

    So the impact of a slate goes down

    I’m reminded of hayek’s words on the free market: how a bunch of small individual decisions are better than centralized decision making. This principle would suggest the puppy slates would produce inferior results

30. “So, just so I’m clear, everyone here is pretty much in favor of Rabid Puppies recommending 10-15 candidates per category next year?”

    Rabid Puppies wants to burn the Hugos down. Why do you expect anyone to be negotiating with destruction, as if there’s a middle ground to be found with them?

31. As a NESFAn, I’m ashamed to say that I forgot about our list. It’s form and methodology are so different from the puppy slate that I didn’t see how it would be seen as comparable. For SP3 to have been like the NESFA list, Brad Torgerson would have had to include every single suggestion made on his list rather than “currating” the suggestions and forming a slate. If he’d simply copied the methodology of the NESFA list, we wouldn’t be having this discussion.

32. Good, Mike. I honestly think SP and RP might consider increasing the number of recommendations.

    Of course, the likelihood is that an extended list is more likely to do what happened in the past: assure that the vast majority of nominees in all categories are Rabid and/or Sad.

    I just wanted to be sure I was understanding the RP opposition’s general position. A return to 01-05…puppy style!

33. Xdpaul:

    I think if the SPs and RPs had presented a list of 20 novels (or whatever) each, even if there were much overlap, we would not be having this discussion right now. Something tells me that VD won’t go with that gambit next year.

    But yes. I would remove all objection to the various puppies if they had recommended lists rather than dictatorial slates.

34. Yeah, the notion that a “suggested reading list” like the ones Locus and NESFA put out is the same thing as the much shorter targeted SP/RP Hugo slates is bogus. I know xdpaul knows better too as that subject already came up in previous discussions here:

    https://file770.com/?p=20905&cpage=2#comment-246352

35. xdpaul:

    You’ve got it! The implicit message of precisely 5 suggestions is “Vote this slate.” The implicit message of 15 suggestions is “Here are some interesting works to check out.”

    Of course, the likelihood is that an extended list is more likely to do what happened in the past: assure that the vast majority of nominees in all categories are Rabid and/or Sad.

    Look at it this way: 5 nominees out of 61 suggestions means 92% of the suggested novels on the combined 2003 Emerald City and NESFA lists failed to make the cut. Getting a book onto the list still means its chances of being nominated are low. What’s setting people off this year is that getting onto a Puppy slate nearly guaranteed nomination. If future Puppy lists expand far enough to have a similar failure rate, few people will mind as much if some of the nominated works do appear on them.

36. I will state for the record that if Sad Puppies #4 has a list of 10-15 works per category, I, for one, will not make any anti-puppy comments about that.

    I’ll get back to you on the Rabid one, but if they do the same so be it.

    If we see another list of 4 or 5 items per category, then I can guarantee we’ll be back here in 12 months digging deeper trenches.

37. Huh. Interesting to see the usual wingnut discussion tactics work in a virgin field. That little idiotic gambit of comparing the Puppie slates to recommended reading lists should just be laughed off, because any moron who isn’t willfully blind knows the two are nothing alike.

    xdpaul knows this, because he’s not an idiot, just a troll.

38. xdpaul:

    Yes, a long list of recommendations would be constructive; a slate is not.

    90+% of people who hate what SP/RP have done to this year’s awards would be completely fine with either publishing longlists of recommendations.

    Actually everyone who publishes their ballots or recommendations should take this year as a lesson and publish longlists instead.

39. Mike, thank you so much for doing the hard work of compiling all these links. Much appreciated.

40. @ Xpaul

    “So, just so I’m clear, everyone here is pretty much in favor of Rabid Puppies recommending 10-15 candidates per category next year? The error of the rec list is not in presenting a list, but in not presenting enough of one?”

    Speaking strictly for myself, as I don’t give a flip what VD does – if we could get multiple slates that managed to cover more than than the top 10 novels of the year (and get even more indepth for short fic -lets remember that we’ve had several years where we’ve had less than 5 noms for short story) between them, I would be ecstatic.

    The recent Hugo slates have been, overall, *good*, but they’ve failed to sample ALL the good in SFF, and that’s bad.

41. If they present a list that is _longer_ than the slots for nominations, and switch to recommending that people READ the works and make their OWN decision, which to be fair to the SP’s they did. Then yup, you got no argument from me.

42. Petrea Mitchell wrote that the implicit message of precisely 5 candidates is “vote this slate.” Not even implicit but explicit in the case of the Rabids. It has an intro urging voters to vote precisely these candidates.
    Sads are not so explicit, but their intro still has a campaigning air that the Nesfa lists lack.

43. I’d like to see at least ten candidates for each category, reviewed and discussed prior to nomination. If we’re going to have slates, then build them with new readers in mind– Introduce someone to a book they wouldn’t have read otherwise, tell them why you like it and what it means to you. I’m an atheist, I’ll always be a hard sell for Christian SF&F, but if it’s presented in good faith I’ll read it. I’ll give it a chance. I know who Aslan is and I don’t mind.

    –Besides, the only way to win “Culture War” is not to play.

    In that spirit, I’ll read the ballot and if a story has the spark I’ll vote for it.

44. @rcade

    “Rabid Puppies wants to burn the Hugos down. Why do you expect anyone to be negotiating with destruction, as if there’s a middle ground to be found with them?”

    You keep saying this. It doesn’t make it true. SP 1 was, “well make your own slate,” SP2 was, “well get people to vote [for your trash]”, SP3 was, “include people you don’t agree with!” (which was done in 2 but done more forcibly in 3 and RP). We completed our goal every time. I imagine we can do it with 15.

    As XD notes this will likely give us 100% control. I wouldn’t mind you folks officially announcing you would not complain about such a thing. In very public comments. This might be fun. And I want _specific_ wording. I’ll present the proposal, as only a minion can, to the Supreme Lords of Evil.

    @Daveon,

    “I’ll get back to you on the Rabid one, but if they do the same so be it.”

    Nope no weasel language. In or out. Don’t be a pansy.

    “and switch to recommending that people READ the works and make their OWN decision”

    Nope we did this. Every. Damn. Time. People here have tried and completely failed to catch me on that multiple times now. I’ve even already read the “Upsidedown” replacement story. Do try harder to pay attention/lie less.

45. There is no “culture war.” The biggest secret in the world the social justice warrior community doesn’t know about is this: people are people.

46. “A good novel tells us the truth about its hero; but a bad novel tells us the truth about its author.” – G.K. Chesteron

    “What follows is the list of Hugo recommendations known as Rabid Puppies. They are my recommendations for the 2015 nominations, and I encourage those who value my opinion on matters related to science fiction and fantasy to nominate them precisely as they are.” [Vox Popoli (2 Feb. 2015) Located at http://voxday.blogspot.co.uk/2015/02/rabid-puppies-2015.html (Last Accessed April 17, 2015.]

47. I’ll still object to ANY campaigning, slates, bloc voting…and given thisnyear, if the puppies want to do a list that isn’t going to continue to receive condemnation, they’d do well by themselves to knock it off for a year or two.

    What I see in this discussion is nothing more that someone saying “OK then, if they take their campaigning underground, it will be OK then, right?”

    No. Between them they’ve now got a list of a couple of hundred people who have already demonstrated the ability to march in unison. All you’re really saying XCDPauld is – we’ll put a socially acceptable recommended reading list out there, and meanwhile, under cover of trying it accommodate, they’ll still be activating their minions and voting a recommended slate.

    I’d much rather have them do no agitating at all. failing that, do your dirty business in public.

    Someone who “unintentionally” fucks up an entire culture usually has the courtesy to back off, at least for a little while, if they’re looking for acceptance.

    If they want to erase all suspicion, do the mea culpa and then wait a year or two. If there is never any acknowledgement that what they did was wrong – within the rules but wrong – then they’re still wrong.

48. Hey, Davidson, at this point I’d say any fiction not funneled through an ideology of mindless identity-hatred to be “acceptable recommended reading.” Captain Future would look like frickin’ Shakespeare right now. For someone who’s been a victim of that mindless swarming ideology I’m surprised you’re ignoring the man behind the curtain.

49. The problem with the diagram is it is missing a circle of non-Emerald City/NESFA nominations(the ‘human’ votes that didn’t make the nomination cut). And even then this will only be significant if we presume all books on the EC/NESFA list got at least one nomination.

    Brad had a chance to put together a long list, but chose not to. He said that when he put out requests he got back 35-40 suggestions. He could have just put them up for consideration.

50. I note that after an easy proposal people are moving the goal posts again. Afraid of us secretly conspiring, how pray tell, do almost two-hundred people + do that on the interwebz?

    In other words, it boils down to go home. We aren’t.

Comments are closed.