How “Notability Still Matters” Would Have Affected the 2017 and 2018 Hugo Long Lists

Guest Post By Dave Wallace: One of the proposed WSFS constitutional amendments up for ratification this year in Dublin is C.3, “Notability Still Matters.” [(Dublin 2019 Business Meeting Agenda as of August 5)]  What it would do, if ratified by this year’s business meeting, is allow the Hugo Award Administrators to omit from their report on the nominations any entry that got less than 4% of the nominating votes in that category, unless there is a previous entry getting more than 4% of the votes that was eliminated in an earlier round which was reported.  (Currently, the Hugo Administrators are supposed to report the last 10 rounds of eliminations in their report).

While it is true that the Hugo Administrators are allowed to voluntarily publish more information in their report than the constitution requires, the constitution is the primary way the business meeting can give the Administrators binding instructions, and there’s not much point in amending the constitution if we expect the Administrators to routinely ignore their instructions.  Therefore, I’ve taken a look at how this amendment would have affected the Hugo Long Lists in each category for 2017 and 2018 (the previous years for which EPH was in effect) if the amendment had been in effect and the Administrators had followed it strictly.

I took the nomination data from the following published reports:

2018 (pp 20-26): https://www.worldcon76.org/images/publications/2018DetailedResults.pdf

2017: http://www.worldcon.fi/wp-content/uploads/2017/08/HugoReport2_nominations.pdf

http://www.worldcon.fi/wp-content/uploads/2017/08/HugoReport3_nomination_details.pdf

For 2017, most of the necessary information is in Report2, although the details of the eliminations are in the tables in Report3.  For 2018, both the total nominations and the % of the nominating vote each entry got are presented in the original tables, making it much easier to see the effect of the amendment.  One further note: in 2017, the last 10 rounds of eliminations were presented in each category, yielding 16 entries in each long list.  In 2018, only the last 9 rounds were presented in most tables (except Novelette), so most long lists only had 15 entries.  Thus the effect of the amendment would likely be even greater in 2018 than what I present here, if the #16 entries were also considered.

Here is my summary of the number of entries that would have been deleted from the long list for each category if the amendment had been strictly observed, followed by the details of what would have been omitted:

Number of Nominees Lost from Long List with 4% Notability Threshold

Category20172018
Novel00
Novella11
Novelette10
Short Story74
Series00
Related Work30
Graphic Story62
Dramatic Long00
Dramatic Short30
Editor Long01
Editor Short02
Pro Artist02
Semiprozine00
Fanzine04
Fancast14
Fan Writer21
Fan Artist30
Young Adult/Lodestar0
Campbell02

In Best Novella, we would have lost Chimera in 2017, and In Calabria in 2018.  In the Best Novelette category, we would have lost Tansy Rayner Roberts’ Kid Dark against the Machine in 2017.  Best Short Story would have been most affected.  In 2017, we would have lost Lavie Tidhar’s Terminal (Tor.com), Seanan McGuire’s Ye Highlands and Ye Lowlands (Uncanny), Cat Rambo’s Red in Tooth and Cog (F&SF), Aliette de Bodard’s A Salvaging of Ghosts (BCS), Rebecca Ann Jordan’s We Have A Cultural Difference, Can I Taste You? (Strange Horizons), Peter S. Beagle’s The Story of Kao Yu (Tor.com), and Aliette de Bodard’s Lullaby for a Lost World (Tor.com).  In 2018, we would have lost Mareen F. McHugh’s Sidewalks (Omni), Naomi Kritzer’s Paradox (Uncanny), Nick Wolven’s Confessions of a Con Girl (Asimov’s), and Nancy Kress’s Dear Sarah (Infinity Wars).

The 2017 Related Work category would have lost Rob Hansen’s THEN: Fandom in the UK, 1930-1980, Diana Pavlac Glyer’s Bandersnatch, and André M. Carrington’s Speculative Blackness.  Graphic Story was another category that would have been heavily affected.  In 2017, we would have lost Clean Room, Vol. 1; Injection Vol. 2; Lumberjanes Vol. 4; Pretty Deadly, Vol 2; Decender, Vol. 2, and Oglaf (Bodil Bodilson). In 2018, we would have lost Unbeatable Squirrel Girl, Vol. 6; and Above the Timberline.  The 2017 Dramatic Presentation Short Form would have lost Chapter Seven: The Bathtub and Chapter One: The Vanishing of Will Byers from Stranger Things, along with Salvage from The Expanse.  2018 didn’t lose any Dramatic Presentation entries, but Game of Thrones: The Spoils of War came pretty close with only 4.03% of the vote.  In the Best Editor categories, 2018 would have lost Gillian Redfearn from Long Form, and Marguerite Kenner and Trevor Quachri from Short Form.  In Pro Artist, 2018 would have lost Likhain and Dan dos Santos.

Fanzine and Fancast are also categories that would have lost significantly: 2018 Fanzine would have lost Camestros Felapton, Quick Sip Reviews, Ansible, and SF Commentary.  2017 Fancast would have lost Vaginal Fantasy, while 2018 Fancast would have lost Kalanadi, Fast Forward, Get to Work Hurley!, and Eating the Fantastic.  In Fan Writer, 2017 would have lost O. Westin and Cora Buhlert, while 2018 would have lost the memorable Chuck Tingle.  The 2017 Fan Artist category would have lost Liz Argall, Lauren Dawson aka Iguanamouth, and Simon Stålenhag.  Finally, the 2018 Cambell Long List would have lost Annalee Newitz and Erin Roberts.

Comments and Opinions:

Having looked at what the impact of the amendment would have been, I think this amendment would do significant harm to the value of the current long lists, and should be rejected.  I don’t know how many other people look at the long lists, but I do and I value the information that is there.  This value can take several forms:

First, for those who nominated entries on the long list, knowing how your entry placed gives you a form of validation that other Hugo voters also found that entry worth of nominating, and a sense of how close you came to getting that work on the ballot.  I nominated two of the short stories that would have been left off under the amendment, and I appreciate having this information.

Second, it should be apparent from the Short Story results above that we are not just talking about omitting minor works from artists very few voters care about, but significant stories from some major names in the field.  While many of these stories were published online, there were also several stories first published in traditional print media, unlike most of the finalists.  Keeping such stories in the long list helps others seek out these stories, and may help make a case for splitting the category in the future, if there is a persistent bias against print media with our current categories.

Third, particularly in categories where entries tend to repeat from year to year (e.g., Editor, Artist, Zines, Fancast, Fan Writer, Graphic Story), presence in the long list can help an entry find an audience for future years.  In 2017, I made a point of trying to listen to at least an episode of each fancast on the long list, in order to be able to better appreciate and nominate fancasts in the future, and I found several fancasts that I now listen to regularly.  I did not do the same for 2018, but I see at least one entry from the list of potential losses that I would like to check out further.

Fourth, those who have nominated long list entries that are ultimately eliminated have an opportunity to audit the results of the EPH implementation by seeing if the change in points when that work was eliminated are consistent with the rest of their ballots.

Finally, there doesn’t seem to be any compelling reason to make this change for the normal Hugos.  The system we have now seems to be working well, providing good information to those members who choose to read it.  The minutes last year mention the tail of the retro-Hugo nominations having relatively few votes in some categories, but most future years are not going to have retro-Hugos.  It doesn’t make sense to me to make such a drastic change just to shorten the rare retro-Hugo report a bit.

[Originally published on Dave Wallace’s Livejournal.]

31 thoughts on “How “Notability Still Matters” Would Have Affected the 2017 and 2018 Hugo Long Lists

  1. Thanks for this, Dave.

    I didn’t know this amendment was up for voting and while I understand that the workload on Hugo administrators is already immense, I still like getting the detailed longlists. And with the increased number of Hugo nominators in recent years (which is a good thing), chances of nominees hitting the longlist without getting 4% of the nominations are increasing.

    Finally, I also like being on the longlist for best fanwriter in 2017.

  2. Thank you Dave, that’s a lot of great work!

    I highly encourage anyone interested in this or any other item on the agenda to attend both the Preliminary Business Meeting on Friday at 10:00am at the Gibson Hotel in the Stratocaster A room, and the Main Business Meeting session on Saturday at 10:00 in the same room.

    Items can be voted off the agenda at the Preliminary Business Meeting and we also set debate times. So it’s a very important meeting!

  3. Is there any downside to providing information below the 4%?
    Having said that, I don’t think I’d noticed that fanzine result in 2018 and if I didn’t notice then I can hardly claim it is of information of any value.

  4. Camestros Felapton: This is what they’re supposed to do now:

    3.12.4:The complete numerical vote totals, including all preliminary tallies for first, second,…places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period, the results of the last ten rounds of the finalist selection process for each category (or all the rounds if there are fewer than ten) shall also be published.

    The “Notability Still Matters” change would knock a few lines out of the report, but so far as I can tell the calculations themselves would still have to be done by the administrators.

  5. Well, if the calculations still have to be done anyway, I don’t see the point behind this amendment at all, because adding a few lines to the report can’t be that much work

  6. Mike Glyer on August 6, 2019 at 6:49 pm said:
    The “Notability Still Matters” change would knock a few lines out of the report, but so far as I can tell the calculations themselves would still have to be done by the administrators.

    Hmmm…well I’m not the one having to do the work but I’ve wrangled a lot of spreadsheets and data into reports over the years. It sounds like the expenditure of effort spent debating the measure would exceed the effort saved.

    I feel like I’m missing what the objective is here.

  7. Camestros Felapton: If you aren’t taking at face value the makes-it-easier-on-the-Hugo-Administrator argument, I’d say look to the caption — “Notability Still Matters.” The implicit message is that the movers think it’s their job to keep some works from having whatever cachet there is from being on the longlist report.

  8. Thank you for this, Dave. I was planning on making a similar (if much briefer) point to the Business Meeting when this came up.

    There was some discussion at last year’s Business Meeting that this was (in its original form, with 5% instead of 4%) an attempt to restore the reporting requirements to what they were prior to the adoption of E Pluribus Hugo. However I couldn’t find any similar rules in my perusal of old WSFS Constitutions and Hugo longlists, except for not reporting anything that got less than 5 total nominations. (In that spirit I offered an amendment to replace “less than 4%” with “fewer than 10 nominations”, but it was rejected.) I’d like to hear thoughts on this from those who have been going to Worldcons longer than I have.

    Martin

  9. Well, one thing that hitting the Hugo longlist does is that “Nominations below cutoff” gets added to your ISFDB entry, if you have one. How much value that has is the question.

  10. Hi, Cora. Thanks for your comments. I’m also glad you got to be on the long list for Fan Writer. I’ve been reading some of the stuff you’ve written this year about the Hugos, and hope we’ll get a chance to meet in Dublin. (Alas, I did not have time to watch Münchhausen before the close of Hugo voting this year, but your post got me interested in seeing it.) My wife Jan (TeaLover on Ravelry) mentioned that she’s exchanged posts with you about the Raksura project, so you’ll probably meet her at some point also.

  11. I think others above have disposed of the “admins workload” argument. If the other element of the argument is that the works being removed were in some way not notable enough, then I’d very much disagree with that.
    I recognise most of the people/works named which to me suggests they are at least moderately notable.
    As an example, 2017 short story seems to be the worst affected and I’m pretty sure I read every story listed – and think each was at minimum worth noting on a longlist. In fact I think I nominated two of them!
    (ETA, having found my old email, Red in Tooth and Cog and We Have A Cultural Difference, Can I Taste You? were on my ballot that year.)

  12. Cora Buhlert: Well, if the calculations still have to be done anyway, I don’t see the point behind this amendment at all, because adding a few lines to the report can’t be that much work.

    My understanding is that the intent of the proposal (or at least its stated intent) is to prevent people who only got a handful of nominations from being embarrassed and/or humiliated by it becoming known that they got “only” a handful of nominations.

    My contention is that even a handful of nominations, these days, is an indication of honor and esteem that people like you would be pleased to attain, and that people like me are glad to know about, since it may give me ideas of new works and authors to read.

    Given current participation numbers in nominations, the only real concern for tiny numbers is in the Retro Hugos — and those people are almost without exception deceased at this point.

    And given that the sponsors are two people who are well-known in WSFS for gatekeeping attempts in the Hugo Awards, I think, as Mike points out, that the proposal’s title is a dead giveaway: The proposers think that getting “only” a handful of nominations means that the nominee is not “notable”, and they should be denied a place on the longlist due to a lack of importance.

    I disagree, and I will be voting against this proposal.

  13. ULTRAGOTHA: The ratification of “Notability Still Matters” will come up at the main business meeting on Saturday (or a later session if it doesn’t come up Saturday before we have to adjourn for the day). We established a couple of years ago that amendments up for ratification are not subject to being Postponed Indefinitely at the preliminary business meeting on Friday, so it can’t be stricken from the agenda there. (But the preliminary business meeting will set the debate time, so that meeting is still significant).

    I am, however, thinking strongly about moving to Postpone Indefinitely on the new constitutional amendment D.7 (Five and Five), which would get rid of 5&6 permanently, on the grounds that it is unnecessary, given the ability of the business meeting to suspend 5&6 each year and then sunset it in 2022 if they really don’t like it. As I noted in my comments on the thread about that, I think there is a stronger stand-alone case for the one-year suspension motion B.4 (Suspend 5 and 6 for 2020) if it isn’t tied to D.7, and we can have the debate about the merits of 5&6 there.

  14. Mike Glyer on August 6, 2019 at 8:47 pm said:

    Camestros Felapton: If you aren’t taking at face value the makes-it-easier-on-the-Hugo-Administrator argument, I’d say look to the caption — “Notability Still Matters.” The implicit message is that the movers think it’s their job to keep some works from having whatever cachet there is from being on the longlist report.

    oh…ok
    I missed the nuance — which coincidentally is what will be my epitaph

  15. Mark: I too nominated Red in Tooth and Cog, and I thought it would make a good example for how I could use the info to audit EPH. But it turns out that my nomination didn’t count. Why? Because I was one of 12 people who trusted the classification in the F&SF table of contents and nominated it as a Novelette rather than a Short Story. And because my Short Story nominations were otherwise full, my Novelette nomination got discarded rather than transferred to the story as a Short Story (Rocket Stack Rank has it around 6800 words).

    So I think the lesson is: (1) don’t trust the publisher’s classification of the story for print media, (2) either audit the word count yourself, or leave room on your nominating ballot in adjacent categories. The other lesson is that the only reason I know about all this is because the story was on the long list, and the transfer of category got discussed in the Administrator’s report for that year (and because I got interested in what happened to my ballot because I was writing this article).

  16. @Dave,

    Yep, I always check word count myself – but I get everything on ebook so that’s easier to do, whereas print readers have to trust the publishers categories.

  17. “Notability still matters” as a title reminds me both of Wikipedia — which I suspect the proposal writers had in mind — and TVTropes’ “there is no such thing as notability.”

    Given the choice, I’d rather be saying “Congratulations, you are one of today’s lucky 10,000” than “why do you care about that crap, nobody else does?”

    Wikipedia’s idea of “notability” can seem a lot like “other people with websites already think this is notable,” and award nominations can be used as evidence that a writer is notable. The assertion that “notability still matters” implies that someone is worried that the Hugo longlist will be used as a form of advertising, and that this is somehow worse than putting “Hugo Award Winner” on a book cover..

    I don’t see the harm in someone, possibly, using “that story was on the 2017 Hugo longlist” as an argument against deleting a Wikipedia article.about an author.

  18. Well, one thing that hitting the Hugo longlist does is that “Nominations below cutoff” gets added to your ISFDB entry,

    Sometimes. I don’t think my non-finalist nominations have.

  19. I have seen that argument about “to prevent people who only got a handful of nominations from being embarrassed and/or humiliated” before in several discussions. But looking at the table above, I’m not sure how relevant it is anymore.

    I’m against this for a specific reason. The table shows that six candidates would have been removed from the 2017 Graphic Novel category and two candidates from 2018. It is, apart from short story, the one category most affected.

    Looking at the list below of what would have been removed, I do find all of them I have researched being good comics and nothing that would be put on a list to humiliate anyone. Above The Timberline is wonderfully drawn and is not something I would want people to miss out on knowing it exists. Many items suffers from being later volumes in a series, but that goes for the typical Hugo finalists too.

    I do think this category suffers from Hugo Voters not really trying to do original research into comics, instead preferring to read what they know they have liked before, voting consistently for a small number of candidates. To then shorten the longlist would be to make a disservice to those who want to see what different comics are available.

    2018
    * Unbeatable Squirrel Girl, Vol. 6: Who Run The World? Squirrels
    * Above the Timberline

    2017
    The Wicked and the Divine, Vol. 3: Commercial Suicide
    Mockingbird, Vol. 1: I Can Explain
    Clean Room, Vol. 1: Immaculate Conception
    Injection Volume 2
    Lumberjanes Vol. 4: Out of Time
    Pretty Deadly, Vol. 2: The Bear
    Descender, Vol. 2: Machine Moon
    22 12.17 Oglaf (Bodil Bodilson)

  20. @Dave Wallace
    Glad you enjoy my posts. So Tealover is your wife. We’ve been enjoying photos of her contributions to the Raksura Colony Tree project over at Ravelry.

    @JJ
    Personally, I think it’s a reason to be thrilled that people think my work was one of the best five of the year, even if it’s only a handful of people. And while I recall a pro artist with only 4 nominations on the Retro Hugo longlist last year, that artist is long dead and the cut off for the regular Hugos are usually in the 20s or higher.

    Also the whole phrasing of the proposal “Notability still matters” is more offensive than five nominations could ever be, because it implies that works/people who’d drop off the longlist are not notable (so Cat Rambo, Naomi Kritzer, Stranger Things, Squirrel Girl, Mockingbird Descender, etc… are not notable now) or at least not notable enough. Which seems to be the intention

    Finally, as others have said, I like checking out if other people liked that great short story I nominated that didn’t make it as much as I did. I also like looking at the longlist to find new things to read or watch or even to be reminded of someone/something I have forgotten to nominate, but could nominate next year. For example, seeing O. Westin of MicroSFF on the longlist (who’d have fallen off according to these rules) made me go, “Oh, I love MicroSFF, but was never sure where to nominate them. But best fanwriter is a great solution. Jot them down on my personal longlist for next year.”

    Also, for future genre historians it is as important to what didn’t make it as what did. Because that person with ten nominations might be a future genre superstar.

  21. JJ Said

    My contention is that even a handful of nominations, these days, is an indication of honor and esteem that people like you would be pleased to attain, and that people like me are glad to know about, since it may give me ideas of new works and authors to read.

    While I agree with you that even being in the tail of the long list is an honor, given how MANY works are out there in any category, I have seen authors on Twitter be unhappy about seeing their position on the long list. I hope any that feel that way come to the main business meeting and make their feelings known.

    I haven’t seen very many folk that feel this way, but if they’re out there I think we need to listen to them and take that into account when we vote.

    Dave, I’d forgotten this was a carryover from last year. Though my point about attending both sessions still stands for the new things on the Agenda.

  22. Hampus Eckerman: I have seen that argument about “to prevent people who only got a handful of nominations from being embarrassed and/or humiliated” before in several discussions.

    That’s also the argument that was made against revealing the Hugo statistics at all, or why some pre-1978 lists of winners showed the first three places (out of 5 finalists).

    Over the past decades, I have seen many people interested in where they showed up on the longlist, hoping to do better in another year, not embarassed.

    There have been a few finalists who were vocally unhappy to finish behind No Award — of course, for the longlist that’s not an issue.

  23. ULTRAGOTHA: While I agree with you that even being in the tail of the long list is an honor, given how MANY works are out there in any category, I have seen authors on Twitter be unhappy about seeing their position on the long list.

    The Hugo Awards, and the Longlists, are done for the benefit of fans, not for the benefit of authors. If someone is disappointed to see themselves on the longlist, it’s difficult for me to feel a huge amount of sympathy: would they rather that they hadn’t even gotten enough nominations to make the longlist? Really?

    The Longlists exist so that fans can see what other fans liked and nominated. While I recognize that a lot of authors are also Worldcon members, it’s not an author’s place to dictate to Worldcon members how to run the Hugos in order to cater to their authorial ego.

    If it bothers them so much, then they don’t need to look at the longlists.

  24. John Hertz responds by carrier pigeon:

    There’s also the historical perspective.

    Historians – including archeologists – all too often find reason to learn about something deemed insignificant in its own time. Having to search through waste paper, or garbage dumps, is hardly unknown.

    A lot of work is saved if records someone bothered to keep are found.

    I mention this point not seeing it discussed.

  25. I appreciate that the longlists exist. They point to creators & works that at least some people have found worthy and I regularly see people in the longlists that become more prominent in succeeding years. So for me, it is a useful resource to bringing attention to good works & up & coming creators.

  26. Kalanadi making the Longlist in Fancast was very exciting news because Rachel was the first BookTuber to do so. That community was thrilled to see it.

    There are a few of my nominees in other categories that would have been left off too. Definitely against this proposal.

  27. Mike Glyer on August 7, 2019 at 11:45 am said:

    John Hertz responds by carrier pigeon:

    There’s also the historical perspective.

    Very true. I’ve had caused to look at longlists recently and it is useful to see who and what wasn’t quite on people’s radar.

  28. Looking at this proposed amendment, I’m not sure what it’s supposed to accomplish. I don’t view “because we did so in the past” as a good reason in and of itself. The paperwork should be all in place for keeping track of the last ten rounds used in the EPH process, all this does is adding an extra (and not mandatory) thing for the Hugo admistrator to check.

  29. Hugo Administrator Nicholas White has published the Retro-Hugo results for this year and has done the equivalent analysis for the Retro-Hugos in this twitter thread. I can see his point for the Retro-Hugo ballot, and I think people would care much less about the long tail there. (It does, however, provide a useful example of how the EPH tiebreaking rules work, and may provide some data about Hugo nominating patterns that is harder to see with bigger datasets). I very much appreciate all the work he has done on the Hugo reports, including this analysis.

    I remain convinced that the 4% rule would be really bad for the regular Hugo long lists (unless the administrators could be convinced to routinely ignore it), and I think it’s a good thing we narrowly defeated it. However, in retrospect, I think that we could have achieved a viable compromise if someone had proposed a “lesser change” amendment to restrict the rule to the Retro-Hugos only, if that was the main concern of the supporters. If it really does make a difference to the workload of the administrators, and if there are enough future Retro-Hugos to matter, perhaps such a more restrictive version could be proposed in the future.

  30. Pingback: Dublin 2019 WSFS Business Meeting Day 2 | File 770

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.