2016 Hugo Administrator Answers Question About EPH Testing

Mad Genius Club columnist Dave Freer received an answer to an accusatory letter he wrote to MidAmeriCon II’s Hugo Awards administrators about voting data shared for purposes of testing the “E Pluribus Hugo” proposal.

He published Dave McCarty’s reply as part of his column today, interrupting every single sentence McCarty wrote with a boldfaced rebuttal three times as long. I became curious what McCarty’s letter would have looked like if it had been properly quoted. This is the text, minus Freer’s fisking.

Mr Freer

With the passage of the EPH proposal at the Sasquan business meeting, the members of WSFS began the process of substantially altering the method for selecting Hugo finalists.

The method being proposed is novel.

There is no prior example to let the members of WSFS understand completely how this method might operate when used in the Hugo awards.

As such, it was imperative that EPH be tested with meaningful data and those results reported to the next business meeting at MidAmeriCon II.

To accomplish this, Sasquan made its Hugo nominating data available to MidAmeriCon II for testing. MidAmeriCon II provided access to two researchers (Bruce Schneier and Jameson Quinn) and worked collaboratively with them to test the counting method.

The data was anonymized prior to it being shared for testing. A random key was assigned to the voter, and this key was reset to a new value in each category, so a voter who participated in six categories was given six unique keys. The data was also normalized to standardize choices and further anonymize the ballot data.

The researchers were given the data under an NDA, and while some analysis results were released prematurely, the NDA was not broken, as no voter data was shared by them.

The researchers’ technical paper is under academic review.

A more complete report to the business meeting will be made public later this spring or early summer.

The particulars of handling the privacy and secrecy of the Hugo nominators and voters is a responsibility handed each year to the Hugo subcommittee of each year’s Worldcon. Additionally that team is charged with protecting the interest and the integrity of the Hugos. We take these responsibilities very seriously.

I can assure you that neither we, nor the administrators of the Sasquan Hugos, contacted the employees of Tor, or any other publisher, prior to the release of the Sasquan final ballot, except to the extent that we will attempt to contact all nominees, and such contacts may be through their publisher, in the cases where the individual nominees have no other published contact information. This may allow a publisher to know some works that have qualified for the final ballot, but only the works published by that house, and only to the extent that we do not have any other contact information.

MidAmeriCon II welcomes all fans, as exemplified in our code of conduct, available at http://midamericon2.org/policies/code-of-conduct/. Each Worldcon’s convention committee and staff is unique to that Worldcon, and experiences at Sasquan should not be assumed to be relevant to MidAmeriCon II.

I understand your inclination to publish my response publicly, I would only ask that you publish the response in full if you do.

Dave McCarty – Hugo Administrator

MidAmeriCon II


Discover more from File 770

Subscribe to get the latest posts to your email.

54 thoughts on “2016 Hugo Administrator Answers Question About EPH Testing

  1. Wow. With the exception of the text of McCarty’s letter, I don’t think that there’s an entire paragraph in that rant of Freer’s that doesn’t contain an error or falsehood.

    I don’t think there’s any way to reason with someone who is that intent on being wrong.

    Thanks for posting McCarty’s letter in unadulterated form, Mike.

  2. Well, it is Freer; my experience with his writing about Puppy issues has not led me to expect much better.

    McCarty’s letter is well reasoned, thoughtful and informative and I appreciate the opportunity to read it. Thank you for reassembling the grains of wheat from among the chaff, Mike.

  3. I went and read Freer’s drivel. I don’t think “conspiranoid ranting unfettered by reality” even comes close to adequately describing how ridiculous his “fisking” is.

  4. Saying that voters for a voting system has no business knowing how the system works? Dave Freer is an idiot. And quite possibly also a lunatic.

  5. Normally I think that an active imagination makes life more pleasant. But this guy has dreamed up a rather icky reality for himself to live in.

  6. Thanks for posting McCarty’s letter here, so I can read it. I did click on the link to see it in context…. but I never even got to the letter. I lacked the stamina to wade through Dave Freer’s lengthy post (and I agree with Aaron that “conspiranoid ranting unfettered by reality” doesn’t even come close to describing Freer’s essay), and I quit before McCarty entered the text.

    Mr. McCarty’s letter is cogent, clear, and reasonable. Thus, it was destined to be badly received by Puppies.

  7. On the one hand, his fact-free take on, well, everything WSFS-related, supports his “I don’t care at all about the Hugos” stance, but on the other hand, why does he publish long rants about the Hugos?

  8. That’s an interesting snippet about how they handled the anonymisation.

    Is it me or does Freer’s fisk-interruption start with him getting really confused about who the WSFS are? He appears to be saying that the WSFS business meeting is separate from WSFS, and to be unaware that the Hugo admins are acting at the request of the business meeting.
    (It’s possible that I’m just misreading him, I suppose, it’s terribly incoherent at times)

  9. Again I can’t help noticing that Dave’s greatest failing is to use some sort of critical-thinking mirror and apply his own advice to the puppies and Mad Genius. He is keen to lecture aspiring authors, Worldcon, traditional publishing and everybody about how important brands are. Yet once again, he is putting extraordinary effort into making the MGC & Sad Puppy brand look like InfoWars-level-wingnuttery. Seriously, who does he think that will win over to his cause?

  10. No, Mark-kitteh, I drew the same conclusions about Freer’s confusion.
    …and of course one wants to test a proposed system *before* it is voted into law; and no, it isn’t a trivial math matter but something that must be run on actual datasets. You can generate fake ones, but historical ones are better.

  11. Indeed, I hope that the vote on whether or not to ratify EPH comes after the Hugos are announced, so we’ll have another dataset to look at. (Since the admins can reasonably guess they’ll be asked for this, I trust they’ll run EPH on this years’ data as well.)

  12. Mike, that Mr. Dave Freer thinks that we who read File 770 in its web incarnation let you do our thinking for us surprises me greatly, given the number of times I have inadvertently put my foot in my hands and angered you to the point at which you refused to communicate with me until you had enough distance to forgive my stumbling fingers.

    He has, if this is an indication of his general belief, a mistaken view of many — perhaps most — of us who are active contributors to the mass conversation here. I know I cannot speak certainly for anyone but myself, but I have trouble grokking that this raucous crowd would contain a groupthink.

  13. Is it me or does Freer’s fisk-interruption start with him getting really confused about who the WSFS are? He appears to be saying that the WSFS business meeting is separate from WSFS, and to be unaware that the Hugo admins are acting at the request of the business meeting.

    I keep having to take breaks, because, wow, there should be something like a Kelvin scale for howlers at the moon, with James May at one end and semi-reasonable gun nuts at the other, which puts Dave Freer somewhere past the mid-range on the May side. But, yeah, Freer seems to think that the WSFS business meeting is somehow a thing that happens without WSFS members, who then must buck up and accept any rulings. That’s a pretty fundamental misunderstanding of how it works.

    Back to the fray, but when I come back I will expect File 770 to tell me how to vote. I think. That’s where I bailed last, because my head hurt.

  14. Given the enormous number of awards which now exist for sf, fantasy, horror, comics, etc., I admit that I have difficulty understanding the fixation that the Puppies (who I do not wish to kick, and I hope I am not perceived as kicking) here that the Hugos are biased. There is no reason nor compulsion which prevents another award from being created, one which from their points of view would be conducted and/or juried more fairly than their Hugo perception.

    I admit that I am somewhat distanced from the idea by the fact that I don’t write fiction which would be in contention for any award, so I have no experience of any sales boost which might be prompted by winning any kind of award — it isn’t an issue for me, I’m not writing for my living, so whether any such with any prestige would affect me materially isn’t a factor for me.

    I can understand their anger if as working writers they do perceive bias, regardless of anybody else’s opinions, but am at a loss as to how to suggest a solution which they would find acceptable short of one or more of them recusing themselves from the balloting in order to volunteer to be part of any given worldcon bid or committee to administrate the awards to make certain that the stated concerns of what is bias don’t take place.

  15. For all you non-USians, if you want to know why some of us can seem a little punchy when right-wingers rant, it’s because this year has been a year of millions of Freers. Literal millions. And the wind continues to rise.

  16. David K. M. Klaus,
    The attraction is that the Hugo nomination system is easily gamed *and* the awards have prestige. Other awards are either juried (so you can’t influence them unless you happen to be one of the judges), or have a much higher membership hurdle (you need to be a published writer & dues paying member of SFWA to nominate & vote on the Nebula awards).

    I mean, they could try to game the popular & free-to-vote awards like Goodreads, but that doesn’t have anywhere near the prestige the Hugos have. They* could start their own awards (and many have made that suggestion), but they would then have to put the requisite amount of effort required, over many years, and even then there is no guarantee that their new awards will ever achieve the prominence of the Hugo awards. In ecological terms, it’s parasite behaviour.

    *By “they”, I mean literally anyone. We see new awards getting established on a regular basis. The hard part is keeping the new awards going & doing well long enough to gain prestige.

  17. Why did I read that whole thing, and the comments? Freer’s wide-eyed innocence regarding the WSFS business meetings actually giving a vote to any WSFS member who cares/is able to attend, his multiple misunderstandings of the weak correlation bit that Quinn released, the comments about Schneier’s credibility, the rush to report Quinn to his school, the general furious inanity… is stunning. You’d have to fisk this guy clause by clause to unpack that essay.

    Though, at least, I don’t think he ever consciously lies. He just doesn’t understand anything he’s talking about.

    ETA The bit about reporting Quinn makes me cringe a little. As with CUL reporting Aaron to his employer, it’s another kindergarten tantrum going public that makes it more likely that people will associate fans with whiny man-boys and woman-girls.

    Wow, “woman-girls” really doesn’t work.

  18. The oft-mentioned “sales boost” of winning a Hugo Award (or getting nominated for one) has a LOT to do with where your career is at the time of the win (or the nomination). If you are a little-known or new author making four-figure advances for your novels with a small press (or still haven’t sold a novel), a Hugo Award is tremendous publicity and can be a terrific boost to your visibility and, indeed, your earnings. (It can also be a non-event if you don’t capitalize on it fairly quickly, because there will soon be another round of nominees and winners, and another, and another.)

    If you’re already well-established, the effect on your earnings will be somewhere on the spectrum from moderate to minimal to non-existent. For example, Neil Gaiman and John Scalzi won Hugos for novels which had been on the hardcover New York Times bestseller list. That’s very much the high-end of commercial/fiscal success and visibility in publishing, and the amount of effort and money invested by big publishing corporations in promoting those authors makes the Hugo win financially irrelevant, though professionally/personally rewarding for the author.

  19. Soon Lee: Of course they did set out to game Goodreads… At least the Rabids did.

  20. @ David K.M. Klaus

    One may not wish to kick the Puppies, but they rush between your feet and then when you’re trying to avoid pitching head-first down the stairs they wail that you have kicked them.

    Faced with that kind of culture of victimhood all you can do is to make sure to use the hand rail.

  21. Sounds to me like Freer thinks WSFS members are con administrators not realizing we are all members. Several of the strange statements flow from that.

  22. @Camestros

    Seriously, who does he think that will win over to his cause?

    I think to Freer, this isn’t the aim. If you see all his hypotheses, they’re based on the underlying assumption that the majority of people already share in his preferences – whether it be conservatism or preferences in writing reading. Given that, (in his worldview) there is absolutely no need for him to convince anyone – they’re already on his side.

    Which, btw, he’s so insistent that it should be the “Puppy-Kickers” who change their ways – in the same worldview, they are the minority, and exist only due to the kindness/ could-not-care-lessness of the Pups.

  23. @Kathodus: Wow, “woman-girls” really doesn’t work.

    Right? Nice attempt, though.

    However one genders it (or doesn’t), they do seem to resemble a bunch of small children running around in big-people’s physical semblances, don’t they? It’s like the first third of the movie BIG, multiplied.

  24. Mike, I commend you for trying to inject some reason and sanity into that slimepit.

    I hope someone tells Jameson Quinn that Freer is attempting to lodge an ethics complaint against him.

  25. The other fun bit is watching the Pups proclaim that Torgersen’s self-proclaimed slate wasn’t a slate and there was no lockstep voting, with a side order that anyone disagreeing doesn’t understand basic math and has a below average IQ.

    Given that it’s now been shown twice that a call for Hugo suggestions by the Pups results in something with as long a tail and lack of five or so runaway favorites by substantial margins in categories, that much of what was on SP3’s slate had zero buzz from *any* source beforehand (hell, remember that “Wisdom From My Internet” resulted from Torgersen asking Williamson if the latter had anything eligible that he could put on his slate; it didn’t even have buzz with the slatemaker), and that the final ballot had such a huge and mathematically ridiculously unlikely correlation with that one slate ballot, well, it really doesn’t take someone with my well above average IQ and effective math minor in college including graduate level math courses to understand that, yes, it was a slate and there was block voting out the wazoo on it.

  26. Cheryl S. on March 21, 2016 at 4:19 pm said:

    But, yeah, Freer seems to think that the WSFS business meeting is somehow a thing that happens without WSFS members, who then must buck up and accept any rulings. That’s a pretty fundamental misunderstanding of how it works.

    To be as generous as possible, there are as far as I can tell a lot of people who assume that’s how it works, because the entire concept of an open participatory democracy where every single member of the organization who shows up is able to participate as an equal member of the society is completely alien to most people. Most people think that, at best, things are run by some sort of Elite, whether it’s an elected elite or some Shadowy Group of Secret Masters. So of course there must be a Board of Directors of WSFS, Inc., because no sane person would ever actually create an organization where all of the members can show up, debate, and vote.

    If your mind cannot process the concept of WSFS = the members of Worldcon and if you assume that “WSFS” is some group of people who makes decisions separately from the members of Worldcon who show up and participate, then many other things follow logically.

    This is an ongoing battle. I’ve had Worldcon members ask me where the “observer’s gallery” for the “ordinary members” (i.e. the ones who weren’t elected to the Business Meeting could watch the proceedings. Nobody ever told them there was some elected/appointed “other” body; they just assumed that it must be that way because that’s how everything works.

  27. @Nicole J. LeBoeuf-Little

    @Kathodus: Wow, “woman-girls” really doesn’t work.

    Right? Nice attempt, though.

    Heh. Thanks. I left it there because it was so dumb-sounding.

  28. Okay, reading that was actively damaging to my intellect. I’m going to complain to Freer’s ethics committee for the loss of brain cells I just suffered.

  29. Incidentally, Freer has now updated his post with the unfisked original, so you can see where the paragraphs were meant to be etc.

    Freer obviously has a snark at Mike about it as well, carefully avoiding the fact that a quote with a link allows anyone to see the original, in contrast to Freer not publishing the unaltered original to start with.

  30. Mark: Well there’s a surprise.

    You know there are several main ways of generating blog content from other people’s material. The most common is simply to list titles and links. The next most common is to do that and add a very brief description. At the other end of the spectrum is to take the linked material and recast the information with a few value-adding comments and present it as a fresh article.

    What I do at File 770 is in between. My “value adding” consists of selecting an excerpt from the linked post. I want people to know why they might be interested in clicking through. So I’m looking for “hooky” kinds of quotes that make a person want to read on. (Though in rare instances I choose something a little less volatile than I might, simply because there’s certain kinds of things I’d just as soon not read on the front page of my own blog….)

    It’s also worth bearing in mind that most people get pissed off when you gank entire posts from their sites. I feel the same way. And in any case, I have no interest in doing that.

    However, Freer is already pretending that’s exactly what I have “promised” to do with his posts now. Which isn’t going to be happening.

  31. That’s a great letter. I’m pleased that the Hugo Administrator addresses queries like this, I’m afraid I don’t read MGC so thanks for posting this Mike. I look forward to hearing more about EPH and whether it is viable.

  32. @Mike Glyer

    What I do at File 770 is in between. My “value adding” consists of selecting an excerpt from the linked post. I want people to know why they might be interested in clicking through. So I’m looking for “hooky” kind of quotes, the kind that make a person want to read on.

    This has been bothering me for a while. You are studiously ethical in your excerpting and linking. You don’t shy away from editorializing, but you assume your readers will read the the articles you link to if they interest them. This is patently obvious, I believe, to anyone who isn’t of an authoritarian mindset. It’s one of those infuriating Puppy things, the constant meowing about your “out of context” quotes.

  33. Mike,

    A fair number of sites that I read do an “interesting links” post, and they tend towards the “lots of links, little context” style. I certainly find interesting things in that sort of post, but it’s a bit needle in a haystack. There’s no one right way to do roundups, but I particularly enjoy your added-value style here.
    In terms of Freer’s objections, I fail to see how a quote+link automatically destroys context – anyone who wishes to follow the link can do so. Obviously it’s possible to quote out of context, or concentrate on the worst-sounding part of an article, but I’ve yet to see a clear explanation of how any of your quotes are unfair, beyond just “quoting us is bad.” You certainly don’t just pick out the worst bits; I’m frequently able to find much more outrageous passages than what you quote.

  34. @Mike Glyer

    What I do at File 770 is in between. My “value adding” consists of selecting an excerpt from the linked post. I want people to know why they might be interested in clicking through. So I’m looking for “hooky” kinds of quotes that make a person want to read on. (Though in rare instances I choose something a little less volatile than I might, simply because there’s certain kinds of things I’d just as soon not read on the front page of my own blog….)

    I really like your quotes. In many cases they encourage me to go read the entire post on sites I wouldn’t normally read. I think you do a pretty fair job in selecting what to quote. I find the more intelligent discussions here happen when filers go read the linked post. There are one or two rabid puppy sites I won’t go to as I’m unwilling to expose my IP address to them. I know my conversation on those topics suffers. Good job is what I’m saying.

  35. @Aaron

    Was he ever on the rails?

    Heh. No, I guess not.

    I watched Snowpiercer the other night, and I have this vision of the scene at the end, where the avalanche (Hugo voters) cascades down the mountain and sweeps the train (Puppies) right off the track and down into the ravine.

  36. I think what Mike Glyer does here is the most appropriate approach. Linking plus a selective quote allows the reader to see if they want to go to the original site to read the rest of the article. The parts quoted are meant to pique interest, and they generally do. In general, I certainly don’t think the quotes chosen misrepresent the original article, and more often than not, the selected quote has been the most cogent part of the article.

    The practice of quoting a whole article is generally frowned upon: not only may it infringe copyright, it also reduces the number of visits the original site receives, which is arguably dickish. Obviously, for some people, there are sites they wish to quote without increasing traffic to them, hence the use of donotlink.

  37. I would like to applaud the Administrators for not tossing Dave Freer’s letter in the bin. The moment he started to be rude to me(or at least excessively snide), I would be hitting the delete button and moving on.

  38. I finally gave in and tried to read the Freer post. My eyes kept glazing over. I realized I was scanning looking for anything which made sense. I did make it about 1/3rd of the way through his letter fisking before I realized he was just shouting the same conspiracy over and over again and quit.

    As to why any of Freer or his 20,000 fans/readers/friends should give money to and participate in Worldcon, if they feel like him, they shouldn’t. It’s very simple. Don’t trust the people taking your money, they aren’t the government or utilities, don’t give them your money. Find something else to be involved in. Problem solved. With 20,000 fans you should have no problem creating a new and relevant award and grow its prestige. Go for it.

  39. Problem solved. With 20,000 fans you should have no problem creating a new and relevant award and grow its prestige.

    One would think that if their ardent fans were are numerous as they claim, they’d be able to create a new and better “Worldcon” – maybe call it something like “Universecon”. If they really wanted to show all of the evil SJWs they hate so much just how great they are, nothing would do that more effectively than something like that.

  40. Tasha Turner on March 22, 2016 at 8:23 pm said:
    As to why any of Freer or his 20,000 fans/readers/friends should give money to and participate in Worldcon, if they feel like him, they shouldn’t. It’s very simple. Don’t trust the people taking your money, they aren’t the government or utilities, don’t give them your money. Find something else to be involved in. Problem solved. With 20,000 fans you should have no problem creating a new and relevant award and grow its prestige. Go for it.

    Isn’t LIBERTYCON pretty much the home town for the Puppy types? Conservative, libertarian, gun friendly, and so on? Why don’t they just sponsor an award for that con? It would seem to be a perfect fit. No need to start from scratch at all. Just build on what is there.

  41. TechGrrl1972: Isn’t LIBERTYCON pretty much the home town for the Puppy types? Conservative, libertarian, gun friendly, and so on? Why don’t they just sponsor an award for that con? It would seem to be a perfect fit. No need to start from scratch at all. Just build on what is there.

    Several people have already suggested that to the Puppies. But they can’t do that, doncha see?
    1) They would have to put in the work to set everything up.
    2) They would have to put in the work — a LOT of work, over a period of numerous years — to grow participation in the award and raise awareness of it.
    3) The award would have to gain the kind of cachet that the Hugo Award currently has — which would take many, many years — and still might not be attainable.

    Really, it’s just so much easier to try to come in and take over another group’s award, isn’t it?

  42. @TechGrrl1972 – In addition to what JJ has said, IIRC LibertyCon is by design a reallllly small con, so it’s self defeating for them to set up something there, as WorldCon remains a numerically superior con.

    Additionally, the Pups exist in perpetual terror that if they come up with an award, evil EssJayDubyas will take over it. So they either try to come up with convoluted gatekeeping (witness Jay Maynards attempt last year) or hostile takeovers.

Comments are closed.