Hitch in Sasquan Nominating Data Turnover

Plans to make transcribed data from the 2015 Hugo nominating ballots available upon request have been put on hold.

E Pluribus Hugo advocates, who want to use the data to demonstrate the EPH vote tallying method is effective at coping with slates, got the Sasquan business meeting to pass a non-binding resolution (item B.2.3) asking for the release of anonymized raw nominating data from the 2015 Hugo Awards.

When the resolution passed, Sasquan Vice-Chair Glenn Glazer announced Sasquan would comply with the request. The intent was to provide equal access to the data, and those interested in receiving a copy were invited to e-mail the committee.

However, Glazer confirms he recently e-mailed the following update to a person who requested the data, as reported by Vox Day:

Back at Sasquan, the BM passed a non-binding resolution to request that Sasquan provide anonymized nomination data from the 2015 Hugo Awards.  I stood before the BM and said, as its official representative, that we would comply with such requests.  However, new information has come in which has caused us to reverse that decision.  Specifically, upon review, the administration team believes it may not be possible to anonymize the nominating data sufficiently to allow for a public release.  We are investigating alternatives.

Thank you for your patience in this matter.  While we truly wish to comply with the resolution and fundamentally believe in transparent processes, we must hold the privacy of our members paramount and I hope that you understand this set of priorities.

Best, Glenn Glazer

Vice-Chair, Business and Finance

Sasquan, the 73rd World Science Fiction Convention

And Hugo Administrator John Lorentz added information in this follow-up e-mail:

What wasn’t included in Glenn’s statement is that this year’s Hugo system administrators are working with a committee composed of proponents of EPH, so that proposal can be tested without any privacy violations that might occur by releasing the data with no controls.

As Hugo administrators, we have always assure members that their votes are private and secret, and we don’t want to do something that might change that. That is our primary responsibility.

John Lorentz

Sasquan Hugo Administrator

On September 1, in an exchange between several commenters, Lorentz remarked the difficulties of anonymizing voter data, here at File 770:

[Commenter] “With the Hugo data, the only identifying info is the membership number. Remove that, and the ballot has been anonymized.”

[Brian C] No, it’s not nearly that simple.

You also need to eliminate any nominations that are unique to one or a handful of people, as otherwise those nominations could be used to identify people. But then those ballots aren’t actually representative for the purpose of testing the algorithm. So you need to actually replace those with other nominations, that happen not to perturb the algorithm in any way.

[John Lorentz]And that is the problem that our Hugo system admin folks have been running into. When one of them generated a draft of anonymized nominating data, it didn’t take the other very long to determine who some of the voters were, simply from the voting patterns.

Vox Day terms the latest development a “scandal.” Peter Grant was equally prompt to accuse Sasquan of having something to hide in “What, precisely, is going on with the Hugo Awards data?”

Folks, back in the 1980’s I was a Systems Engineer at IBM.  I’ve had well over a decade in the commercial information technology and computer systems business, in positions ranging from Operator to Project Manager, from Programmer to End-User Computing Analyst to a directorship in a small IT company.  Speaking from that background, let me assure you:  I can ‘anonymize’ almost any data set in a couple of hours, no matter how complicated it may be.  To allege that ‘it may not be possible to anonymize the nominating data sufficiently to allow for a public release’ is complete and utter BULL.  Period.  End of story.

However, one of Grant’s commenters pointed out: “Anonymizing data is harder than you think, if your goal is to actually make it truly anonymous. See what happened when AOL tried to anonymize search results, or when Netflix tried to anonymize movie recommendations.” And he cited a 2009 ArsTechnica article, adding “and metadata analysis hasn’t exactly gotten worse since then.”

The article says —

Examples of the anonymization failures aren’t hard to find.

When AOL researchers released a massive dataset of search queries, they first “anonymized” the data by scrubbing user IDs and IP addresses. When Netflix made a huge database of movie recommendations available for study, it spent time doing the same thing. Despite scrubbing the obviously identifiable information from the data, computer scientists were able to identify individual users in both datasets. (The Netflix team then moved on to Twitter users.)…

The Netflix case illustrates another principle, which is that the data itself might seem anonymous, but when paired with other existing data, reidentification becomes possible. A pair of computer scientists famously proved this point by combing movie recommendations found on the Internet Movie Database with the Netflix data, and they learned that people could quite easily be picked from the Netflix data.

EPH backers want to use the data to demonstrate their voting system. In comparison, a commenter at Vox Popoli said he wants to analyze the data to learn —

  1. How many slates there were in competition
  2. How good party discipline was for the various slates
  3. How many voted mixed slates of sad/rabid, TOR/SJW, etc.
  4. How the 4/6 and EPH proposals would have affected the outcome of the competing slates

Update 09/08/2015: Corrected the attribution of Brian C’s comment.

770 thoughts on “Hitch in Sasquan Nominating Data Turnover

  1. And now I find if I can switch back without incurring the wrath of WordPress.

    Edit: Komputer says Da.

  2. @Meredith

    “Filers” as a nickname for the commenters here — I like it!

    “Turncoat” is, to me, an interesting test case. Vivienne R and Brian Z have both said they like it, and I have no reason to believe they’re lying to me. Yet, I find it utterly incomprehensible that anybody could have liked that story. I thought it failed in every way a story can possibly fail. I was astonished that anybody, even VD, thought it was worth publishing.

    So maybe if your tastes are calibrated such that you were able to enjoy that story, I could almost see where you might think the Hugos have been going downhill? Okay, now I’m not sure exactly where I was going with this.

    On feminism — I can deal with “feminism neutral” or “feminism that I don’t fully agree with” but actual, consciously-chosen anti-feminism is a complete non-starter with me. I won’t read your fiction, I won’t read your blog posts, I won’t hang out with you at parties. I suppose puppy types might consider that exactly the kind of overly political box-checking they’re complaining about — but it’s important to remember that it’s a NEGATIVE checkbox. I have never loved any fiction for message alone, but I have definitely hate-hate-hated fiction for its message.

  3. Brian Z — thank you for that clarification. I couldn’t remember if you indicated how you voted.

    My impression — based on the commenters here — is that they failed to inspire much enthusiasm with their picks even among people who were otherwise puppy-sympathetic.

  4. This is one of my obstacles to my plan to get to Worldcon; the travelling home before the week-in-bed bit is likely to prove very challenging.

    This. I have to schedule to arrive ahead to have a few crash days. To stay after for a few crash days so I can travel home where I’ll then have a major crash.

    My trips home to visit my family are timed around cons I want to attend and my family is slowly adjusting to my crash/totally bedridden days around arrival and after they’ve talked me into doing things with them or attending a con.

  5. Comrade JJ

    … snowmarx …

    No no, tovarishch, please call me snowkarl. I aim for a certain level of informality after all….

  6. @McJulie

    Full credit for “Filers” ought to go to Jim Henley, I think. 🙂 (I’ve largely adopted it because its a lot quicker to type than “the File770 commentariat”.)

    @Tasha Turner

    The problem is that building in the rest I’d actually need puts it way, way, WAY out of my budget, which would be very stretched with the most basic stuff (and I can’t do the sleep-on-someone’s-floor thing, either, so that’s a cost-saving measure struck off). I haven’t figured out a solution yet. I’m trying to, because I’d really like to go.

  7. @Richard Gadsden Apologies. I missed that further up. Nice to see you too. Sorry that I wrecked your voting discussion thread 🙂

    @Mark

    How do you actually see it?

    Seriously, that would need three essay-length blogposts… Which I’m planning to write. Fair to say, I don’t think there’s an organised extremist anti-kyriarchy group controlling the Hugo nominations, and I don’t think most (if not all) of the Puppies are proactively trying to keep women and minorities out of SF. Or to make all stories about (in the words of Wired):

    Would sci-fi focus, as it has for much of its history, largely on brave white male engineers with ray guns fighting either a) hideous aliens or b) hideous governments who don’t want them to mine asteroids in space?

    I mean, seriously? Has science fiction EVER been all about white male engineers fighting monsters/asteroid mining?

    The next question is ‘what is it about?’ And I think it’s – simplified enormously – a cultural battle between Scott Alexander’s Blue Tribe and the Red Tribe and emerging Grey Tribe. This is why it’s hard for the various groups to understand each other – it’s hard to see your own culture. And that’s why I was so confused.

    There’s a separate issue that the Blue Tribe ubiquity of “personal is political” means cultural debates are often nastier than they should be . Debate can look like an attack on personal and social identity – not simply a disagreement with the views someone holds [yep, quoting one of the ex-RCP crowd here. Can’t comment on the Tatchell incident].

    Also underlying the Hugo flare-up is – I think – some combination of:

    One. The mainstreaming of geek culture without a consideration of how this affects a prestigious genre-wide prize;
    Two. Problems with the short fiction market, both the proliferation of stories across many outlets, and the decline of commercial short story writing (speculative – need sources);
    Three. The transformation of publishing, including the rise of Indie and self-publishing publishing, the growth of eBooks and a bunfight over prestige between authors in different camps (some evidence of generic hostility in this article).

    This is compounded by the politics (as discussed). Also the near-collapse of advertising-funded journalism a few years back, which means that click bait rabble-rousing by mainstream media (on both sides) is a real problem.

    @JJ

    I realised why I look libertarian on File770 (finally). We’re talking about political-social issues – I test very very political-social libertarian on the Political Compass, but also economically left-wing. Right-wing means (to me) either social conservative or economic libertarian.

    @Mark

    I was particularly struck by the unneeded tangent into defending Hitler’s military record, and rather thought it was in there for standard-issue “making heads explode” purposes. I considered it a strong argument for VD deserving to come in a distant last in Editor.

    Glancing through that again, I don’t think it was there to provoke a head explosion. I think the author was talking to an audience assumed to be familiar with WWII military history or – at least – to be sufficiently interested in the topic to have views on German decision-making.

    After all, if you’re – say – talking about the D-Day landings and you’re a military strategist, you do need to know what the Germans were doing. And why they did that and on whose orders. As the German army was, at the time, taking orders from the Nazi Party, discussing this topic requires reference to senior Nazis.

    I appreciate this is controversial. For example, Gitta Sereny received criticism as a Nazi sympathiser for her biography of Albert Speer.

  8. @Vivienne — Whoever the writer thought he was writing for, that work appeared in an anthology at least intended to be aimed at a general sf-reading audience. The unnecessary Nazi reference is the kind of thing a good editor would have caught.

  9. @VR:

    I think your casting of the Hugo flap in tribal Culture War terms overlooks one key element: before the Puppies, the award had not been part of that fight. Yes, there have been politics in fandom since the beginning, but it’s seldom been real-world politics at work. That the Puppies saw a political stance where there was none, and responded by fighting “back,” does not make this a two-sided political conflict. It is more akin to a mugging or a home invasion than a battle, in that one party was quietly minding its own business when they were attacked without provocation. As long as that element is overlooked, any analysis is doomed.

    One. The mainstreaming of geek culture without a consideration of how this affects a prestigious genre-wide prize;

    Just who is supposed to be doing this “consideration”? The very framing of this point is rooted in a false narrative, that Someone is in control of this cultural trend and has rushed forward with their Agenda without giving proper thought to long-term strategy. There is no big Someone, no Agenda, and no such strategy on the long or short terms. There are only ordinary people, doing the same thing they’ve been doing for decades.

    Until they – we – were attacked. The Puppies concocted a fantasy that we were scheming to oppose them, and responded with a preemptive strike. One might even be tempted to say they expected to be greeted as liberators, only to be surprised when they were seen as an invasion force by people who were minding their own business before the bombs started falling. Even now, they crow about their noble victory against the evil enemy, when anyone not in their bubble can see just how warped that vision is.

  10. @VR

    Lis has expressed what I would have said. I’ll add that of the 6(? 8?) general principles he was explaining in rather general terms, it struck me that this particular point, with this particular example, seemed to get an unusual amount of the word count to deal with a side point. The Marmot enjoys exploding brains, including with Nazis if he can work them in (see Watch on the Rhine for one of his finest hours in this regard), and I very much read it as a digression for his personal amusement that an editor would usually have taken some red ink to.
    Anyway, as I don’t think anyone is seriously defending VDs editing skills, I digress.

    Your other point to me started out being about Entertainment Weekly and the media, so I think we’ve gone off track. Rev Bob has replied eloquently anyway.

    I’d be interested to hear your further thoughts and data on short stories. From my perspective, I fell out of reading shorts some years ago when the market was still paper-based and decaying, but have since returned to them with real pleasure, and it seems to me that there’s a real resurgence going on with online outlets adopting freemium and crowd funding models, although whether authors are making any more money out of them is an open question.

  11. @Vivienne Raper

    I’d still like to know whether, prior to commenting here, your main sources of information were Puppy ones, perhaps plus a couple of not-Puppy posts that the Puppies linked to (with commentary)? You didn’t, for example, follow the round-ups here?

  12. @Mark The short story stuff is a working theory, based on some numbers I’ve seen thrown around for short story -vs- magazine feature payments. There seems to be a lot of interest on File770 and I’m interested, so I’ll go away and research properly.

    @Rev. Bob

    I don’t think there’s a Someone with an Agenda. The Puppies are entryist and I think they know they are. Most of them didn’t expect to be showered with prizes, from what I’ve seen. There do appear to be exceptions… I think those people were a bit naive.

    @Meredith. I read the File770 roundups, Eric Flint and Difficult Run. I came originally from some articles in the Guardian newspaper (in April, I think). Almost first place I went from there was Brad Torgerson’s blog, right around the time he posted the picture of his wife. I have read a number of other blogs (including Philip Sandifer’s long post on fascism, although not (yet) his Vox Day interview).

    I also read the works receiving negative attention, e.g. If You were a Dinosaur… And The Water that Falls. I’m familiar with #racefail and Rossgate.

    The issue for me was that, until I came here, I didn’t analyse any long-term data of women nominated, for example… Or types of fiction. Or nationality of writer. Nor did I have a good chance to chat with fans with a long history of fandom (rather than prominent bloggers). The discussion caused me to do that analysis, and get a better handle on what was going on.

    You’ve also got to remember that I enjoyed Turncoat and didn’t think Redshirts was Hugo worthy. I can see exactly why the Puppies complain that ‘If You Were a Dinosaur’ was questionably fantasy and not terribly interesting ideas-wise. For me, the problem was deciding what the issues were… Not deciding there were issues.

  13. @VR: “I don’t think there’s a Someone with an Agenda.”

    So why are your arguments set up as if there is such a person/group?

    I think it’s – simplified enormously – a cultural battle between Scott Alexander’s Blue Tribe and the Red Tribe and emerging Grey Tribe.

    One. The mainstreaming of geek culture without a consideration of how this affects a prestigious genre-wide prize;

    Those are just two examples. You’re talking about a one-sided assault as if it were a boxing match, and it just ain’t so. As Bill Hicks once said, “A war is when two armies are fighting.”

  14. You’ve also got to remember that I enjoyed Turncoat and didn’t think Redshirts was Hugo worthy.

    I don’t think Foundation’s Edge was Hugo-worthy. Sometimes books you don’t think should win do. This isn’t new. It has always happened.

  15. Mark: The Marmot enjoys exploding brains, including with Nazis if he can work them in (see Watch on the Rhine for one of his finest hours in this regard), and I very much read it as a digression for his personal amusement that an editor would usually have taken some red ink to.

    Out of curiosity, after the Marmot bragged about having written an Afterword for the Baen reissue of Sixth Column, I checked it out from my library. The book has a good Foreword written by William Patterson, Jr (aka Heinlein biographer) giving the background for the story in terms of Heinlein, Campbell, and the state of the world at the time.

    The Marmot Afterword consists of (and of course, I’m simplifying here) 1) this story isn’t really racist because of this and this and this, 2) analysis of the difference in effectiveness of terrorism when it is conducted by individuals vs by a government, and (I kid you not) 3) let’s just admire how effective terrorism is when it includes making sex slaves of the opponent’s women.

    It’s a mystery to me why Weisskopf felt that the Afterword was required, or why she felt it added value to the book (another strike against her in the Best Editor category, in my opinion). It seemed to me to be the Marmot saying “see how outrageous and provocative I can be, and isn’t it impressive”?

  16. @JJ

    I’ve got other Heinlein’s with Patterson introductions, and they’re pretty good. I don’t see the need for afterwords, let alone ones from the Marmot.

    The thing is, I know he’s a perfectly serviceable author (the action scenes in BBDC were well done, for example), but he lacks any sort of internal editor about spilling exaggerated versions of his unpleasant personal opinions into a book, and apparently lacks any sort of external editor prepared to tell him not to either..

  17. It is indeed easy to fail at anonymization. One approach would be to use pseudonyms for all the works. For any work that was listed on the detailed results, one could easily figure out what is what, but for minor works that got only a few nominations you would learn the nominator simply nominated “Candidate 143.” This might do a good job at removing things which would disclose who a nominator is. (For example, people who nominated their own minor works.) The only works that would be identified would be the ones which got enough support to lose the nominator in the crowd. Or one would hope. But somebody actually looking at the data would have to figure if that would work.

Comments are closed.