Delusions of Masses

James at Big Dumb Object wasn’t impressed by the record-setting number of Hugo nominating ballots cast in 2012.

…a thousand ballots for what is supposed to be the premier Science Fiction awards really doesn’t seem like very much to me.

Why get excited that voter participation has increased by more than one-third since 2009 when the total is still insignificant?

James would like to see current information technology used to measure story popularity:

It feels a bit stone-age when you have music sites like Hype Machine or We Are Hunted compiling a real time barometer of music tastes. We should be able to track which stories were the most read, we should be able to analyse every thought on those stories, we should be able to craft that into a coarse grained voting system, we should be able to extrude the real wisdom of the crowds.

If it’s possible, why not? Everyone would like to know.

I have nothing to add to the main point of James’ post — it was his passing thought about the number of Hugo voters that set me to thinking. I wondered if James might be wrong about a thousand Hugo voters being an insignificant fraction of the literary marketplace.

Of course 1,000 is a less-than-trivial number in proportion to the audiences for sf on TV or in film, seen by hundreds of thousands, possibly millions, of people. No one disputes their audiences are magnitudes greater than the corps of voters which picks the winners of the Hugo Award’s Best Dramatic Long- and Short-Form categories.

But that’s not what James wants to measure. He’s interested in the response to stories – text, in print or digital form.

Checking online I saw a lot of inconclusive discussion about the average print run of a novel (and that leaves aside the even more impenetrable universe of e-book statistics.) Pros blogging about the writing business typically said print books had press runs of 5,000-10,000 copies (unless they were expected to be bestsellers).

As for prozines, when I last checked, the circulation of Analog was less than 30,000.

I can’t say how many tens of thousands of different people make up the marketplace for written sf, but the figures I’m seeing suggest to me that 1,000 Hugo voters is a statistically meaningful sample of the audience for literary sf.

9 thoughts on “Delusions of Masses

  1. It’s not statistically meaningful, because the sample is self-selected rather than being selected at random, which destroys the statistical validity of almost any sample. If you could run through Analog’s mailing list, pick 1 in 30 of them at random, and get only those people to vote in the Hugos, then 1,000 would be a plenty big enough sample.

  2. The problem faced by “James” and everyone else who wants more participation in awards-voting is that it is the people likely to be nominated who show the most interest in the process, and the level of committment drops from there. Voting is an entirely altruistic act, and expecting the average Worldcon member to feel some degree of investmet in it is probably silly,

    Only 47 ballots were cast in this year’s FAAn awards, but the winners should still feel very good, because the people who voted all had a high dgree of knowledge about fanzines. I respect their opinion almost infinitely more than that of the average Hugo voter, who only needs to know how to buy a Worldcon membership.

  3. @Mike: You’re right, I have gotten myself tangled up by using jargon which expresses a specific meaning that my facts do not support. Haven’t decided what to change it to yet. Other than to restate there is a finite total number of sf story buyers, perhaps about the same number as the population of a small city. And in a small city 1000 voters is not trivial.

  4. @Andy: Two of the five fanzines receiving the most votes in the 2011 FAAn Awards were also 2011 Hugo nominees and a third was nominated for the Hugo twice before. I don’t see how there could be such a convergence if the Hugo nominators were as ill-informed as you think.

    Just as not all Corflu members vote in the FAAns, it’s clear from the voting statistics that Hugo voters generally nominate in categories where they feel a higher level of confidence in their knowledge about what deserves an award — and skip the rest.

    Last year, 66% of those casting a Hugo nominating ballot did not make an entry in the Best Fanzine category. If we knew the names of the other 34% I’m sure we’d each have an opinion whether that person is well-informed about the fanzine field. But what do we really know? There are about two dozen prolific LoC-writers whose knowledge we can be absolutely sure of because they had to read all these zines to write their comments. Beyond that we’d be guessing.

    There’s another thing to consider about the knowledge of Hugo voters. Unlike the FAAns, which go to the winners of a single-stage vote, the Hugos use a nominating vote to select the finalists. And for the past several years there has been available a Hugo Voter Packet with samples of the nominees. There is good reason to expect that the people who vote in the Best Fanzine category have familiarized themselves with the finalists.

  5. You support your point magnificently, Mike, but my idea was more to praise the FAAn award voter than to denigrate the Hugo voter. The Hugo voters should not be presumed ignorant until they vote for a novel by Lois McMaster Bujold.

  6. “Statistically significant” must have changed meanings since I took statistics courses in college. Self-selecting doesn’t change whether a statistic is “significant”, just whether it’s random or necessarily a good predictor of what the result would be if total “class” could be gotten to give responses. 10 (out of 30,000) nominators or voters is not going to give you a statistically significant result, whether or not they’re randomly selected. 1,000 will. Being self-selecting is a factor on reliability, just as if you only got male voters or voters over the age of 60. (And, of course, all surveys and votes are affected to some extent by “self-selection” because no matter if they seek you out or you seek them out, you only get responses from “the sort of people willing to answer or fill out surveys”.)

  7. Mike,

    the largest number I’ve seen quoted is for a mailing list, sold by a marketing firm, that is described as “self-selected science fiction and fantasy readers”. The list has well over 6 million names in it.

    I do not know the sources they put the list together from, though I will assume, barring evidence contrary, that they were good sources since this is a firm that charges a pretty penny for their lists and has to maintain some level of credibility with their product to remain in business.

    So, taking that number as ‘good’ – even if most respondents to whatever questionnaire was “sometimes read science fiction or fantasy”, I still think it is a pretty hefty number.

  8. Well then,I’m not going to be able to make 1,000 sound like very much compared to that number. Wonder what they read? One percent of that figure would still be twice as many as subscribe to Analog.

  9. @Andy: I’m reminded of your well-written piece in the 1988 Fanthology which includes this dialog —

    “Peachy, what the hell do you think it means that a man can just walk into a mud hut out in the very back of be­yond and find better fanzines than get nomi­nated for the Hugo awards these days?”

    “That the people nominate the Hugos are a small group, and the people that are on the mailing lists for good fanzines are a small group, but they’re not the same small group.”

    Andrew P. Hooper, “The Fan Who Would Be King,” Fanthology ’88, SF3.

Comments are closed.