The day of reckoning is here for E Pluribus Hugo. The change in the way Hugo Awards nominations are counted was passed in 2015 and ratified in 2016 to counter how Sad and Rabid Puppies’ slates dictated most of finalists on the Hugo ballots in those years. It came with a 2022 sunset clause attached, and E Pluribus Hugo must be re-ratified this year in order to remain part of the WSFS Constitution.
It’s the first item of passed-on business in the Chicon 8 Business Meeting Agenda. (If you want to read the rule, you’ll find it quoted on page 30.) Six Hugo ballots (2017-2022) have been produced using EPH, however, if re-ratification looks automatic to you, don’t take that for granted. The arguments used to try and stop it from passing the first time still have adherents and may be heard again when ratification comes up at this year’s Business Meeting. Some of them are –
IT’S HARD TO EXPLAIN. What was the nominating system before EPH? The five eligible nominees with the most votes made the ballot (and in case of a tie including fifth place, all the tied eligible nominees were listed). That’s easy to remember. Easy to explain.
“Vox Day’s bloc voters dictated nearly all the finalists on the ballot” is also much easier to explain than EPH. Ease of explanation is not the only priority in play.
EPH DOESN’T GET RID OF SLATED NOMINEES. That’s right. EPH doesn’t get rid of slated nominees. People initially may have wished for an algorithm that would somehow weed out slate candidates, however, upon sober reflection it had to be admitted these slate voters had paid their memberships and it would be undemocratic to prevent their legal votes from having an effect on the finalists. What EPH was represented to do, and does, is make it more likely some nonslate nominees will also make the final ballot, giving people something to vote for beside No Award if they are opposed to the slated nominees.
EPH DIDN’T DETER SLATING ACTIVITY. Some will argue that neither EPH, nor anything else the business meeting did (like adding “4 and 6”) stopped Vox Day and his confederates from placing things on the ballot. And when their efforts diminished, whatever caused that to happen had nothing to do with the rules changes.
EPH received first passage in 2015. Before it was ratified, Vox Day turned around and ran another slate in 2016 and placed 61 of his picks on the final ballot. It was only because of the passage of EPH that Vox Day changed his strategy with his 2017 slate, making it one item per category (in most of them) — and 16 of his 22 picks had enough support to make the ballot. (In the end, because he didn’t check the eligibility of what was on his list, 5 of those items were kicked off — still leaving 11. Which was a lot less than 61.) The evidence is that EPH is what changed his behavior. What’s more, Vox Day himself said that’s what changed his behavior. [Internet Archive].
UNDER EPH, THE HUGO ADMINISTRATOR’S WORK CAN’T BE CHECKED; IT’S A “MAGIC BOX”. When people make this argument, it seems clear to me they’re visualizing the voting statistics that are published after the winners are announced, and how easy it is to check the spreadsheet arithmetic for the winners. They seem to overlook that under the old system it also was not possible to check the Hugo Administrator’s work in the nominating phase. The public could never review what the Hugo Administrator did to verify voter eligibility, or perfect the data prior to input (correcting all the mismatched ways of expressing a name or title). The public could only see the bottom line result — the totals in each category’s longlist.
If people believe the Hugo Administrator is competent to operate the software that generates the totals for the winners, it is reasonable to ask them to have the same faith in the person’s ability to operate the EPH software to determine the finalists.
EPH REWARDS SINGLE-BULLET VOTING. The contention is that there is a sense in which EPH is antidemocratic because it rewards single-bullet voting.
That there is a reward for the behavior appears to be accurate. Nicholas Whyte told the business meeting after he ran the 2017 Hugos, the first under EPH, that his “conclusion was that EPH made it relatively easier in 2017 for a nominee with a large number of bullet votes to get on the ballot; it made it much more difficult for a slate to get more than one candidate on the ballot.”
Since Vox Day’s 2017 slate only had one entry in each category for which he made a pick, it would have been a candidate for single-bullet voting. But again, EPH is designed to open the ballot to others, not to make it impossible for a slate nominee to become a finalist.
EPH WAS PASSED IN A “PANIC”. I’ve seen an opponent characterize as “panic” the decisions made about EPH at two different years’ business meetings. But really? This was probably the most thoroughly-discussed proposal in the history of the WSFS Constitution. Besides everything said on Making Light where it originated, File 770 posts on the topic received hundreds of comments. For example: “E Pluribus Hugo Tested With Anonymized 2015 Data” 2/8/2016 – 407 comments. “Analyzing EPH” 5/16/2016 – 356 comments. The hours of discussion at the business meetings were just the tip of the iceberg.
THE THREAT IS GONE, SO THE CHANGES CAN BE DISCARDED. Any analyst could have seen from the voting statistics published over the years that it was theoretically possible to monopolize the final Hugo ballot. What kept that from happening? It wasn’t worth the expense for people who have no interest in the Worldcon to buy memberships so they could vote. Until it was. Until it became an expression of Sad and Rabid Puppy tribal loyalty.
However, some will argue that the threat they represented has gone away. The protections adopted against them made the process less transparent and less fair and should be dismantled.
Yes, there are people who think the slate-driven Hugo ballots of 2015 and 2016 were more fair than the present arrangement.
If you’re not one of them, and considering all the hard political work the community had to do to overcome the damage from the Puppies by passing and ratifying the existing rules changes, it doesn’t make sense to reopen a known vulnerability.
Discover more from File 770
Subscribe to get the latest posts to your email.
Absolutely 100% EPH shifted Puppy behaviour. True, many of the Sad Puppies chose to not understand what the rule change was but it was repeatedly cited as a closing the doors on them. Vox Day had a better grasp on the rule change but still misunderstood it.
Did it algorithmically stop slates? No, probably not. It tips the scales a bit against slates or even organic clumping of choices but not massively
Does it reward bullet voting? Not really.
As you say, the amount and depth of the discussion on EPH was substantial.
The nominating stats are now more informative than under the old system, so EPH increases rather than decreases transparency. I don’t believe fraud or voter manipulation has ever been a credible issue but if it were then EPH also makes it harder and more complex for somebody to fabricate voting statistics.
I’m not at all convinced that bullet voting is a problem. There have been about 1800 nominators over e last few years. If someone, be it Vox, or Seannan, or a YA writer with massive numbers, but no Hugo presence (and we know many exist) can motivate their fans to join and vote and get a thousand nominations, then that’s democracy for you.
The problem was never that there should be a work of dubious quality MilSF on the final ballot, it was that there was briefly nothing but.
It seems obvious to me that EPH should be re-ratified.
Is EPH perfect? Of course not. No system is. But it is undeniable that the Hugo Awards are the better with EPH in place.
Getting rid of it would be a retrograde move.
The threat has gone away?
Please repeat after me: Nazis NEVER go away.
Nazis may be forced underground. They may be curtailed, diminished, reduced, bankrupt, chastened, de-platformed and uprooted, but they NEVER go away.
They’re fungus that has learned how to encyst during times of drought…but the spores are still there, just waiting for a warm, moist, dark place to settle in and grow. They always find one and they always grow.
(“Listen, and understand! Those Nazis are out there! They can’t be bargained with. They can’t be reasoned with. They don’t feel pity, or remorse, or fear. And they absolutely will not stop… ever, until they have won!” A very enlightened Tech Sergeant from 2029. Coming from our future, Kyle has a bit more perspective on the subject than we do.)
One of the reasons that puppies were able to make such inroads was because fandom failed to educate newer generations of fans in the basic concepts of (Worldcon) Fandom, three of which are:
We’re family
Don’t bowb your buddy
We really do want the awards to be egalitarian expressions of the community, not bought and paid for or politically motivated tools.
Oh, and “FANDOM”, the institution, is supposed to be generally more important than any one individual’s goals or aspirations or even desire for undeserved accolades.
NAZIs NEVER go away.
“Bullet nominating” is not necessarily a bad thing – without EPH, an organic small but devoted group of fans that have a particular work they wish to be nominated might be recruited to a ballot-dominating slate strategy as the only means of getting their choice on the ballot – but with EPH, they can get their choice on the ballot without joining a slate.
I agree with Steve Davidson. Well said, and sadly, quite true.
No, EPH doesn’t reward bullet voting, except in some very specific cases. I wrote an explanation why it’s in the best interest of every nominator to put every work they find worthy on the ballot several years ago.
“If you meet with those historians, I’ll tell you what to say / tell them that the Nazis never really went away.” – Chumbawamba, 1993
Thank you for reminding me that, being at WorldCon in person for the first time in quite a few years, I should attend the business meeting and cast my vote for the re-ratification of EPH.
“Serpentine!”
Under EPH, the Hugo Administrator can’t check the work.
The results all depend on computer algorithms that can’t be duplicated manually. This is one of the reasons that I’ve retired from administering the Hugos (after doing so in 1998, 2002, 2006 and 2015).
They can be duplicated manually — they’re not doing anything magic! — it’s just infeasible in the face of the number of nominations that are likely to be present.
Nominations and the contents of the final ballot are, by definition, fairly opaque. The Hugo administrators have a fair amount of leeway to make decisions about eligibility, both of the works themselves and of the ballots nominating them. That filter occurs before any ballots are ever even made visible to the final accounting we see post-awards.
As with so many other such systems, it’s probably worth trying to understand what the goals are (of the overall nominating system) before trying to decide which threats to mitigate. If the goal of the system is to be comprehensible to the Hugo admins, then yeah, EPH is probably not the best one. If the goal is to generate a ballot where all of the varied parts of the SF/F readership (and WSFS membership, also) are able to plausibly propose works for nomination, I don’t see that it does harm and in some cases may actually provide some benefits (the one-bullet scenario described above is, IMO, actually a bit of a feature, if an accidental one).
I think 6 and 5 does more to prevent slating, for sure, but both serve to reduce the impact of it. I’m not sure which way I’ll be voting on it, but I don’t think it’s automatically a bad thing.
I think this falls squarely under the rubric of “if it ain’t broke, don’t fix it.”
It seems to be working reasonably well and is fulfIlling its intended purpose. I believe it should be left alone.
The results all depend on computer algorithms that can’t be duplicated manually. This is one of the reasons that I’ve retired from administering the Hugos (after doing so in 1998, 2002, 2006 and 2015).”
They were hard to check before that. But it is possible to do it by hand. Just much faster with a computer. (Which will treat every nominee by the same set of rules.)
EPH is weird, but it has also worked. 6 and 5 certainly does help, and the two together appear to be pretty dang useful as a deterrent if nothing else.
Chris
My thought is that bullet voting may improve the chances of getting THAT work on the ballot (but not by much). But it’s at the cost of other works that might have been nominated,
If all but one of the works you nominated have been eliminated you will be in the same position as if you had cast a bullet vote for that work. And since that one will have at least as strong support as the others, it’s your best hope of getting one of the works you nominated on the ballot.
Let’s say we get rid of EPH. It’s fixed the problem it was meant to fix right? That threat is neutralised right? We don’t need this opaque nomination system anymore right?
Let’s say we get rid of EPH (which by all accounts is working well), and the emotionally distraught juvenile canines resume slating activities. What then?
We will have another two years of Hugo chaos before we can implement a new system. Is that what we really want?
Soon Lee: Even if the canines themselves don’t come back, the capability has been demonstrated. Anyone who thinks the Worldcon needs a lesson about anything could follow their model if EPH is removed.
The late Dr. Robert Forward in a lecture about his ideas for antimatter propulsion systems pointed out why he was working on that and not CERN. CERN thought that once the existence of antimatter was proven the rest was “just engineering.” Likewise, the Puppies’ scheme for dictating the outcome under the previous Hugo nominating rules is now “just engineering.”
Bullet voting slightly increases the chances that a single specific work you like becomes a finalist, but it also increases the chances that none of the works you like become a finalist! The thing to remember is that pretty much every ballot becomes a bullet before the end, as candidates are eliminated. (This is how organic candidates survive in the face of slates!) So, if you put two items on your ballot, it will increase the chance that one of them will be eliminated, but the remaining one is now a bullet! And, better yet, it’ll be the more popular of the two, meaning it’ll have a better chance than the other one would have if you’d bullet-voted that!
Same logic applies with three or more nominees. The less popular ones will be eliminated, leaving the most popular of your choices as a bullet!
(This is slightly complicated by the fact that there are multiple finalists, so it’s possible for a ballot to have had more than one listed, and never actually become a bullet. But that’s really the opposite of a problem; it’s yet another reason not to bullet-vote!)
Too much for my grumpy little mind.
I don’t grudge the Puppies a vote, but they seem to grudge me one, and that makes me resentful and inclined to retaliate.
Democracy is the closest thing to a fair system that humans have come up with so far — which doesn’t mean it’s a fair system.
I work the polls. I believe that, here in Ohio anyway, the elections are rigidly controlled, policed, and verified for impartiality and honesty, and that the results are a genuine expression of the will of those citizens committed enough to vote. Which is not even close to everybody, by a long shot, and does include many persons whom I personally believe are hugely wrong. (Which is a critical factor in democracy.)
But what I suspect most people don’t realize is the enormous expense involved in setting up a system that comes as close to fair as ours is. EVERY precinct must print enough ballots to accommodate every single voter within the precinct. Most are thrown away (or at least recycled) unused. Each one costs $1.00. I work at one of the smallest precincts in the Hamilton County (we have 46) and we print 1,500 ballots. That’s nothing compared to the cost of voting machines, and electronic poll books. Plus the rental fees for the use of the hundreds of homes and businesses where the elections are held. Plus the staff. Each precinct gets at least 4 on-site poll-workers, more if it’s a large precinct. Trust me. we don’t work for free. And that’s not counting the huge base of support workers.
Why am I telling you all this? My point is that, beloved as our Hugos are, they are makeshift elections. WorldCon cannot possibly put in the kind of money necessary to render them — not fair –but significantly more fair than they are.
I personally think the EPH should be retained. But in all honesty, I didn’t really believe the Hugos were fair, even back before the Puppies. I just thought they were fun, and gave me a short list of books, to start the year’s reading adventures.
How is EPH actually implemented? An Excel spreadsheet that is passed from convention to convention? Is it a package on Github that gets downloaded every year? Does each new committee have to write up some software and hope it is bug-free? Does the Hugo Administrator take all the ballots home one weekend and come back on Monday and say “I took care of it, here’s the list!”?
(My assumption has always been that when EPH was passed, someone wrote up some software, did a bunch of test runs with made-up “typical” ballots, ran the numbers, and then crossed checked the results by hand. And then that software has been passed from convention to convention each year. But I don’t know, which is why I’m asking.)
Why anyone would think that giving the Puppies (or anyone else) the chance to completely swamp the Hugo nominations again is a good idea completely escapes me.
@Xtifr: every ballot becomes a bullet before the end
That sounds remarkably aphoristic!
@Karl-Johan Norén: Thank you.
@bill: The SDV-LPE algorithm that must be used is what is described in plain English in WSFS Constitution Section 3.9, “Finalist Selection Process”. The Constitution doesn’t dictate to the current Worldcon committee what toolset to use to implement 3.9’s method.
I see several open source implementations, including one by Keith “Kilo” Watt, who with Jameson Quinn (and help of many including noted coder/cryptographer Bruce Schneier) led the proposal back in 2015. I’m honestly unclear on which implementation is now in primary use. (Someone here know?) Any implementation should be tested against a standard set of input data, which I recall being done exhaustively in 2015-6.
“This enemy you cannot kill. You can only drive it back damaged to the depths and teach your children to watch the waves for its return.” – Quellcrist Falconer
Hi folks,
The Kansa EPH code, originated by Eemeli Aro, and updated in recent years by David Matthewman, is the best and we used it in 2017, 2019, 2021 and this year, so that is four of the six years that EPH has been operating. There is another one out there but it is way more clunky (we used it in 2020, it was also used in 2018 and will probably be used next year). If Glasgow wins the 2024 bid we will certainly use Kansa again. Obviously every year we test the new code extensively (there are always some changes to be made).
Just to be clear, though, the constitution should not mandate a particular code and future administrators should feel free to innovate. I like Kansa a lot, but I can totally anticipate that a better system could be invented, or that global changes in software fashion may force innovation upon us.
My “Issue” with EPH is the black box nature of it.
I hate bringing in paintball analogies but this one is an example of the consequences of a very similar thing.
The guns we used were originally single shot pump guns. They required the user to load a round (by working a pump: it loaded the round and set the valving mechanism. Pulling the trigger opened the valve, firing the ball.)
There were numerous ways to “cheat” (increasing velocity over limits imposed for safety and fairness), but all of them required a physical alteration to the guns’ mechanisms, all of which would reveal themselves upon physical examination.
A lot of gun cheating went on until we instituted a policy of inspection prior to testing velocities (which were done before every competition) and imposing penalties for cheating. When knowledgeable referees were inspecting guns prior to testing them, the incidences of gun cheating were greatly reduced.
A few years later, electronically operated guns were introduced – and objected to (on the grounds that they could not be reliably inspected for rules compliance). It got very, very bad (and is still not under control) because players could purchase “cheater boards” – replacement circuit boards programmed to perform illegal functions, either increasing velocity or rate of fire beyond safe levels.
Nothing can be done because the gun’s operation is now taking place inside a “black box”. The gun’s operation can not be physically checked.
It’s the same problem with electronic only voting. Without a paper record, we really have no way of reliably determining how people actually voted.
Its neither good nor bad by itself, but the fact should be acknowledged that utilizing “black boxes” in the process makes that process inherently more vulnerable to “cheating”.
Thank you very much, @Nicholas, for that and for your tireless work for fandom over many years. I notice you blogged in 2019 about adapting Eemeli’s codebase for Dublin 2019, with (as you say) help from David Matthewman and also Dublin 2019 IT head Arnaud Koebel, after its earlier successful use for Worldcon 75 (Helsinki).
That link upthread is broken, so: Kansa convention member management system code, and related repos. (“Kansa” is Finnish, translating to “people” or “tribe”.) The codebase addresses numerous convention-management needs, and appears to be coded in Javascript/React using node.js, with PostgreSQL used as a database back-end. It is architected to use a client/server model, such that the convention volunteer interacts with it using Javascript Web code.
I’m glad that, in Nicholas’s experience, it works well. Necessarily on account of the Web 2.0 design and integration with other convention-running functions, this ends up being a complex codebase (but admittedly, I’m not rushing to delve into the code because Javascript is not at all my cuppa). The actual algorithm in Section 3.9 is really not very complex, and I note that some of the other implementations are pretty easy to read in source code form. (That is not a recommendation.)
@Steve Davidson wrote:
My “Issue” with EPH is the black box nature of it.
Let’s explore what amounts to. Prior to WSFS members’ decision at the Kansas City and Helsinki Business Meetings to switch to novel, approval-ballot, proportional polling method Single Divisible Vote with Least-Popular Elimination aka SDV-LPE (under the less ponderous name E Pluribus Hugo aka EPH), the Worldcon committee (via delegation to its Hugo Administrators) used a simple “approval voting” algorithm: Members could nominate up to five (later six) works published in the previous year, per category. Whichever set of five (later six) works got the greatest number of nominations became finalists. I’m sure there was a great deal of complex record-keeping (not to mention work verifying eligibility and parsing people’s varied and peculiar ways of denoting what/who they want to nominate). It’s a fair bet that software was heavily involved.
After 2017, the same complex record-keeping and parsing occurs, and now software is again used, but it’s different software. Being open source, the code can be audited by anyone.
Both through 2017 and after 2017, fandom places unavoidable trust in the commitees and Hugo Administrators to do the job competently and without favour. IIRC, administration involves multiple people as a cross-check. Because the raw nomination data cannot be fully revealed on account of privacy issues, independent verification from outside the current Worldcon’s staff is not, I believe, possible. Time and energy permitting (which is saying a lot, and I’m not spending people points here, just speaking of possibilities), in theory a year’s administration staff (or a second team of convention staffers) could cross-check one EPH codebase’s results against others likewise believed to accurately implement the algorithm. Again very theoretically, the data could also be crunched on paper.
My point is that calling the arrangements up through 2017 not involving a black box, and arrangements after 2017 involving a black box, is… a perspective, but I’m picturing some dark rectangular things in the old methods, too. (I might be missing something.)
It’s unlikely I confused any File770 regulars, but my sentence “Members could nominate up to five (later six) works published in the previous year, per category” suffered my sloppy copyediting (as to the parenthetical words), and misstated the effect of the separate Five of Six nominations proposal we also ratified in 2017 in Helsinki: For years following, nominators still may specify up to five works/persons per category, but the six most-nominated ones become finalists.
I don’t see how this is a concern about E Pluribus Hugo. Hugo nominations and votes were taken electronically long before EPH.
The EPH process is not a black box. The code can be inspected and tested by anyone because it is open source. Results from test data can be compared to a hand count conducted with the same data. Because there are multiple EPH implementations, the same pool of nominations could even be run under each implementation for comparison of the results produced.
Before EPH, the software used to count in the Hugo Awards was not open source. During one business meeting where a proposal to require open source was deliberated, the maintainer helped defeat it by saying he would allow it to be inspected upon request.
So I made a request a few months after the con. It was denied.
EPH is not a black box: if the ballots were publicly available, I could verify the results myself, given nothing more than a description of the algorithm. (IOW, just the constitution.)
Of course, the ballots are not publicly available, so nobody can verify the results, but that was true before EPH, and was not changed by EPH, and could be changed even if EPH were kept! Because EPH is not a black box.
@Xtifr: As I said, anyone the current Worldcon committee is willing to trust with the ballot data, i.e., some variety of trusted insider, could vet the EPH results using the Kansa code, or using any of the other implementations deemed to correctly implement Section 3.9’s method, or by hand.
I personally doubt that is worth anyone’s time and effort, as, if I recall correctly, the amount of QA that went into all this was pretty epic, and committees are competent in whom they trust and in what amount of cross-checking (e.g., involvement of multiple staffers) is present.
Anyway, quite. The implementation used is not a black box any more than was/were the means used before — and arguably less so. At least, the contrary position would require some explaining.
Thanks, Nicholas Whyte and Rick Moen, for answering about the EPH code.
The claim that EPH is a black box that can’t be checked is false. If the administrators were willing to disclose the surviving ballot groups at the start of the last 10 rounds (e.g., 10 ballots nominating A, B, and C, 13 ballots with E and F, and so on), I could do the last 10 rounds myself by hand (although an Excel spreadsheet would make it easier).
But it is not necessary to replicate the full calculation in order to spot check the results. Anyone who nominated a work that was eliminated in the 8-16th position and had nominated at least one other surviving work on their ballot can check the published points table to see if the results are consistent with the points from their ballot being redistributed correctly according to the algorithm when that work was eliminated. If not, they can show proof of error or shenanigans by disclosing how they voted in that category, using their final confirmation email to confirm to administrators or an auditor that there was a mistake. Multiple ballots from a group of friends or family who are willing to share how they voted with each other can provide an even stronger test.
If lots of voters are in a position to generate falsifiable hypotheses about how the points should be redistributed in a given round based on their own ballots, and yet the data keeps not getting publicly falsified, that should increase our confidence that both the code and the administrator’s handling of the ballots was correct. After several years, that confidence should rise to a virtual certainty. We could further improve this checking by creating and distributing tools that make it easier for people to check their own ballots against the result tables.
I don’t have a concrete example handy, and I may not have time to check my old ballots before leaving for Chicago, but I can explain the general idea. For ease of use and to get rid of rounding errors in the data, I usually multiply all the point totals by 60 before processing to produce “megaPoints”, rounding to the nearest integer to clean up roundoff in the published tables. So instead of each ballot having 1 point that is distributed 1/5, 1/4, 1/3, 1/2 or 1 to each surviving work, each ballot has 60 megaPoints that are distributed 12, 15, 20, 30, or 60 to each surviving work. The basic algorithm is the same, this just gets rid of fractions and makes it easier to see the constraints on what point adjustments are consistent with a given ballot.
Then I take the difference between consecutive columns in the megaPoints table to see how the megaPoints were redistributed when one of my works was eliminated. My ballot should have contributed 3, 5, 10, or 30 additional megaPoints to each of my remaining works depending on how many surviving works I had initially. If those works don’t show the expected gain, or if the remaining gain when I subtract off my contribution isn’t a number that can be produced as some combination of 3, 5, 10, and 30 points contributed by other people, then I can call shenanigans on the calculation and disclose my final ballot to prove it.
It’s often easier to see the exact redistribution of your own ballot points in some of the smaller categories where there aren’t as many points being redistributed each round, but you should be able to check consistency with your ballot even in the bigger categories.
@Dave Wallace
You are probably correct in a technical sense, but practically speaking, for the vast majority of nominators and voters, EPH is definitely a black box.
The math skills necessary to follow, understand, and duplicate the results of the old system are held by pretty much anyone who is competent to independently nominate and vote — counting, addition, etc. Basic arithmetic.
OTOH, I (an electrical engineer with 35 years experience) could not describe off the top of my head what goes on in EPH. If I wanted to double check results, I’d have to spend time reading a description of the algorithm, write some software that implemented it, come up with some input data, and make sure that my version ran properly. Then I’d have to download the software linked above, figure out how to get it running, run the same input data, and compare results.
Essentially, you need to be experienced in implementing software, which is a skill set that certainly some members of the WSFS have, but the vast majority don’t.
I’m better positioned than many (with years of experience in math, some experience in writing software, and comfort with computers and how they work in general), but there’s no way that I could convince myself that the algorithms are working properly without a major commitment of time.
For only a select few is the system anything other than “I vote, everyone else votes, the admins dump the votes in a machine and turn the crank, and winners pop out” — the classic black box.
Saying that most people are unwilling to undertake the time and effort necessary to independently verify the results is not at all the same as saying that they could not do so if they wanted to.
Under the old system you, as a member of the public, could not check the nominations tallies either, regardless of your ability to add one and one together to get two, because you would never be allowed to have copies of people’s ballots. Further, you would never be allowed to perform your own validation of voter eligibility using confidential registration information. The old system was opaque, too.
@Kyra — I’m saying that I think I could do it, if sufficiently motivated. I’m also saying that a significant fraction of nominators and voters could not do it because they don’t have the necessary skills. And finally, I’m saying that practically speaking, there’s no difference between the two groups; only a theoretical one.
@Mike Glyer —
True, but that’s a matter of trusting the integrity of the Hugo Administrators to do the basic counting and data entry correctly. The Hugos have always had to accept that the Admins are acting in good faith. EPH is new in that the process of going from a stack of raw ballots to eventual nominees is not by any means intuitive.
This is an absurd premise. If you don’t want to take the time to understand the process, fine, but it doesn’t become a black box because you lack the desire to do that.
The math isn’t complicated to do by hand; it’s tedious. Software can do it easily and the reliability can be assured through testing.
That’s not true of the old system UNLESS you have the ballots. The EPH breakdown provided with the published stats provided way more information on HOW people voted than the old system did.
It’s the old system that was a black box: Work X got N votes. That’s it. With the EPH states, we get Work X got N votes…and an indication of how those votes interacted with the other works. That is quantifiably more transparent than previous systems
I’ll also add that I’m one of a select group of people who lost out from being a Hugo finalist because of EPH (I think Patrick Nielsen Hayden is another).
@rcade
Agreed. But most Hugo people can’t write the software, and can’t put together a validating test program. They have to have faith that the admins have done this part correctly. Therefore, the WSFS implementation of EPH is, from their perspective, a black box.
@bill: They have to have faith that the admins have done this part correctly.
And this differs from the old system how, exactly?
You really don’t seem to understand the meaning of ‘black box’.
I wrote a version of EPH in 2015 that worked in Excel only using formulas (no VBA). It’s not a trivial algorithm and sure maybe not MOST Hugo voters but a hell of a lot of them could…and come to think of it a lot of them did in 2015.
I suspect, if I had the time and inclination I could make a no-code-only-formulas Google spreadsheet version that could practically handle the Hugo nominations.
@Camestros Felapton: Careful. If you take an algorithm that cannot be understood by mortal humans, and get it working in Google sheets, which knows what it might do next?
@Tom if it’s anything like Excel, it’ll incorrectly assume one of the finalists is a date.
And the Hugo for best Fan writer goes to November 11th, 1912!
@PhilRM
I think I do.
But give us your definition, and we’ll see if it applies to EPH as implemented by the Hugo Awards.
And this is different from all the previous years, how?
LOL