Tor released a curious best of 2011 list today as voted by us, the apparently incompetent internet populace. It's a bad list and they know it, as evidenced by their attempts to ferret out the causes behind the self-described "interesting" results.
- The Wise Man’s Fear by Patrick Rothfuss (140 votes)
- The All-Pro by Scott Sigler (105 votes)
- The Alloy of Law by Brandon Sanderson (63 votes)
- The Seventh Throne by Stephen Zimmer (63 votes)
- Ready Player One by Ernest Cline (55 votes)
- The Final Arbiter by Mark Rivera (55 votes)
- A Dance With Dragons by George R. R. Martin (53 votes)
- Fuzzy Nation by John Scalzi (52 votes)
- Dancing With Eternity by J.P. Lowrie (50 votes)
- Among Others by Jo Walton (49 votes)
I agree with Tor.com that something is definitely amiss. I'm not pretending to be the be-all, end-all when it comes to genre literature but when I haven't heard of 3.5 (familiar with Scott Sigler but not The All-Pro) novels on a voter defined Top 10 list, it's a bit unusual.
It appears certain authors may be gaming the system. Getting on to the list only takes 50 votes, and you would only need 141 to climb to first. Both of those numbers are definitely doable with a sufficiently motivated fanbase.
But that would be unfair of me to assume right?
There's no way that Stephen Zimmer's shameless plugging on his website had nothing to do with his 4th place slot, nor the fact that this monstrosity took home Best Cover of 2011.
 |
| It's like putting Dan Fogler on the cover of the SI Swimsuit Edition. |
Then there is Mark Rivera's constant updates on his book's facebook page. It is surprising that he got 55 of the 82 people who liked his page to go vote for him. I'm impressed - I've got 40 odd cousins and I thought that was a big family.
J.P. Lowrie got in on the act as well. And internet campaigns? Not without Scott Sigler.
BUT WAIT!, you say. You're being unfair, you're not using logic. Maybe it's possible that these books were just really good self published novels. Maybe a lot of people read them, liked them, and voted for them without being asked to by the authors.
Fine. Let's get freaky. -nomically at least.
Let's compare by matching this up with the number of amazon reviews each book got. While the # of Amazon reviews isn't a good indicator of quality, it is a fairly good number of the people that read your book, assuming a comparable % of readers are reviews. I would even assume that people that read self published fiction are MORE likely to review it based on what is likely a closer familiarity with the author. Face it, it's harder to stumble onto self published books than mainstream ones.
My theory is that the more people that read your book, the more people that can vote for it. (Assuming that there are no people who voted for a book without reading it, who would do that?) And since this is purely a numbers game, if 10% of 1000 that read a book like it, you would have more votes than a book that was liked by 50% of 100 readers. And that's being considerate - the numbers of readers for Name of the Wind versus The Final Arbiter are probably more in the neighborhood of 500,000 to 1000.
Lets take a look at the numbers.
Now, I reduced A Dance With Dragons's numbers because it skewed the chart too much at a whopping 31.1 reviews/vote but I think you might be able to guess which of these books might have had their own voting campaigns. Three books have a virtually non-existent ratio and the 4th (Dancing With Eternity) appears to be boosted by some fake Amazon reviews. A pseudonymous reviewer who writes a very short, very positive 5-star review for that specific book and nothing else on Amazon? Look out for herbivores - you are a plant.
Blegh.
What is the point of having these lists if you can't trust them to be anything more than the most popular books of last year combined with the most mobilized fan community? Complain all you want about juried awards but I'm starting to think those are the way to go.
And Tor.com, is this really a list you want to put your name on? You're better than that.
Here is the raw data for those who care.
The Wise Man's Fear - 713 reviews / 140 votes - 5.09 reviews/vote
The All Pro - 16 reviews / 105 votes - 0.15 reviews/vote
The Allow of Law - 150/63 - 2.38 reviews/vote
The Seventh Throne - 3/63 - 0.05 reviews/vote
Ready Player One - 613/55 - 11.15 reviews/vote
The Final Arbiter- 2/55 - 0.04 reviews/vote
A Dance With Dragons - 1650/53 - 31.1 reviews/vote
Fuzzy Nation - 113/52 - 2.17 reviews/vote
Dancing With Eternity - 40/50 - 0.80 reviews/vote
Among Others - 48/49 - 0.98 reviews/vote
This keeps happening in polls for online book groups like on Goodreads too. Makes you not want to join in because it's not really getting picked by people who actually want to read.
ReplyDeleteJust thinking about this, I wonder if they could say to authors that if they catch any advertising then their book gets booted from the list. Otherwise, I think you're right about the judge-voted awards.
John Scalzi's Old Man's War won best book in their Best SFF book of the Decade elction last year in a similar manner if I remember correctly. It's a shame that this seems to happen with most (online) non-juried award. It's not a case of merit anymore, but a popularity contest. In my case I didn't even bother voting this year.
ReplyDeleteBravo.
ReplyDeleteInteresting. Might blog about this shortly, linking here of course.
ReplyDeleteExcellent post. Of course I'm not interested in/impressed by juried awards either ... so what's left? Personally I like to read the top 10 (or 23 if you like) lists of individuals (ones whose tastes and peccadilloes I know through their blogs). These listings are nothing but bias with no pretence at being anything but one person's tastes. And all the more appealing for it.
ReplyDelete- imagine a boot stamping on a yeti face — forever
Didn't we figure this out when Yao Ming was dominating the All-Star ballots for the NBA through his whole career?
ReplyDeleteI think there's merit to online polls, I think it's a mistake to let just about anyone vote though. The other aspect you don't touch on (if you did I missed it) is people who vote multiple times, erasing cookies, changing IPs, etc.
The problem is not that internet polls sucks, but the way the polls are set-up from the get go, particularly the processed used to arrive at the books you can vote for, and those you allow to vote for them.
As for juried awards, the problem there usually comes down to a set of criteria that they use to pick winners that many just don't agree on or representative of what the "community" feels might be the most important, and there's always complaints of bias.
For these types of things, if you really want to get good results, I've always been of the belief that you should be inclusive while being exclusive, whatever that is.
I ceased being impressed by these kinds of polls since I saw a Goodreads group of independent authors who talked openly about scamming the Amazon and Goodreads systems by voting en masse for one another's books. If they will do it on Amazon and Goodreads, then they will promote one another across the board.
ReplyDeleteAlthough to be fair, I'm not sure how Tor.com (or any web site running a poll like this) could possibly police it.
I'm like Mark in this respect: I quit putting stock in polls. Once in a while, they showcase a well-written novel, but popular doesn't always equal a well told story. The only polls I watch are top 10+ lists compiled by book bloggers who share my tastes in books.
Wonderful post, though. I find your results quite fascinating.
I wouldn't say its a travesty that cover won, but it is clearly a case of the problem of gaming polls.
ReplyDeleteI used to think that Hugo voting should be free--but then we might wind up with that cover for "Best art" because of a mobilized fan base.