Misunderstanding How Reviews Work

I don’t often talk about non-bookish things here on Bibliotropic, but yesterday I came across an article that’s about video games but has a lot in it that applies to what I do here. Please, take a moment to read over the article: Devs Concerned As Steam Makes Big Adjustment To Player Reviews.

Done? Okay, awesome.

So how does this apply to what I do, exactly?

It’s all about reviews. And about whose opinion should actually count.

For those who didn’t read the article, it’s all about how the company behind the popular gaming platform, Steam, has made the decision to by default only allow weight to reviews of games purchased through their platform. This is an attempt to cut down on instances of game developers giving out free redemption codes for their games in exchange for reviews.

Which sounds good on paper. Until you realise that there are a whole load of problems with this well-intentioned idea of theirs. The quote from Valve even makes mention of said problems:

An analysis of games across Steam shows that at least 160 titles have a substantially greater percentage of positive reviews by users that activated the product with a cd key, compared to customers that purchased the game directly on Steam. There are, of course, legitimate reasons why this could be true for a game: Some games have strong audiences off Steam, and some games have passionate early adopters or Kickstarter backers that are much more invested in the game.

In other words, they already know that there are a whole load of ways to get games on Steam that don’t require purchasing them through Steam. The popular site Humble Bundle often sells bundles of games at a great price, and Steam keys are part of said bundles. Now anyone who buys their games through Humble Bundle, either to take advantage of good pricing or because they also like to donate money to charity, now will have their reviews not count on Steam. Ditto anyone who supports a game through Kickstarter and gets a redemption key that way.

But the kicker for me is that Valve draws no distinction between “free game in exchange for review” and “free game in exchange for positive review.” And there is a difference.

For one thing, many reviewers, no matter what they review, end up getting free stuff for review, if they do it long enough and end up well-established. This holds true not only for books, but also for video games. The smaller the company, the more likely they are to reach out to smaller reviewers, too, so both sides of the exchange can benefit. “Please considering reviewing my game if I provide a complimentary copy for you.” Bigger companies do this too. I can’t tell you how many reviewers and Let’s Play channels I’ve heard drop mentions of getting games from gaming giants like Nintendo or Square Enix, either for review or for play, to drum up hype about their games. It happens more often than you may think.

Just look over a few random reviews of mine here. About 90% of what I review, I get as a review copy.

But according to Valve, reviews like mine don’t deserve to count. Because they can’t guarantee I wasn’t essentially paid off to say something good.

Even Amazon, with all their controversial decisions over the past decade, didn’t go that far. Yes, they weight more heavily the reviews written in connection with a verified purchase through them, but they don’t take away all the weight from reviews of products that are purchased elsewhere. When last I checked, if a book has a 5-star rating from a verified purchase and a 3-star rating from a not-verified purchase, the book will still have a 4-star rating. The written review from the verified purchase will show higher on the list than the other one, but the overall rating stays the same. What Valve did is taking that mentality up to 11, and saying that not only should verified purchases carry more weight, but they should carry all the weight.

So not only does this screw over people who get their games through Humble Bundle or by supporting things on Kickstarter, not only does it screw over developers who hand out complimentary copies in exchange for honest reviews, it also screws over reviewers in general who get their games a variety of ways. Valve has just made it harder for people who work hard and work legitimately to establish themselves, whether they’re establishing themselves as a good producer of games or as a voice that can be trusted to speak the truth.

This, in the name of cutting down the possibility to inflated reviews that might be related to buy-offs.

Let’s just say that I’m both unimpressed with this, and also very thankful that my own reviews aren’t treated with such disdain by the platform upon which I review.

2 comments on “Misunderstanding How Reviews Work

  1. Pingback: Highlights of the Week: Sept 16 [Book Cover Reveal] - J. A. Alexsoo

  2. Pingback: September 2016 in Retrospect | Bibliotropic

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s