News, Posts

Behavioral Study Analyzes the Influence of Review Scores

by Jul 8th 2010 10:00AM 4 Comments

Behavioral Study Analyzes the Influence of Review Scores

A Study conducted by the Southern Methodist University (SMU) and Electronic Entertainment Design and Research (EEDR) has recently attempted to isolate the effects of review scores on a person’s opinion about a video game experience. This test’s intention is to show the effect (if any) that “low” and “high” scores have on a person’s opinion of a game.

In Group A participants were shown high review scores of a game they had never played and then played the title for 20 minutes. In Group B low scores were shown before the game-play commenced. Group C was the control group of this experiment which was not shown any review scores or review information about the game before playing.

The game in question was Plants vs. Zombies. After 188 people participated in this experiment (165 of which qualified for the study by not already playing the game) each person completed a review of the game including giving it a score from 0 to 100. Read on for the results breakdown to that study and what EEDAR has concluded from it.

The result of this test was that Group A, who was shown high review scores before playing, had a distinct tendency to score the game higher than Groups B and C. While the opposite appeared to be true for Group B who scored the game lower after seeing low review scores prior to playing.

The average score from Group A was around 85 while Group B came in at 71 and the control Group C had an average review score of 79. The participants were then asked if they would recommend the game to a friend and/or relative. Willingness for each group to recommend the game varied in a seemingly correlating fashion. Again, exposure to review scores before playing seemed to effect the scores given from Groups A and B compared to the control. 91% of survey participants said they would recommend Plants vs. Zombies to a friend from Group A, who had seen high scores, whereas only 65% of Group B would. In Group C 79% said they would recommend it.

EEDAR has come to some rather informative conclusions from this study. Even though Group B had been consistently scoring the game lower, and less willing to recommend it, 65% of them would still recommend it. Beyond this, the survey participants were offered $10 cash or a copy of Plants vs. Zombies after playing the game. In Group A only 38% of the participants took a copy of the game over the cash yet 91% of Group A would positively recommend the game to a friend.

Behavioral Study Analyzes the Influence of Review Scores

The main conclusion of EEDAR is that free demos and instant purchase ability may lead to a greater word of mouth, even if a game has somewhat negative reviews. They compare recommendations between gamers to recommendations between friends and co-workers about a good dentist or real estate agent.

With all of the findings in mind EEDAR also mentions the limitations of this study and reminds us that Plants vs. Zombies is a game with a news aggregate score considered high, and that these results may be extrapolated for a low quality game as well, but that more accurate results would come from an all-new study done with a universally accepted “low quality” game. This study also used qualitative remarks such as “Game of the Year” in replacement of numbers in some instances and EEDAR states that it is “unclear” which of these were more important in influencing the data.

Behavioral Study Analyzes the Influence of Review Scores

  • http://www.nerdsontherocks.com Joe Doe

    That’s actually an interesting study, granted like any study, its hard to take away too much vital information. I’m curious how the inverse works if you show people negative/positive reviews of a game that is generally dubbed bad.

  • http://www.dualshockers.com Joel Taveras

    Very interesting read, and great write up Jon. It shows that review scores are pretty much a form of indirect / direct marketing. I wonder if it would have been different had they read a review with no score like the ones found on Kotaku. A much higher percentage would have probably just taken the money.

  • Red

    I think this study proves what most level headed gamers always had an idea of, and what many may have noticed in themselves.

    This is a dangerous thought, when taken from a corporate standpoint. We are insane if we think that companies haven’t been using this review principle in their business models for years. The issue is that consumers are too easily manipulated into believing something is of high quality, and that it is this perception that is one of the major factors in the decision to buy.

    Critics are as easily manipulated as the consumer base as a whole, it is no secret that games that are hyped heavily stand a much better chance of being highly rated on a place like Metacritic than one of similar quality that has received less hype. We could try to refute this, but it is the basic principle of advertising: create a perception of quality that likely has nothing to do with the reality of the product.

    Big picture, though. It has become clear that publishers are now using Metacritic scores to not only dole out bonuses, but influence whether or not sequels get made. This study proves that a Metacritic style score has a direct and profound impact on the purchasing habits of gamers. What we get is a situation where the manipulation of a Metacritic score through hype, and the subsequent tallying of this score, becomes a highly important business tool that can influence a purchase consideration greatly. Mind you, it does not matter if hype gains it only one or two points on Metacritic, any extra points could end up paying millions in dividends.

    In today’s gaming landscape, it is all about maximizing profits. This study proves that a superficial measure of quality that was completely made up greatly influenced sale possibility. You want proof that console gaming is being wholesale homogenized? All we have to do is look at how the profits are made. Only make games you know will score a high Metacritic, and abandon nearly everything experimental. You want to know where gaming is going? Ask me in 15 years when we are all playing Resident Evil 16, and Killzone Episode XXIV: A New Hope.

    • plmko

      But when you talk about sequels being made for profit, it brings a very negative ring to the game, almost as if it’s a souless game that is less than great.

      There are sequels are out there, that do the industry good by not only racking in the cash but also set higher bars. Looking at Uncharted 2 is a good example of what a sequel should be: bigger, better and badass; certainly a series I would continue playing even if they are up to the 20th game by 2030. Then in stark comparison when you look at a series like Halo, that refuses to evolve beyond its roots and continually baits gamers into upgrading…well thats just wrong.

Latest Stories

Recent Comments

Powered by Disqus

By continuing past this page, you by your continued use of this site, agree to be bound by and abide by the User Agreement. © 2012 DualShockers.

All content, including editorials, comments, and any other written works on DualShockers, are licensed under a Creative Commons License permitting non-commercial sharing with attribution.