Marc Doyle on the Metacritic Effect

Interview - Behind the scenes with the site's founder

For many, Metacritic is just a quick, digestible way of getting a cross-section of critical opinion on new releases. For others, it is harming the credibility of game reviews themselves and holding dangerous sway with publishers. We sat down with the site's founder, Marc Doyle, to get his views on the site and the recent controversies surrounding it (readers may also be interested in Stevie's article on the subject).

Thanks for speaking with us Marc, what is your role at Metacritic?

I founded Metacritic with my partners Jason Dietz and Julie Roberts in 2001. I'm currently Metacritic's Games Editor and the Senior Product Manager for the site as a whole, which is now owned by CBS Interactive.

What was the original vision for Metacritic, and how has the site changed over the years?

The initial spark for Metacritic was ignited by my law school classmate Jason Dietz, who had created several websites during the internet's infancy, including "The List of Possible Band Names" which was amazing. He approached me and we developed our mission for the site which really has not changed since our initial discussions in 1999. Before the rise of the world wide web, consumers were at the mercy of their local critics for advice about which movies to see, what games to buy, etc. Furthermore, the influence of advertising campaigns and the emphasis on fawning quotations from obscure critics that nobody had ever heard of in newspaper/magazine ads was huge. Bad movies and games could be thrust on consumers without a great deal of education to rebut the messages from PR companies or the potential biases of individual critics. Metacritic's mission is to bring together the most professional, skilled and respected critics in each section of our site (movies, games, music, and TV) to provide our users with the most reliable indicators of quality upon which they can base their purchasing decisions. Again, this type of service would not have been possible before the web was developed. Our individual editors spend a great deal of time researching their individual industries to stay abreast of who the most reliable critics are.

How does Metacritic assess and score those review submissions arriving without a source points score?

All product pages on Metacritic state the following: "All critic scores are converted to a 100-point scale. If a critic does not indicate a score, we assign a score based on the general impression given by the text of the review." The following is from a forum posting of mine: In other sections of Metacritic, we regularly track publications that do not assign a score to their own reviews (LA Times, NY Times, Hollywood Reporter, Variety), and we've always estimated the scores based on the impression gleaned from the review. I've only very rarely taken this approach in the Games section of Metacritic because gamers and the games industry are so sensitive to our scoring system and process. However, I did track the New York Times game reviews for years - a site which had no scores. In that case, the lead critic from the Times, Charles Herold, emailed me his unpublished scores every time the NYT published a review and I used those numbers on Metacritic (See the scoreless New York Times review of The Orange Box by Charles Herold which appears on Metacritic's Orange Box page with an 88 score). My argument is that if the score comes from the horse's mouth, from the critic him or herself, so that I know the critic's intent definitively, I am satisfied that I can maintain 100% accuracy (incidentally, we stopped tracking the NY Times when they laid off their veteran lead critic a few months back).

So with respect to Variety (a recent addition to our games publication lineup), I've come to the same arrangement with their lead editor. Each time a game is reviewed, he sends me the link to the review along with their unpublished 0 - 100 score - a score which I post at Metacritic without alteration or personal input. I've vetted a great many reviews and scores from Variety's team before deciding to pick them up like I do with every website and magazine that I decide to track.

Who is charged with that process, and how are they qualified in terms of industry experience (does MC have editorial staff)?

Each editor and his/her staff is in charge of assigning scores. Our team members have read tens of thousands of reviews (often hundreds of thousands) and we've honed our skills in assigning scores. However, in our movies section in particular, most of the critics we track have kept up a running dialogue with us, so that if we're "off" in the scores we've applied, they'll let us know about it, and we'll adjust the scores accordingly. Again, we don't do this in the games section - we simply receive the scores directly from the critics themselves (see Variety).