Thoughts on game reviews

[Graham Mitchell]

So, we've definitely debated to death on these forums about how basically useless number-based reviews are. Yet every time I get an issue of Game Informer and I read the review section I want to go punch the wall.

Now, I haven't even played the game yet, but the review that triggered me into this rage today was Game Informer's review of Shin Megami Tensei: Catherine. They gave it a 7 and talked about how boring and frustrating it gets. They recently gave the new Alice a 6.25 or something awful as well, but it's a perfectly fine game, and does what it's supposed to do admirably IMO.

I was pondering all of this and trying to figure out why I disagree with these guys so often (they take a shit on nearly every game that sounds interesting to me) and it suddenly hit me. I think that in order to write the review, the writers are expected to play the game compulsively and obsessively until they beat it and squeeze as much life out of it as they can. But that's not how I play. I change games a lot. I'll get into one game intensively one week, then shelve it and come back to it in a couple months. Know why? Because ALL games get boring after too much repitition.

Even the Zelda and Metroid games, as much as I love them, can get old. After 2-3 hours, sometimes I've just fallen down that hole in the dungeon one too many friggin' times, and I really don't feel like going through the whole level again. So I stop playing. When I pick it up the next day or whenever, it's fresher to me, and my interest is easily re-captured.

Anybody who's worked in game testing will tell you that compulsively playing a game for 40 hours+ a week will make you hate it. I have friends that play tested for Microsoft and Nintendo and they ended up just not playing video games at all for a year after they quit. If that's how reviewers are expected to play the games, then their interpretations are basically meaningless to a phenotypically normal person with no impulse control disorders who plays the game in a healthy fashion.

I also think this is why Heavy Rain got a 9+ from many reviewers, despite the fact that the gameplay is basically Dragon's Lair with more buttons and minimal replay value. The story has lots of sex and violence in it, and is pretty captivating at times. I think this lends to the game being more palatable in extended plays, despite it being not much of a game.

Thoughts? You guys think I'm nuts or am I on to something here?

[Icarus Moonsight]

I am certain getting high quality, objective with reasonable analysis and comparisons, review prior to or on release is entirely possible. As it stands, the people charged to doing it are failing at it. But then, print is dead, so who cares?

[Aussie2B]

I don't think professional game reviewers are playing games anything like game testers. I don't think they're playing games obsessively at all, and I definitely don't think their employers expect that of them either. The employers just want the piece done and written, as fast as possible, so they have something to go along with all of the ad space they're trying to sell. Because of that, it's true that the reviewers are sometimes forced to marathon games because they're given so little time to complete a review. They do aim to beat a game at least (but I can almost guarantee you that some games are written about without the credits ever being seen), which is is no sweat for a game under 10 hours, but it can be a nightmare for something like a long Japanese RPG. Because of that, I'd say the problem is the exact opposite. They're not playing the games enough. They're rushing like mad to get to the end, so they don't have the opportunity to dig deep and really learn and appreciate (or loathe) what it has to offer. Bottom line, reviewers need more time to more casually and extensively play a game, and their employers need to appreciate them more, pay them better, and encourage excellence in the quality of their work. There are indeed bad game journalists, no doubt about that, but the main problem is the structure of the industry created by the higher ups.

[Graham Mitchell]

I don't think professional game reviewers are playing games anything like game testers. I don't think they're playing games obsessively at all, and I definitely don't think their employers expect that of them either. The employers just want the piece done and written, as fast as possible, so they have something to go along with all of the ad space they're trying to sell. Because of that, it's true that the reviewers are sometimes forced to marathon games because they're given so little time to complete a review. They do aim to beat a game at least (but I can almost guarantee you that some games are written about without the credits ever being seen), which is is no sweat for a game under 10 hours, but it can be a nightmare for something like a long Japanese RPG. Because of that, I'd say the problem is the exact opposite. They're not playing the games enough. They're rushing like mad to get to the end, so they don't have the opportunity to dig deep and really learn and appreciate (or loathe) what it has to offer. Bottom line, reviewers need more time to more casually and extensively play a game, and their employers need to appreciate them more, pay them better, and encourage excellence in the quality of their work. There are indeed bad game journalists, no doubt about that, but the main problem is the structure of the industry created by the higher ups.

Well, if that's true, then it's no wonder all the JRPG's get dumped on all the time.

It's not so much the lack of objectivity that bothers me about these reviews. It's more the fact that so many of these reviewers (especially in the print world) seem to expect their games to be more of a movie than a game. If the game has any kind of story, they expect gameplay and cinematics to be seamlessly integrated like a Valve game or Mass Effect. I'm constantly reading how "out of place" the actual gameplay seems in the contrast to the cinematics. They comment about this in the Catherine review, in fact. I also remember Velvet Assasin getting just trashed on basically because it's level-based, and because each segment of the game is a little vignette, and none of them seem interrelated. They completely ignore the game's stellar visuals, intelligent writing, and all around FUN stealth gameplay. Yeah, when you die, you restart from the checkpoint. So what? That's a theme that's been around since Super Mario Bros. Why is it suddenly so uncool for a game to be challenging or demand a certain level of precision?

But, maybe you're right, Icarus. Maybe it all just because of a lack of objectivity. But I think if these people went back and played stuf from the pre-PS1 era they might understand that a game can be enjoyable despite not having scripted sequences and being slightly hard.

Just my 2 mesetas.

[Icarus Moonsight]

It's genuinely hard to write about experiences while taking yourself out of the picture. Reviewing works well also requires a broad experience base in the area you are writing about. Seems to contradict, but that's how it is. The only way I could write a review for a shmup is for other shmup fans without really pulling hard on bias to get it down in a fair light. I couldn't write a Star Trek game review for a similar reason, but rather for people that generally dislike it. And when I say could/couldn't I mean, with my integrity intact. If you are just doing the review for yourself and anyone that cares to look and read, then the standards relax quite a bit.

Perhaps they are serving their positions completely well to these points and expectations, we're just on the outside of it. I'm also fine with that.

[Aussie2B]

Objectivity is impossible, seeing as reviews are critiques, which are completely subjective by definition. I think what you're getting at is bias. There are too many reviewers out there that will love any garbage just because it's in a particular genre or features some kind of content that they worship, or, conversely, will rag on something just because it's in an unfavored genre or such.

[G-Boobie]

On the subject of numbered reviews being worthless, its interesting to note the difference in perceived value between a literary, movie, or music critic and a game reviewer(the fact that the word 'critic' is seldom applied to game reviewers is pretty telling, too).

A critic of film, novels, or music is held in high regard, and his opinion is often weighed when formulating an opinion on something when you're deeply into whatever their chosen subject is. A professional game reviewer is an asshole and pretty much everyone hates him. There are certain people whose opinion I trust in the gaming world, like Jeremy Parish or the Rock Paper Shotgun guys, but normally I just consult the forums.

[dgdgagdae]

They comment about this in the Catherine review, in fact.

You mention the review of Catherine twice. Is that really what's upsetting you? And have you played the demo? That was enough to tell me that the gameplay had nothing whatsoever to do with the story, and that I wasn't going to play slideyblocks to watch some amime film.

To summarize: The Japanese are weird.

[Graham Mitchell]

You mention the review of Catherine twice. Is that really what's upsetting you? And have you played the demo? That was enough to tell me that the gameplay had nothing whatsoever to do with the story, and that I wasn't going to play slideyblocks to watch some amime film.

To summarize: The Japanese are weird.

No, this has been bugging me for years. But the Catherine article was what prompted me to write this post. It's also just an example that's fresh in my mind because I read it yesterday. I haven't played the demo yet because I preordered the game and I prefer to be surprised when it shows up.

I do agree with some of their reviews. I just saw that they gave Portal 2 a 9.5, and I think it deserved it.

Definitely, this stuff is not worth freaking out about. But I am/was curious to understand why my they consistently knock just about anything that's not a 2-D fighting game from Capcom or an FPS.

The bottom line is that you have to just digest these reviews as entertainment, I guess. Trying to understand why I disagree with them so often is basically pointless. I don't think I'll ever have an answer. And that's fine.

[Icarus Moonsight]

The bottom line is that you have to just digest these reviews as entertainment, I guess. Trying to understand why I disagree with them so often is basically pointless. I don't think I'll ever have an answer. And that's fine.

Bingo. We want information, sopping dripping meaty information. It's like the difference between technical articles and articles in People Magazine. This isn't serious, it's fluffy soft-propaganda funded primarily through media monopolies and payola. Might as well read National Inquirer.

[Rob2600]

So, we've definitely debated to death on these forums about how basically useless number-based reviews are.

To me, 10-point and 100-point rating scales are useless and juvenile. What's the difference between a 7.1 and a 7.2?

All we really need is a 3-point scale:

3 - Totally worth playing.
2 - Maybe worth playing. Rent it and see.
1 - Not worth playing at all.

[kafa111]

To me, 10-point and 100-point rating scales are useless and juvenile. What's the difference between a 7.1 and a 7.2?

All we really need is a 3-point scale:

3 - Totally worth playing.
2 - Maybe worth playing. Rent it and see.
1 - Not worth playing at all.

i would write 2 everytime. impossible to mess up

[kafa111]

wah double post

[Berserker]

I look at professional game reviews the same way I look at Ron Popeil infomercials: as chunks of paid programming. Does anyone actually base their purchases on what comes out of this small handful of institutions?

It seems to me that, by this point, we've almost completely cast aside any pretense that these serve to inform us in any real capacity. Instead, we view them now only as public measurements of games we've already purchased and played, having been informed by sources we find trustworthy, such as peer recommendation or our own intuition.

Whatever discussion occurs now focuses only on the public measurement as an object, and how close or far it comes to our own personal measurements; but the question of whether or not they are at all genuine or uncompromised is beyond discussion.

In a nutshell, we've twisted our views of these things to synchronize with the advertising industry that created them, because that's the only way their existence makes sense to us.

So my thought isn't that professional game journalism is bad - it's that it hasn't been invented yet. My hope is that at some point, the number of professional writers who are disillusioned and tired of trying to conform to this medium's boney grip reaches some sort of critical mass which might lead to its true birth.

[Icarus Moonsight]

To me, 10-point and 100-point rating scales are useless and juvenile. What's the difference between a 7.1 and a 7.2?

All we really need is a 3-point scale:

3 - Totally worth playing.
2 - Maybe worth playing. Rent it and see.
1 - Not worth playing at all.

Traffic light gaming reviews. That could serve well for a fast at-a-glance reference, at least when it comes to plow-in or full-stop. The middle ground is always the murkiest waters to navigate, and if you look, that's where a majority of the reviews range already. That's when the article itself is critical for informing a potential customer. The Metacritic aggregate review process just makes the middle-ground wider, but also accentuates the top and bottom. I've been using this myself already, so it does have some pragmatic quality.

Berserker is spot on. There is a better way, not yet realized. For most of us, at our level of involvement, we can disseminate information extremely well on our own. So the need to invest in this new way is somewhat diminished. But, as more folks come over to the dark-side that incentive will increase, and the usual sources will become that much less relevant. It will emerge when it's needed, like the Hero of Time.

[kupomogli]

I read reviews to find out what a game is like rather than whether I should purchase it or not. I do look and see how people have enjoyed it, but it's really if I find a value in it for myself before I purchase it or not.

As for the 10 point scale, it's not that it's bad, it's just that no one uses it properly. A 6/10 is supposed to be an decent game, yet the way professional review sites make it look is that 6/10 is the crap shovelware. It's rare that a game is 6/10 at all if it's at any of the professional revew sites. I think most of the games this gen should be below 6/10 or 3/5.

I think everything should really go by the five scale. It's more simple. When you have people looking at it and Infamous comes at a 3/5, it's giving the game a 6/10, where on the 10 scale people would flip out about it but on the five scale they'll be more like "oh, okay."

With the five scale. Five is near perfect or perfect. Four is very good, three is good, two needs some work, and one is crap. I don't really like the "three is average" as most games would fall under three then, and rather than looking like the game should be skipped, it looks like it should be played instead. Three being average is nice when over half the games in that specific gen don't suck.

[Gameguy]

I usually use Metacritic to average out reviews from many sources, rather than just use one source. Idealy it's best to find a reviewer who has a similar taste to yours rather than someone who's just "objective". Also, plenty of reviewers don't like certain genres and when they have to review a game from a genre they don't like the review is obviously more negative than it should be.

[Icarus Moonsight]

Just objective is boring, more so than just opinion that may or may not entertain (entertaining articles are popular for a reason, while technical articles are not popular but factors more important than a populist one), but it's a very important part because those points are relevant to anyone. For example, if a game is broken in some way. It should also be easy to tell from which angle the review is taking on an issue. Which is an existing problem. Of course it should be fun and engaging to as many as possible, I just would like that the vital wasn't sacrificed to the LCD.

[bangtango]

Biggest problem for me with Game Informer isn't their scoring system for reviewing games. You'll notice numerous people write the reviews, so obviously one person's 7 won't carry as much weight as somebody else's 6. Best you can do is figure out which of their writers you agree with the most and stick with their reviews.

With GI, I notice they leave out a lot of games. For example, I don't care if PS2 Madden 12 is the same as Madden 08 or 09 for the system. Write a damn paragraph for it and give it a damn score and don't give me some bullshit about EA didn't send you a copy to review because they probably did. But you and I know they won't even mention it.

I wish there was a legitimate magazine or a web site that had the balls to review the 5th consecutive roster update game, the $12 budget titles or the horrendous shovelware for each of the systems on the market, but nobody will do it.

[Gamereviewgod]

Because of that, it's true that the reviewers are sometimes forced to marathon games because they're given so little time to complete a review. They do aim to beat a game at least (but I can almost guarantee you that some games are written about without the credits ever being seen), which is is no sweat for a game under 10 hours, but it can be a nightmare for something like a long Japanese RPG. Because of that, I'd say the problem is the exact opposite. They're not playing the games enough.

The thing is, with games, they don't really need to develop.

Today, I sat down with Call of Juraez: The Cartel. The second I fired the gun it was trouble. It was sloppy, collision was suspect, and the guns were weak. It was 30 seconds. Four hours later, I'm taking a break for a bit, and I can assure you, for another four hours today I'll be using those same mechanics, doing the same things. I don't really need those other 10 or so hours to know it's pretty much a mess. The story is only the hole in the process now.

Reviewing games is vastly different than other media. Movies need to be a complete narrative, so do books. Games are really mechanical, selling themselves on their feel. You don't always have to play a game "enough" to know where you stand. You do because it's a job and it's required of you, but the opinion isn't going to change whether or not you beat it. I'm not going to hate Juarez anymore than I do now simply because I have a deadline.

People love to pick apart reviews, like the Catherine thing. I talk to the reviewer all the time. He was prepared for a backlash. Gamers are simply rabid when it comes to this stuff. They see a "7" and freakout. Alice? Yeah, Alice was rough. Levels dragged on for hours with no end in sight. One sitting, two sittings, doesn't matter. Maybe that didn't bother you. I was invested in the world but there was too much of it. A 6.5 is more than fair in my eyes, maybe not in yours. 'Tis the nature of reviews.

Keep in mind that in a great game, something truly special, you're never tired of it. The design is such that it takes an inherently repetitive medium and makes it something spectacular. Most games today are a lot of filler to fill some requisite length, and they're going to be called on it. The great ones stand out and you don't want that marathon session to end.