Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

[Column] General: Reviews and the Bell Curve

SBFordSBFord Former Associate EditorMember LegendaryPosts: 33,129

Reviews are always a sticky wicket because of the subjectiveness of the process itself. In today's Devil's Advocate, we take a look at reviews from all angles. Read on and then leave your thoughts in the comments. 

I’m mentioning this now because I’ve spent the past month reading the four-part review of Darkfall: Unholy Wars by George Dimmock and observing the comments of people on MMORPG.com and on the Darkfall forums. It seems that, while new games have sprung up since 2010, the talk about spending adequate time to play an MMO for review still occurs.

Read more of Victor Barreiro Jr.'s The Devil's Advocate: Reviews and the Bell Curve.

image


¯\_(ツ)_/¯ 


«1

Comments

  • CheookCheook Member Posts: 16

    As a reviewer you have to face fanboys these days. People just suck in the anonymity of the internet.

    That said, personally I appreciate two types of reviews:

    1. The emotional one of a true gamer, who explains why he enjoys the game in an informative manner. It is okay if it is a little biased.

    2. The down-to-earth one of a professional writer, who can explain the basics of what to expect in the game. And yes, this includes a list of features like sound, bugs, gameplay, ...

    I think the latter one can be achived in 20 hours. If you have read some reviews of a person, with some intelligence you can derive if the reviewed game is good for you. And that is the most important and completely subjective.

  • maplestonemaplestone Member UncommonPosts: 3,099

    We're a large and diverse audience with wildly different expectations and reactions to large and complex games.  I'm not interested in which MMO fits the reviewer's tastes, I'm interested in which MMO is the best fit for my personal tastes (or, if I was part of a wandering guild, our tastes).  No one reviewer is likely to see a game with the same eyes I will look at it with.

  • LivnthedreamLivnthedream Member Posts: 555
    What you should be doing with ALL reviews is find someone who likes the same things you do. They will give you a much better impression of if you will like a thing or not. Its much the same for movies or music.
  • itchmonitchmon Member RarePosts: 1,999

    i'ma just say i think i deserve "spotlight poster" now that i've been quoted :)

     

    point taken as well Barriero :)

    RIP Ribbitribbitt you are missed, kid.

    Currently Playing EVE, ESO

    Every gun that is made, every warship launched, every rocket fired signifies, in the final sense, a theft from those who hunger and are not fed, those who are cold and not clothed.

    Dwight D Eisenhower

    My optimism wears heavy boots and is loud.

    Henry Rollins

  • BahamutKaiserBahamutKaiser Member UncommonPosts: 314
    Makes perfect sense, reviews should not consider dedicated players because they are not in need of an explanation to a game their fully experienced with. Furthermore, the bias of a dedicated MMO player makes their judgement unreliable, as they have their own personal preferences, and have naturally gone about the task of validating their decision.

    If you want a review or analysis of a game based on dedicated players, you read player reviews. If you want the review of a seasoned and broadly experienced reviewer, you read a review article.

    This is just a simple misunderstanding about what a review should provide.

    Before you criticize someone, walk a mile in their shoes.
    That way, if they get angry, they'll be a mile away... and barefoot.

  • victorbjrvictorbjr Member UncommonPosts: 212
    Originally posted by itchmon

    i'ma just say i think i deserve "spotlight poster" now that i've been quoted :)

     

    point taken as well Barriero :)

     

    Hi Itchmon!

     

    FYI, Barreiro is the proper spelling. :) But thank you! :D

    A writer and gamer from the Philippines. Loves his mom dearly. :)

    Can also be found on http://www.gamesandgeekery.com

  • sketocafesketocafe Member UncommonPosts: 950
    I've not read the DFUW review, and I just kind of skimmed this article but, the reviewer didn't act like he knew everything about DF combat and group play, did he? It's one thing if you're talking about your experiences gathering and crafting and another if that's what you did but then you deliver sweeping statements about the areas of the game which you didn't explore. It's foolish to expect a reviewer so be physically able to get into everything and still deliver a timely review and it's fine when they don't, so long as they're honest about where they spent their time in the game.
  • MkilbrideMkilbride Member UncommonPosts: 643
     
     

    Darkfall fanboys are some of the worst I've seen. One other reviewer gave a negative video review. It exploded on the forums. He played 20 hours, and give it a fair and honest review.

     

    However, it was not positive about the game. He got cries of of "You have to play at least 100 hours before you can review the game!'

     

    Really now? 100 hours? If the game is not FUN before you've played 100 hours, it is FLAWED.

     

    What makes them so bad is though, after the reviewer took the video down, because he was getting so much heat about it...they found his website, his email, and even managed to discover his phone number, and if you read the forum topic "Let's ruin this guys career"

     

    I've never seen fans so...toxic.

     

    Help get Camelot Unchained made, a old-school MMORPG, with no hand holding!

    http://www.kickstarter.com/projects/13861848/camelot-unchained

  • itchmonitchmon Member RarePosts: 1,999
    Originally posted by Mkilbride
     
     

    Darkfall fanboys are some of the worst I've seen. One other reviewer gave a negative video review. It exploded on the forums. He played 20 hours, and give it a fair and honest review.

     

    However, it was not positive about the game. He got cries of of "You have to play at least 100 hours before you can review the game!'

     

    Really now? 100 hours? If the game is not FUN before you've played 100 hours, it is FLAWED.

     

    What makes them so bad is though, after the reviewer took the video down, because he was getting so much heat about it...they found his website, his email, and even managed to discover his phone number, and if you read the forum topic "Let's ruin this guys career"

     

    I've never seen fans so...toxic.

     

    were you around for the beta days of vanguard? but ya, i agree DF fans can be mad harsh.  overall its a very polarizing game.  I dont consider myself a real fanboy of the game, though i feel its got potential to be very good if it makes  (a) a few smart changes and (b) makes good use of its budget to prioritize what needs work first, etc.

     

    i think in the cases of both vanguard and DF the fanbase is so rabid because the style of game in question was kind of "a reversal" of the prevalent gaming norms.  VG was more EQlike when everyone was switching to be more warcraft-like; DF is open pvp and harsh when games tend (or, were tending)towards arena "sports" style pvp and less penalizing gameplay.

     

    (i said "were trending" because i wanted to give a nod to the other very promising sandies coming out soon.)

    RIP Ribbitribbitt you are missed, kid.

    Currently Playing EVE, ESO

    Every gun that is made, every warship launched, every rocket fired signifies, in the final sense, a theft from those who hunger and are not fed, those who are cold and not clothed.

    Dwight D Eisenhower

    My optimism wears heavy boots and is loud.

    Henry Rollins

  • MkilbrideMkilbride Member UncommonPosts: 643

    I wanted to play Darkfall. I liked some of the ideas.  So I took to their forums to tide me over until I decided if I wanted to or not.

     

    Community is very important to me in a MMO.

     

    When 30% of your topics are hate threads on clans or a person, the other 30% are people complaining about the game and how broken it is, the other 30% complaining about how "reviewers don't know how to review, Darkfall is amazing!'

     

    It would seem only 10% are actually liking the game, and these 10% are shrinking. Do you know the first response they have is, if you question some feature of the game? "Go back to WoW, you fag"

     

    That's probably the most common phrase on the DFUW forums and it's not just by one forum member. About the beta days of Vanguard, no I wasn't.

     

    Just these people are going to ruin a young mans career over a review he did of their game. He's a budding journalist, his video wouldn't get much exposure, but regardless because they seem to love their game, no matter what happens...it's just so damn toxic.

     

    I wanted to play Darkfall. I like some videos, I see even. But the community, give this powerful sandbox tool of a game, decide to only use it to stroke their own egos. I've played tons of Sandboxes where the communities were nice and helpful, welcoming, with some bad sorts, but that's sandbox.

     

    I've not seen -any- good sorts on the DFUW forums. Maybe they don't post, I don't know. You say you like another MMO? They'll trash you about having no taste, that said MMO requires no skill, has no depth, ect.

    I've seen fanboys, I've seen hype, but I've never seen people like this. I am a huge fanboy of Camelot Unchained. Lotsa negative press about it. Do I go around barking like a wounded dog? No, I either ignore it, or post about how they are  reasonable doubts, given what we know. I am willing to admit, that CU could suck. It could happen. Anything is possible..

     

    DFUW Fanboys on the other hand, will accept that AV can have their servers down for over a week and not make an announcement about it. They will accept that while the game was delayed 6 months to put in content they claimed was ready, but "needed tweaking", isn't in the game yet, 7 months later, and their roadmap for the rest of the year(which hasn't been kept, either) doesn't list this content. 

     

    If CU sucks- CU sucks, I will admit it and move on, or deal with it. I won't lie and tell people that it's great when it's shit.  These people are...devoted in the wrong way, the "drink the punch" kind of way.

    Help get Camelot Unchained made, a old-school MMORPG, with no hand holding!

    http://www.kickstarter.com/projects/13861848/camelot-unchained

  • AnirethAnireth Member UncommonPosts: 940

    I think, if the score of a certain reviewer/magazine significantly differs from the score of the general public on a regular basis (take metacritic scores for example, or IMDB for movies), there is something wrong with the way the reviews are done. This is not limited to video games at all, it happens everywhere, but that doesn't make it right.

    Of course you can limit yourself to review only a detail, like, technical aspects, or the story writing for games, where if it's actually an in-depth analysis, bigger discrepancies can be expected and accepted, but if you do a 20 hour game review, and the general gaming public has a different opinion..well, maybe you should spent more time, as you obviously can't seem to find the things that make or break the game in the eyes of those that you review the game for. Or you should think about how you spent your time. Crafting all day long, when the game also features extensive PVP? Maybe they have a sensational crafting system, but PVP is horrendously broken? Especially if the game is advertised as PVP game, you really ought to take a look there.

    Some of this can be avoided by writing an "emotional" review, if you clearly state that you review this as a player and you always liked open-world PVP with full loot in UO and this game finally gives you the kick, the sensation of danger that is missing in many other games back, sure, give it a positive score even though most people will probably not like this feature.

    Still, if the game crashes every now and then, half the items are still not translated a week before release etc., it definitively has to impact the score, shiny new graphics or dozens of innovative  and fun raids might be nice, but a 90%+ game can't be lacking in any department.

    I'll wait to the day's end when the moon is high
    And then I'll rise with the tide with a lust for life, I'll
    Amass an army, and we'll harness a horde
    And then we'll limp across the land until we stand at the shore

  • WizardryWizardry Member LegendaryPosts: 19,332

    The ONLY game that ever took me longer than one day to fully understand it was FFXI and it took me months.I would say that even with the internet and the more helpful design it still takes players a long time to fully grasp the game.

    Eve ,well imo not tougher to fully grasp than FFXI but probably next in line.The base core concepts are easy though,just takes time to wait for checked boxes to finish their time sink before you can really grasp it,so more a time sink that a learning curve.Eve's whole learning process is PVP based,not like every other game is not the same.You can take any pro at any pvp game,they will eat up noobs all the time.

    Whole point is every game i have played takes all but an hour "NW" to maybe a few hours to fully grasp the game,aside from Eve and FFXi,FFXI being the only PVE game that takes time.

    Never forget 3 mile Island and never trust a government official or company spokesman.

  • TerranahTerranah Member UncommonPosts: 3,575

    I've been playing video games since the 1980's.  Different genres require different amounts of time to get a feel for the mechanics of a game, but it doesn't take very long at all to develop an impression of whether or not I will like it.

  • MkilbrideMkilbride Member UncommonPosts: 643

    Lol, so true.

     

    Go to the Darkfall Forums "So I've played this game for a month, and these are my thoughts..."

     

    Response

     

    "ONE MONTH ISN'T ENOUGH TO UNDERSTAND THE GAME!"

     

    Then the game is flawed.

    Help get Camelot Unchained made, a old-school MMORPG, with no hand holding!

    http://www.kickstarter.com/projects/13861848/camelot-unchained

  • OzmodanOzmodan Member EpicPosts: 9,726

    Really excellent article.  What the hardcore gamers do not understand, it takes hundred of hours to actually grasp the complexity of a MMO.  No reviewer has the time to do that.  Besides the idea of a review is to get a perspective of how a person who knows little of the game initially experiences the game.

    Where the reviewer failed in the Darkfall review was that, games like Darkfall are not intended to be played from the solo perspective.  You must find friends to enjoy the game as it is designed.  Any game like this, that should be the first thing a reviewer does.  I am going to assume that was not pointed out to the reviewer before he started.  It should have been.

  • OzmodanOzmodan Member EpicPosts: 9,726
    Originally posted by Mkilbride

    Lol, so true.

    Go to the Darkfall Forums "So I've played this game for a month, and these are my thoughts..."

    Response

    "ONE MONTH ISN'T ENOUGH TO UNDERSTAND THE GAME!"

    Then the game is flawed.

    Typical response from the "I want it now" generation.  I beg to differ.  A game is flawed if it does not take at least several months to understand, if not more, IMO.  Games that avoid such complexity soon find themselves ghost towns.  

  • AntiquatedAntiquated Member RarePosts: 1,415
    Originally posted by Mkilbride
     
     

    Darkfall fanboys are some of the worst I've seen.

    Without endorsing or rejecting...

    Can you imagine being the reviewer assigned to that title?

  • MkilbrideMkilbride Member UncommonPosts: 643
     
     
    Originally posted by Ozmodan
    Originally posted by Mkilbride

    Lol, so true.

    Go to the Darkfall Forums "So I've played this game for a month, and these are my thoughts..."

    Response

    "ONE MONTH ISN'T ENOUGH TO UNDERSTAND THE GAME!"

    Then the game is flawed.

    Typical response from the "I want it now" generation.  I beg to differ.  A game is flawed if it does not take at least several months to understand, if not more, IMO.  Games that avoid such complexity soon find themselves ghost towns.  

    No, they say you cannot enjoy the game in a month.

     

    That no reviewer should do a review unless he does 100+ hours in game.

     

    It is flawed, if you do not enjoy the game within your first 100 hours. That's flawed. Has nothing to do with ANY generation.

     

    --------

     

    To the poster above me. DFUW seems to be doing it's best to avoid having anyone review it. You review it positively? They get mad you didn't give it a 10.

     

    You review it negatively? THE GAME IS TO COMPLEX FOR YOU TO UNDERSTAND! You ONLY SPENT A MONTH!

     

    There was one reviewer who spent a month playing the game, had 30k PP or so, so he doesn't play ALL the time, but he had fought in some Sieges, been in a Clan, ect.  His impressions were mixed, around a 6.5, which is reasonable.

     

    Every response to his review was "Go back to WoW Fag" "You only played for a month? That's not long enough!"  Ect, so on and so forth for 2-3 pages.

     

    They are devoted to this game the way few gamers are. They will defend it, even when knowing things are wrong. They will say there is not a hacker problem, even when there are 20 YouTube videos showing 20 different hackers, doing various things, including Godmode, saying that "All MMO's have hackers". Not really. Exploiters, sure, but not some guy running around with fucking Godmode. They will ignore all this, and say that all "MMO launches are like this"

     

    I've never seen a MMO launch even close to this bad, not even WoW.  The game soft-launched 6 months ago. The developers have not kept one single promise they've made, and yet they will defend them to the death.

     

    It's crazy.

     

    Help get Camelot Unchained made, a old-school MMORPG, with no hand holding!

    http://www.kickstarter.com/projects/13861848/camelot-unchained

  • MimzelMimzel Member UncommonPosts: 375

    I find this article quite arrogant. Take this for example:

    More to the point, a review by one person is not meant for consumption by all gamer types.

     

    Uhm, ok? So a review is meant mostly for those who agree with it? I'm sure that's not what you are trying to say, but it sure does sound like it. You speak about methods for reviewing games, and say it isnt relevant whether or not the review plays the game the way most gamers would. You arent really saying anything on the matter that makes any sense. Why is it irrelevant?

    Reviewers on this site mostly hide behind their veil of un-professionalism. "I am just a person - just like you, so take this with a grain of salt". Excuse me, but arent you abandoning the very legitimacy you rely upon to do your job? Yes, we are all human beings, I'm not expecting you to have six eyes and orange tentacles. If you write a review, I expect you to have more qualifications than me to do so. You cant say "i'm just like you" and keep your legitimacy.

    And lastly, you talk about what a review is. Nay, it feels more like you are trying to give us a definition which excludes the consumers (us) expectations. Of course a review could be just like you say, but that makes it pretty irrelevant for many consumers. We dont exist for your pleasure and income. Your job exists because of our need for it. You might have meant it slightly less arrogant, but it sure does come off like you own the power to define REVIEWING.

     

  • NephelaiNephelai Member UncommonPosts: 185

    Most reviewers aren't good gamers and so don't have the experience to give a comprehensive review. Its like saying a hobbiest can tell the same story as a craftsman and that's just not true. That's ok because most gamers are average and appreciate an average review.

    For example if you've never played at the highest level in a fluid game you probably still think animation lock games are passable. Well they are for the average gamer who cant use the fluidity anyway so it doesn't matter. There's no right or wrong just understanding the context for example when I see a review done by Pocket it makes me cry. I could never rely on her review for my impressions of a game but many Farmville people could just fine.

    Its not about getting your nose out of joint because people didn't like your review its about understanding your review is only going to appeal to your own demographic and in the vast majority of reviewers that is average.

     

     
  • jbombardjbombard Member UncommonPosts: 598

    You wouldn't do a book review unless you have read the whole book.  The same should be true for games.  Just like the crap the news industry has become, everyone wants to be first so they just throw their crap out there without doing their homework.  If you have only played part of the games do a "first impressions" not a review.

     

    I personally don't expect a review to match my expectations.  I generally read through it and see if their comments apply to what I am looking for in a game.  But then again I don't read one news story and decide that is what happened, I read several and try and find the truth, and I take a similar approach with games.

     

    What I would like if every review would start with a short explanation of what kind of games the reviewer likes, a short list of favorite games of all time, what their expectations were before playing, and an honest appraisal of whether or not those expectations possibly colored their impressions of the game.  That would help tremendously when trying to figure out whether or not the reviewers opinions would be applicable to myself. 

  • maplestonemaplestone Member UncommonPosts: 3,099
    Originally posted by jbombard

    You wouldn't do a book review unless you have read the whole book.  The same should be true for games.

    There's no such thing as "the whole game" for an MMO (at least not for anything I personally would call an MMO).  There's a subjective component of what you choose to do with it beyond simply the story arc at hand.

    But when it comes to the basic look-and-feel, controls and advancement style, the impressions of the first few hours (once you've adjusted to any unique elements of the UI) are unlikely to be wildly different from the impressions of a few months later.  The only question mark is whether there is a radical shift in gameplay at some point (if you want to write a review of the raids, you can't do that at level 1 in most games),

     

  • PlageronPlageron Member Posts: 109

     

     

    A good review is not based on what someone else says....but based on what the reviewer knows to be the truth...and that is what people want.

     

    So this devils advocate article is interesting....if not some form of hidden attempt to protect some reviewers from the angry mob of players who notice when the reviews are not based on factual information.

    The moral is: In this day and age....you have to be careful because you never know who is reading what you write.

  • mmobootsymmobootsy Member Posts: 48

    It's an interesting discussion, and reminds me a lot of the discussion that surrounds board game reviews. I am an avid board gamer, but I wouldn't play a game I don't like more than 1 or 2 times. So should I review it? Should I try it with different (numbers of) people first? How often is enough? 5 times? 10 times? If the latter, could it be anything other than a good to great game?

     

    Books and films are not that good an analogy, because there is no variety in content dependant on player input (as there is in board and video games), and (therefore) cost way less time to fully consume and therefore be able to review.

     

    Now MMOs are among the most complicated and content rich forms of entertainment known to man. And I am a very impatient video gamer and an avid game hopper. There are few games I have played to completion. Can you ever play an MMO to completion? Doesn't 5 minutes of play already lead to a first impression, which is difficult to get rid of if you don't play a considerably longer time? And that's the question: Why would I if I don't like the game? How long do I have to wait before it becomes good? 20 hours? 100 hours?

     

    Yes, it is a problem that many people have the "everything now" mentality, and that doesn't fit at all with playing an MMO. Or playing a long and complex board game. So yes, an MMO player, and especially a reviewer, should be held to a higher standard. Some crucial and fundamental aspects and elements of the game should be experienced to provide decent information (opinionated or not) in a review. I've dismissed MMOs just based on the graphics, or the character creation process, or the style of the HUD. I certainly would disagree with a reviewer doing that! Yet, if he has dipped further in the game, and makes perfectly clear from the start of his review what and how far he has played, then I am fine with taking the review into account when searching for an MMO.

     

    Be clear, reviewer, about your approach of the game for the review, and as an intelligent and experienced gamer I can take it or leave it as I please.

    image
  • MimzelMimzel Member UncommonPosts: 375
    Actually, it would be a good idea if each review started out with a short methodology chapter (i.e. this is how I went about reviewing this game). It would place the score into a meaningful context.
Sign In or Register to comment.