I usually like your articles and reviews. They are often interesting and sometimes quite insightful. I especially like the pieces on how MMOs could be better and worse.
The one thing I would say is that you really need to proofread your work.
"Gamers will no longer buy the argument that every MMO requires a subscription fee to offset server and bandwidth costs. It's not true you know it, and they know it." Jeff Strain, co-founder of ArenaNet, 2007
I have seen some good suggestions for numbering systems here, but these will always be problematical.
I want to "rate the reviewer" instead :-). I want to know how important PvP, endgame-raiding, graphics etc. is to that reviewer first. If my preferences more or less match the reviewers, THEN numbers become usefull.
So, a new game needs a long, good read regarding the various areas of a MMORPG as neutral as possible. Over time other reviewers can chime in with their own numbers, not needing a full review. It would tell me a lot, if I saw a diehard PvP'er saying "naaw" to a game, if thats my preference too.
The problem here is, that you ideally have reviewers with very different preferences.
It irks me in your reviews when the reviewers mention each other by name and say things like "At first...but...convinced". 2 or 3 people reviewing the same game would be great, especially if they were blind to each other. If I came here for three different perspectives on a game, I want 3. I don't want a jury that had to come to a unanimous decision."
"Will the defendant, Greg Street, please rise."
And of course, for issues like PVP, personal perspective is so imporant. If your class is OP and steamrolling the competiion, I don't need you to convince me that PVP is fine. I need to psot my experience.
Spec'ing properly is a gateway drug. 12 Million People have been meter spammed in heroics.
I want to "rate the reviewer" instead :-). I want to know how important PvP, endgame-raiding, graphics etc. is to that reviewer first. If my preferences more or less match the reviewers, THEN numbers become usefull.
...
I had an idea about this some time ago... sort of linked to what Metacritic does.
How it would work is that you would compare Critic (site) Scores to User Scores and assign some kind of rating based on how close that critic was to the user score.
Of course you would need a reasonable number of user votes to make it valid (other wise a review could be made to look 'bad' because of one disgruntled player?)
And in addition to doing it by site - you could do it by the actual reviewer too.
I'd like reviews to have scores for various categories as well as an overall score. That said, the overall score should be decoupled from the categories and be based on the reviewer's opinion of the game as a complete package. Let us decide to weight the overall score based on our own perception of the value of each category as well as the text of the review.
Also, for crying out loud, if you're going to use scores out of ten, use the full range properly. An average game should be a 5 or 6, not a 7 or 8. Reviewers who treat scores like they're out of five then add five points are being dishonest through carelessness. More integrity than that would be appreciated.
While we're at it, no decimal points for anything but user review aggregation (and even that's only a maybe). Whole numbers only, because the more specific you get with something as subjective as a review score, the greater the disservice to the reader's intellect.
Finally, never, ever write a review with the type of person who wants to see a score and move on in mind. Scores should be a place for general summary to cement the content of a review. Catering it to people who don't care about what's actually said is useless and helps no one.
Hopefully you read through all the responses and see this!
I'm not too concerned about the scoring system, as really who can rely upon the opinion of others. The thing I would really like to see is a list of major features prominently displayed.
eg: What type of PvP system? Faction based, consentual, open world. Advancement system - class / skill based / other.. single server / shards.. interesting or new features that makes it stand out from other games..
Id really like to not have to read through a whole review to try and siphon this information out (if its even there). I, like other players, are looking for specific features in their games.. and no matter how well done, some types of mmo's were just not interested in.
Your preview followed by a full review works fine for me, I have understood it and never doubted it. Your previews have always expressed and re-expressed that it is not a complete review and you never give the final rating in the preview.
If someone does not understand the method you have used for years now then they simply cant read a review properly and just read what they want to. If you are dealing with people like this then it will not atter how you review it, they will still complain.
Frankly I would just be happy with the letter grading system. Having a reviewer that likes that style of MMO (FFA PvP, themepark PvE, whatever) and one that generally doesn't favor that style each doing an initial review would be nice. That way you are sure to get the pros of the game from the one who likes that style, and the cons from the one who doesn't; that way those who take the time to read both can make the best informed decision possible. Still, if I could only choose one change it would be the letter grading system, or at least go to a pure 1-5 system; seeing a 7 score for a game when it is all actuality a 2 score is confusing to say the least.
"Oh my, how horrible, someone is criticizing a MMO. Oh yeah, that is what a forum is about, looking at both sides. You rather have to be critical of anything in this genre as of late because the track record of these major studios has just been appalling." -Ozmodan
I like the idea of breaking down reviews into relative categories that appeal to different gaming motivations. I like to see how a game breaks down in terms of graphics and sound, community, soloability, grouping, end-gaming, character development, innovation etc.
Reviews are by their very nature subjective, but a more organized approach may provide more literal insight with greater relevance to a broader audience.
Del Cabon A US Army ('Just Cause') Vet and MMORPG Native formerly of Trinsic, Norath and Dereth. Currently playing LOTRO.
First off good for MMORPG to admit that the current system leaves much to be desired.
Secondly (which I am sure has been said by page 7) An A+ downto C- still gives 9 increments without quite as much stigma attached to low scores. Use all the increments! Also scoring different categories has some merit (though not too many categories). Also try and seperate heavily subjective elements to those that are reasonably quantifiable, one might hate the art direction but still aknowledge that the graphical quaities are good for example. Gameplay is an element that would survive some sub categories.
An earlier poster said maybe 3 reviews pre-release previews take them as you will, lest rigour is acceptable there. An 'early look' (which still should be given enough hours to get past the introductory game and into the meat and potatoes). This would be the 'main review' and should at least try and talk about most of the game elements. You could then have a longer turn review that deals with on later gameplay. Of course you could not hope to review so many games at that level.
You need some sort of 'guidelines' for reviewers so eveyone is singing off the same song sheet. Of course you could have guest contributors that are operating outside this but that should be made clear.
I'd like to see a tendency to less reviews that are more in depth with greater substance. Some currently feel like they duplicate the publishers blurb with a bit of opinion thrown in.
Problem with any review, it is subject to the author's likes and dislikes. You just cannot surmount that obstacle. What tickles a reviewers fancy, might be poison to you. No matter how you look at it, unless of course the game is a buggy morass, about the only way to determine whether you like a game is to try it yourself.
Review vs. preview: I think your previews are quite good. But to set them apart from the reviews you will need something "extra" on the review side of things, something that keeps readers clicking on that link even if the game is out for a month or more.
So, you have to improve the accuracy of your ratings, especially in reviews. You can do that by having more than one rater. It would cost more, but if you took 3 or more reviewers, each playing for at least 7-10 days for a substantial amount of time, you may sum up their scores to get a more accurate review. Of course it would take more time and manpower to do this, but you could split up the article among the raters. There are some ways to improve interrater-accordance, mail me if you want to know more.
Scoring system: nothing wrong with 1-10. Marginal differences will be expressed no matter if they're called 5.6 and 5.7 ot C and C+. Its just another level of differentiation. The real problem here is one of Information loss. If you aggregate 5 or 6 scores (graphics, sound, gameplay etc.), you will not be able to see if it was sound or gameplay that are good or bad. So: Either you give the game a general score, which may be based on factors irrelevant to some gamers, or you split the whole thing up into facets and generate different listings (Top ten in gameplay, top ten in sound, top ten in Graphics etc.).
Maybe it would be kind of an alternative to generate such lists and to present them in a row with the list for general score.
I could care less about the number or letter system personally, I normally dont put much stock into it either way.
I would rather you give us a review of each system in the game and be much more honest about the game other than its fun and I had fun. Im not trying to say you guys arent honest but I think you do gloss over some bad elements of a game, I believe earthrise was one example. Every game is fun at first, if its playable at all, I want to know if this game is worth my time, I want to know if its just another gear grind like wow that I wont have time for, I want to know if im gonna be doing the same thing over and over for 50 levels, honestly its why I dont play rift anymore, or wow and probably wont get swtor, im sick of picking up quests and going and killing x monsters and then repeating. I want your reviews to tell me that before hand. I want your reviews to detail crafting, ive seen it so much, a writer will say im not that into crafting but its there for those that want it. I want the writer to actually try the crafting and let us know what it is about. I also dont really care for reviews the first day, a launch preview is fine, but I think a review of the game should come out 30 to 60 days after the game launches to give the writer some time to play the game and try the systems out.
Well this is probably a dumb idea, and my suggested review format is not easily aggregated with many web sites opinions and reviewing systems. However having bad ideas seldom stop us posters from posting /lol. How about a "review" with a structure like this:
Part 1. Description of game, general description of Requirements, special needs and maybe innovation.
Part2. Summation of a reviewers actual gameplay and pertinant comments.
Part3. Your famous list system. 10 (less if you cannot find 10) things the game does right and that would compel you to actually play the game ..
Part4. 10 (or less if there are not 10) things the game does wrong that might turn you off from playing the game.
This would all be subjective (as all reviews are by nature), but might give us readers more information from the reviewer and thusly a better understanding of what the reviewer based his "judgement" on. That way we could kind of review the reviewer as well (which we do anyway now, but with no standard format of reviews). We are prone to dismiss a review if the reviewer has a "bad pen", (like bad hair) day.
I don't think you should change the 1 to 10 numerical system because you 'll be "incompatible" as you mentioned, we are used to it from games and therefore (for some reason) I still replace a +1.1 str ring with a +1.2 one.
As for the 1-5 it's a really a good rating for unplayable mmorgs due to various of reasons, from a bad client-network perfomance to a bad interface implementation.
Since you have to suggest (or not) a product to us from its launch day just make sure to take just the facts and features of what this game has to offer including unique ones (if any) and leave unmeasurable features (opinions) like gameplay, potential and immersion for a later time.
You can categorize and break a game into its smallest components using a fact and features List which will be actually a union of all facts and features that most mmorpgs have, how many of them this game implements, in what way and then let us decide if the game can satisfy our expectations.
Some facts or features may override other ones in a way that if certain fact or features is absent or badly implemented the total rating of the game should be less than average.
All of these details combined with a lengthy gameplay video that shows them off can even save you the trouble of rating the game yourselves and help us decide better imo.
I know it sounds like "we are the robots" review system but i prefer this than any other way since i don't trust reviews and i don't like wasting money every time a "new generation" mmorpg is released.
A few suggestions. One , go to a letter grade system. At least you can't fudge that , no -/+ ltter grading. If a game falls within A then it can be trusted if it's a C then at least we know that the game runs as passing with issues, etc..
Secondly find a way to generate income other then advertising from game producers. No matter how impartial you may try to be , accepting money from a Dev company to advertise calls into question objectivity. If your site is good enough , they want to advertise regardless.
This incidently brings me to the whole intergrity of the process issue. To promote your site you need inside information or early access to build traffic so you can generate more money from advertisers. The devs need you and give you access to advertises their games and build hype for it. Kinda of a viscious circle? As gaming journalists who for a living basically exsist off the good will of the devs. You depend on that access, it seems to heavily. You're afraid a bad review shuts the doors to future products. You've set up a system that basically is a wink and nod system that, " If I don't say anything overtly bad about a game , they'll grant me access to the next porject . " Here's a novel approach , you have access to Beta the same as all of us , sometimes better. Use that as the start point , because everything before that is controlled hype. You build credibility draw more traffic and thereby more advertising dollars. It all starts with basic ethics .
How about two or three people review a game simultaneously and then sit down and compare notes , what they like and didn't etc.. Oh, and please run through the whole game , not necessarily all the classes , and do the crafting , diplo etc.. to the end. Yes it timely , but it'll lead to more credibility and guess what that leads to ?
Gaming websites are in a position where we have to try to get a review of a game out the door as close to launch as possible because that’s what’s demanded of us so that you all can make a more informed choice as to where you put your money.
I'd add also the pressure to be first with reviews before other review sites do, but anyway, I'd have a two-star system even though it will never happen:
Two stars=outstanding. Not to be missed.
One star=worthwhile if you haven't got anything else to play.
"So we’ll start at the top. The reason that games are reviewed too early in many cases is simple: Gaming websites are in a position where we have to try to get a review of a game out the door as close to launch as possible because that’s what’s demanded of us so that you all can make a more informed choice as to where you put your money."
Perhaps start off with an honest assessment of why you do an early review. It isnt because you want to tell the consumer where to put their money. It's because you want to be the first to market and have a claim on the "first review" of a game. You want to claim viewership as much as anything else. Promoting your web-site is the ultimate goal of, well, virtually any product out there.
This is a little late, but as many previous posters have mentioned the biggest problem with reviews is they rely on the reviewer's opinion for at least part of the article. Adding too many increments to a rating system over complicates it as well. A simple, tried and true, 1-5 works for me. Or just A-F no ".5" or + or -. A good review is going to consist of what the game offers followed by what works and what doesn't. If I review a game no one cares whether I hate elves they just want to no if the game in general works or not and whether it's worth playing.
A pros and cons section works if you include what's in the cash shop for F2P games.
On a personal note; DUMP this idiot RipperX. Who is he that I should care about his first impressions? Is he your main reviewer? Then I don't care about his first impressions. He comes off like a very late to the party version of MMOHut's Omer/Remotay (whatever his name is...) At least with Remotay he's reviewing the game along with an initial look at it. And though those reviews have their flaws too they stand leagues more entertaining and informative than listening to Ripper talk about "pooping his pants" in the LoTRO video.
I get better reviews on a game reading the Survivor Guy articles.
___________~____________ Man is least himself when he talks in his own person. Give him a mask, and he will tell you the truth. - Oscar Wilde
I think the Letter Grade method is much more informative of what it would be to the current overall generation of games. Rather than a number scale that seems to represent 10 being the best ever. but 5 years from now hardware will be 5 times better and overall game designs will have changed with that. Which in turn would have made that old perfect game not so perfect anymore. Not only that we all have different opinions in what we like, thus numbers just seem completely arbitrary. Where as letters are a better representation of our opinions.
As far as Previews/Reviews go, I think that there should be an ongoing news blog for games that the journalist actually play or have played to give each person's insite on the game. Kind of like Movie reviews... IE. the "Movies" app on the Android Market that connects with the same facebook app. You have a section of journalist reviews and then a section of fan reviews. and the overall scores of each are accumulated to the overall Score of "critics" and "movie goers".
So instead of just using 1 journalist, use all that actually play the game for a time more than 1 month. And those that actually actively play could have a running blog that could explain the different happenings within the game, and report news/updates as well.
You could have a special section for each game for the opening month of the game for it's initial release review, then have the overall review at least a month or 2 after release for a more substantial "state of the game".
I tend to judge my games based on their "worth", compared to the price. You could try something like that as your "rating" metric. Game is worth: 1) Box price plus free trial; 2) One month sub; 3) Multi-month sub; 4) Yearly sub; 5) Lifetime sub.
If you tell someone an mmo's not much better than a free trial, they get the point much better than 6.6. Tell them you'd lifetime sub to it, and they'll know what that means too.
Might be weird, and you'd have to adjust for F2P games, but who knows. Just a thought
I found a thought-provoking article about game reviews in general written by a gaming journalist at this link: http://nohighscores.com/node/508
I have no idea whether or not this guy's experience is typical, but it was an interesting read. As for the reviews here at MMORPG.COM, I'm old school. I always found it more informative when various categories were rated, and then used to calculate an overall score.
If a game got a low overall score, I'd always check which categories seemed to hurt it the most. If it was a category I didn't really care about, I'd still give the game a shot.
The categories also seem to give the review more objectivity imo. I know when I'm involved in hiring people, we've used a grid to help us score the person's performance in the interview. I find this helpful. I may get a really good vibe from someone, but when I look at my grid, I see that they have no idea how to do the job. I'm human, and without some distance and some objective criteria, my feelings definitely influence my assessments, whether I want them to or not.
I will give this more thought, but one quick notion...
If there was a quick and easy to to access player reviews next to the staff's review I would find this truly more useful then any other upgrade or change.
While it is true most people in the world are fools, when intelligent I find user reviews seek to be broader and draw more comparisons I understand and also pull less punches.
There is a political correctness (for lack of a better term) that goes with staff reviews. ( I think this is why people suspect corruption.) Therefore the reviews often feel stiff. Often it may feel like a staff review is "safe" Or the staffer may not see a need to compare somthing to other games so as not to offend or start an argument etc.
That being said if you removed user reviews that are not meant to be informative ( likely half or more) so as to not seem like another forum that would help me alot. thanks!
OH and PS, I would love to help review for MMORPG.COM or contribute. Just saying I am willing
Thank you though for this thread. I feel your pain as it seems you cannot do anything like what you are doing and not suffer from an onslaught of "lessers".
Comments
I usually like your articles and reviews. They are often interesting and sometimes quite insightful. I especially like the pieces on how MMOs could be better and worse.
The one thing I would say is that you really need to proofread your work.
"Gamers will no longer buy the argument that every MMO requires a subscription fee to offset server and bandwidth costs. It's not true you know it, and they know it." Jeff Strain, co-founder of ArenaNet, 2007
WTF? No subscription fee?
I have seen some good suggestions for numbering systems here, but these will always be problematical.
I want to "rate the reviewer" instead :-). I want to know how important PvP, endgame-raiding, graphics etc. is to that reviewer first. If my preferences more or less match the reviewers, THEN numbers become usefull.
So, a new game needs a long, good read regarding the various areas of a MMORPG as neutral as possible. Over time other reviewers can chime in with their own numbers, not needing a full review. It would tell me a lot, if I saw a diehard PvP'er saying "naaw" to a game, if thats my preference too.
The problem here is, that you ideally have reviewers with very different preferences.
Norden
It irks me in your reviews when the reviewers mention each other by name and say things like "At first...but...convinced". 2 or 3 people reviewing the same game would be great, especially if they were blind to each other. If I came here for three different perspectives on a game, I want 3. I don't want a jury that had to come to a unanimous decision."
"Will the defendant, Greg Street, please rise."
And of course, for issues like PVP, personal perspective is so imporant. If your class is OP and steamrolling the competiion, I don't need you to convince me that PVP is fine. I need to psot my experience.
Spec'ing properly is a gateway drug.
12 Million People have been meter spammed in heroics.
I had an idea about this some time ago... sort of linked to what Metacritic does.
How it would work is that you would compare Critic (site) Scores to User Scores and assign some kind of rating based on how close that critic was to the user score.
Of course you would need a reasonable number of user votes to make it valid (other wise a review could be made to look 'bad' because of one disgruntled player?)
And in addition to doing it by site - you could do it by the actual reviewer too.
For example:
Star Trek Online http://www.metacritic.com/game/pc/star-trek-online
After 456 user votes (pretty good sample size) the user score was 6/10
The average critic score was 6.6...
Now, how I would do it is give the critics a score based on how close they were to this score
http://www.metacritic.com/game/pc/star-trek-online/critic-reviews
So... sites like Eurogamer and PC Gamer score well (they hit the bullseye)
and sites like PC Powerplay, Strategy Informer (both scored it as 80) and GameSpy (rated it as 40) would get a poor score.
Obviously - sites that consistantly get it 'wrong' would have a poor overall score. You could then 'weight' their reviews.
I didn't go any further with the math and idea than that - so far.
Nothing says irony like spelling ideot wrong.
I'd like reviews to have scores for various categories as well as an overall score. That said, the overall score should be decoupled from the categories and be based on the reviewer's opinion of the game as a complete package. Let us decide to weight the overall score based on our own perception of the value of each category as well as the text of the review.
Also, for crying out loud, if you're going to use scores out of ten, use the full range properly. An average game should be a 5 or 6, not a 7 or 8. Reviewers who treat scores like they're out of five then add five points are being dishonest through carelessness. More integrity than that would be appreciated.
While we're at it, no decimal points for anything but user review aggregation (and even that's only a maybe). Whole numbers only, because the more specific you get with something as subjective as a review score, the greater the disservice to the reader's intellect.
Finally, never, ever write a review with the type of person who wants to see a score and move on in mind. Scores should be a place for general summary to cement the content of a review. Catering it to people who don't care about what's actually said is useless and helps no one.
Hopefully you read through all the responses and see this!
I'm not too concerned about the scoring system, as really who can rely upon the opinion of others. The thing I would really like to see is a list of major features prominently displayed.
eg: What type of PvP system? Faction based, consentual, open world. Advancement system - class / skill based / other.. single server / shards.. interesting or new features that makes it stand out from other games..
Id really like to not have to read through a whole review to try and siphon this information out (if its even there). I, like other players, are looking for specific features in their games.. and no matter how well done, some types of mmo's were just not interested in.
Your preview followed by a full review works fine for me, I have understood it and never doubted it. Your previews have always expressed and re-expressed that it is not a complete review and you never give the final rating in the preview.
If someone does not understand the method you have used for years now then they simply cant read a review properly and just read what they want to. If you are dealing with people like this then it will not atter how you review it, they will still complain.
Frankly I would just be happy with the letter grading system. Having a reviewer that likes that style of MMO (FFA PvP, themepark PvE, whatever) and one that generally doesn't favor that style each doing an initial review would be nice. That way you are sure to get the pros of the game from the one who likes that style, and the cons from the one who doesn't; that way those who take the time to read both can make the best informed decision possible. Still, if I could only choose one change it would be the letter grading system, or at least go to a pure 1-5 system; seeing a 7 score for a game when it is all actuality a 2 score is confusing to say the least.
"Oh my, how horrible, someone is criticizing a MMO. Oh yeah, that is what a forum is about, looking at both sides. You rather have to be critical of anything in this genre as of late because the track record of these major studios has just been appalling." -Ozmodan
I like the idea of breaking down reviews into relative categories that appeal to different gaming motivations. I like to see how a game breaks down in terms of graphics and sound, community, soloability, grouping, end-gaming, character development, innovation etc.
Reviews are by their very nature subjective, but a more organized approach may provide more literal insight with greater relevance to a broader audience.
Del Cabon
A US Army ('Just Cause') Vet and MMORPG Native formerly of Trinsic, Norath and Dereth. Currently playing LOTRO.
First off good for MMORPG to admit that the current system leaves much to be desired.
Secondly (which I am sure has been said by page 7) An A+ downto C- still gives 9 increments without quite as much stigma attached to low scores. Use all the increments! Also scoring different categories has some merit (though not too many categories). Also try and seperate heavily subjective elements to those that are reasonably quantifiable, one might hate the art direction but still aknowledge that the graphical quaities are good for example. Gameplay is an element that would survive some sub categories.
An earlier poster said maybe 3 reviews pre-release previews take them as you will, lest rigour is acceptable there. An 'early look' (which still should be given enough hours to get past the introductory game and into the meat and potatoes). This would be the 'main review' and should at least try and talk about most of the game elements. You could then have a longer turn review that deals with on later gameplay. Of course you could not hope to review so many games at that level.
You need some sort of 'guidelines' for reviewers so eveyone is singing off the same song sheet. Of course you could have guest contributors that are operating outside this but that should be made clear.
I'd like to see a tendency to less reviews that are more in depth with greater substance. Some currently feel like they duplicate the publishers blurb with a bit of opinion thrown in.
Ditch the score, just list Pros/Cons along with the review.
Problem with any review, it is subject to the author's likes and dislikes. You just cannot surmount that obstacle. What tickles a reviewers fancy, might be poison to you. No matter how you look at it, unless of course the game is a buggy morass, about the only way to determine whether you like a game is to try it yourself.
Review vs. preview: I think your previews are quite good. But to set them apart from the reviews you will need something "extra" on the review side of things, something that keeps readers clicking on that link even if the game is out for a month or more.
So, you have to improve the accuracy of your ratings, especially in reviews. You can do that by having more than one rater. It would cost more, but if you took 3 or more reviewers, each playing for at least 7-10 days for a substantial amount of time, you may sum up their scores to get a more accurate review. Of course it would take more time and manpower to do this, but you could split up the article among the raters. There are some ways to improve interrater-accordance, mail me if you want to know more.
Scoring system: nothing wrong with 1-10. Marginal differences will be expressed no matter if they're called 5.6 and 5.7 ot C and C+. Its just another level of differentiation. The real problem here is one of Information loss. If you aggregate 5 or 6 scores (graphics, sound, gameplay etc.), you will not be able to see if it was sound or gameplay that are good or bad. So: Either you give the game a general score, which may be based on factors irrelevant to some gamers, or you split the whole thing up into facets and generate different listings (Top ten in gameplay, top ten in sound, top ten in Graphics etc.).
Maybe it would be kind of an alternative to generate such lists and to present them in a row with the list for general score.
Just my 2ct, cheers
I could care less about the number or letter system personally, I normally dont put much stock into it either way.
I would rather you give us a review of each system in the game and be much more honest about the game other than its fun and I had fun. Im not trying to say you guys arent honest but I think you do gloss over some bad elements of a game, I believe earthrise was one example. Every game is fun at first, if its playable at all, I want to know if this game is worth my time, I want to know if its just another gear grind like wow that I wont have time for, I want to know if im gonna be doing the same thing over and over for 50 levels, honestly its why I dont play rift anymore, or wow and probably wont get swtor, im sick of picking up quests and going and killing x monsters and then repeating. I want your reviews to tell me that before hand. I want your reviews to detail crafting, ive seen it so much, a writer will say im not that into crafting but its there for those that want it. I want the writer to actually try the crafting and let us know what it is about. I also dont really care for reviews the first day, a launch preview is fine, but I think a review of the game should come out 30 to 60 days after the game launches to give the writer some time to play the game and try the systems out.
Too much posts to read through, but I saw one that I fancied.
Use the "Buy", "Try" or "Skip". That's my two cents.
Well this is probably a dumb idea, and my suggested review format is not easily aggregated with many web sites opinions and reviewing systems. However having bad ideas seldom stop us posters from posting /lol. How about a "review" with a structure like this:
Part 1. Description of game, general description of Requirements, special needs and maybe innovation.
Part2. Summation of a reviewers actual gameplay and pertinant comments.
Part3. Your famous list system. 10 (less if you cannot find 10) things the game does right and that would compel you to actually play the game ..
Part4. 10 (or less if there are not 10) things the game does wrong that might turn you off from playing the game.
This would all be subjective (as all reviews are by nature), but might give us readers more information from the reviewer and thusly a better understanding of what the reviewer based his "judgement" on. That way we could kind of review the reviewer as well (which we do anyway now, but with no standard format of reviews). We are prone to dismiss a review if the reviewer has a "bad pen", (like bad hair) day.
If Ya Ain't Dyin, Ya Ain't Tryin
I don't think you should change the 1 to 10 numerical system because you 'll be "incompatible" as you mentioned, we are used to it from games and therefore (for some reason) I still replace a +1.1 str ring with a +1.2 one.
As for the 1-5 it's a really a good rating for unplayable mmorgs due to various of reasons, from a bad client-network perfomance to a bad interface implementation.
Since you have to suggest (or not) a product to us from its launch day just make sure to take just the facts and features of what this game has to offer including unique ones (if any) and leave unmeasurable features (opinions) like gameplay, potential and immersion for a later time.
You can categorize and break a game into its smallest components using a fact and features List which will be actually a union of all facts and features that most mmorpgs have, how many of them this game implements, in what way and then let us decide if the game can satisfy our expectations.
Some facts or features may override other ones in a way that if certain fact or features is absent or badly implemented the total rating of the game should be less than average.
All of these details combined with a lengthy gameplay video that shows them off can even save you the trouble of rating the game yourselves and help us decide better imo.
I know it sounds like "we are the robots" review system but i prefer this than any other way since i don't trust reviews and i don't like wasting money every time a "new generation" mmorpg is released.
A few suggestions. One , go to a letter grade system. At least you can't fudge that , no -/+ ltter grading. If a game falls within A then it can be trusted if it's a C then at least we know that the game runs as passing with issues, etc..
Secondly find a way to generate income other then advertising from game producers. No matter how impartial you may try to be , accepting money from a Dev company to advertise calls into question objectivity. If your site is good enough , they want to advertise regardless.
This incidently brings me to the whole intergrity of the process issue. To promote your site you need inside information or early access to build traffic so you can generate more money from advertisers. The devs need you and give you access to advertises their games and build hype for it. Kinda of a viscious circle? As gaming journalists who for a living basically exsist off the good will of the devs. You depend on that access, it seems to heavily. You're afraid a bad review shuts the doors to future products. You've set up a system that basically is a wink and nod system that, " If I don't say anything overtly bad about a game , they'll grant me access to the next porject . " Here's a novel approach , you have access to Beta the same as all of us , sometimes better. Use that as the start point , because everything before that is controlled hype. You build credibility draw more traffic and thereby more advertising dollars. It all starts with basic ethics .
How about two or three people review a game simultaneously and then sit down and compare notes , what they like and didn't etc.. Oh, and please run through the whole game , not necessarily all the classes , and do the crafting , diplo etc.. to the end. Yes it timely , but it'll lead to more credibility and guess what that leads to ?
Gaming websites are in a position where we have to try to get a review of a game out the door as close to launch as possible because that’s what’s demanded of us so that you all can make a more informed choice as to where you put your money.
I'd add also the pressure to be first with reviews before other review sites do, but anyway, I'd have a two-star system even though it will never happen:
Two stars=outstanding. Not to be missed.
One star=worthwhile if you haven't got anything else to play.
No stars=Same Old Crap
"So we’ll start at the top. The reason that games are reviewed too early in many cases is simple: Gaming websites are in a position where we have to try to get a review of a game out the door as close to launch as possible because that’s what’s demanded of us so that you all can make a more informed choice as to where you put your money."
Perhaps start off with an honest assessment of why you do an early review. It isnt because you want to tell the consumer where to put their money. It's because you want to be the first to market and have a claim on the "first review" of a game. You want to claim viewership as much as anything else. Promoting your web-site is the ultimate goal of, well, virtually any product out there.
This is a little late, but as many previous posters have mentioned the biggest problem with reviews is they rely on the reviewer's opinion for at least part of the article. Adding too many increments to a rating system over complicates it as well. A simple, tried and true, 1-5 works for me. Or just A-F no ".5" or + or -. A good review is going to consist of what the game offers followed by what works and what doesn't. If I review a game no one cares whether I hate elves they just want to no if the game in general works or not and whether it's worth playing.
A pros and cons section works if you include what's in the cash shop for F2P games.
On a personal note; DUMP this idiot RipperX. Who is he that I should care about his first impressions? Is he your main reviewer? Then I don't care about his first impressions. He comes off like a very late to the party version of MMOHut's Omer/Remotay (whatever his name is...) At least with Remotay he's reviewing the game along with an initial look at it. And though those reviews have their flaws too they stand leagues more entertaining and informative than listening to Ripper talk about "pooping his pants" in the LoTRO video.
I get better reviews on a game reading the Survivor Guy articles.
___________~____________
Man is least himself when he talks in his own person. Give him a mask, and he will tell you the truth. - Oscar Wilde
I think the Letter Grade method is much more informative of what it would be to the current overall generation of games. Rather than a number scale that seems to represent 10 being the best ever. but 5 years from now hardware will be 5 times better and overall game designs will have changed with that. Which in turn would have made that old perfect game not so perfect anymore. Not only that we all have different opinions in what we like, thus numbers just seem completely arbitrary. Where as letters are a better representation of our opinions.
As far as Previews/Reviews go, I think that there should be an ongoing news blog for games that the journalist actually play or have played to give each person's insite on the game. Kind of like Movie reviews... IE. the "Movies" app on the Android Market that connects with the same facebook app. You have a section of journalist reviews and then a section of fan reviews. and the overall scores of each are accumulated to the overall Score of "critics" and "movie goers".
So instead of just using 1 journalist, use all that actually play the game for a time more than 1 month. And those that actually actively play could have a running blog that could explain the different happenings within the game, and report news/updates as well.
You could have a special section for each game for the opening month of the game for it's initial release review, then have the overall review at least a month or 2 after release for a more substantial "state of the game".
just my 2c
I tend to judge my games based on their "worth", compared to the price. You could try something like that as your "rating" metric. Game is worth: 1) Box price plus free trial; 2) One month sub; 3) Multi-month sub; 4) Yearly sub; 5) Lifetime sub.
If you tell someone an mmo's not much better than a free trial, they get the point much better than 6.6. Tell them you'd lifetime sub to it, and they'll know what that means too.
Might be weird, and you'd have to adjust for F2P games, but who knows. Just a thought
I found a thought-provoking article about game reviews in general written by a gaming journalist at this link: http://nohighscores.com/node/508
I have no idea whether or not this guy's experience is typical, but it was an interesting read. As for the reviews here at MMORPG.COM, I'm old school. I always found it more informative when various categories were rated, and then used to calculate an overall score.
If a game got a low overall score, I'd always check which categories seemed to hurt it the most. If it was a category I didn't really care about, I'd still give the game a shot.
The categories also seem to give the review more objectivity imo. I know when I'm involved in hiring people, we've used a grid to help us score the person's performance in the interview. I find this helpful. I may get a really good vibe from someone, but when I look at my grid, I see that they have no idea how to do the job. I'm human, and without some distance and some objective criteria, my feelings definitely influence my assessments, whether I want them to or not.
I will give this more thought, but one quick notion...
If there was a quick and easy to to access player reviews next to the staff's review I would find this truly more useful then any other upgrade or change.
While it is true most people in the world are fools, when intelligent I find user reviews seek to be broader and draw more comparisons I understand and also pull less punches.
There is a political correctness (for lack of a better term) that goes with staff reviews. ( I think this is why people suspect corruption.) Therefore the reviews often feel stiff. Often it may feel like a staff review is "safe" Or the staffer may not see a need to compare somthing to other games so as not to offend or start an argument etc.
That being said if you removed user reviews that are not meant to be informative ( likely half or more) so as to not seem like another forum that would help me alot. thanks!
OH and PS, I would love to help review for MMORPG.COM or contribute. Just saying I am willing
Thank you though for this thread. I feel your pain as it seems you cannot do anything like what you are doing and not suffer from an onslaught of "lessers".