Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
General: Forget the Scores
SBFord
Former Associate EditorMember LegendaryPosts: 33,129
Of all the things we do here at MMORPG.com, writing reviews is one of the most difficult for a plethora of reasons. We pour a lot of ourselves into our reviews and all of that hard work, all of the reasons we have for scoring things as we do often gets buried by the easy "skip to the number" mindset. We have a few thoughts to share on our reviews and why you should Forget the Scores. Check it out!
It’s really not an easy task reviewing a game. I mean, we spend hours scouring the thing and then are asked to condense the entire experience down into a few thousand words (no small task with an MMO), and to top it off the general rule of critiquing a form of media is to “rate it”, as if some number is going to be the end all and be all of a product’s value. Ask any reviewer, ask anyone here on the site, and they’re likely to tell you that a score doesn’t matter. And yet, we’re held to these numbers as the standard of how we review things. What I’m getting at doesn’t mean scores on our reviews are going away anytime soon. Rather, what I’m going to ask of you all is to try ignoring the numbers until you read the words. And we’re going to help with that.
Read more of Bill Murphy's Forget the Scores.
)
Comments
Man ..realllyy! ? 7/10 for this piece of crap review..euh ..review.. man it totaly lacks humor! should stick to mmorpgs.com way better!! this get a 5/10 max ..for the picture. atleast its a little artistic!
It is not just the score (even though I dont understand why you have the score if we are to ignore it), you also have this word which is something like Excellent for SW:TOR.
When someone sees 8.7 and Excellent and then see a product full of bling bling (voice overs and what not) but little actual depth and innovation compared to the expectations they have then there will be, naturally, a reaction.
For example, someone made a list on the SW:TOR forum about all the things this game was lacking. From open world PvP to guild functions to UI customizability to character customizability and found alot of things that are lacking, which are basic in most MMOs these days.
So with this in mind it makes no sense to score so high and have an excellent next to it, even if the review briefly mentions some of the issues. So I would say that the SW:TOR review, which is what this article is really about, is lacking both in the score and content. It is basically reviewed like a single player game instead of an MMO with all the features expected in such a game. It is downright terrible and I would even claim, dishonest.
My gaming blog
What you dont grasp though is that the phrase 'Your Mileage May Vary' is especially relevant here.
I am actually flabbergasted when I see people seriously posting that SWTOR isnt a real MMORPG, because *in my opinion* it encourages grouping as much as the other games out there. the dungeons (flashpoints) are fun. There are tons of group quests (something missing in games like Rift) and while the raids are smaller, WoW and Rift have already made 10 mans the preferred raiding method (look at the amount of 10 man content released in Rift vs 20 man) so 16 isnt *that* small of a number.
Also people talk about the companion as if it not soloing when you use one, but its just a pet that the game is designed around you using as part of your solo character. I also played EQ2 so the sharding just seems normnal and I have never had issues seeing other people in game. In other words, YMMV.
I think SWTOR is the most FUN MMORPG since WoW/EQ2 released in late 2004. So the details like the UI, which will be changed anyway, dont atter to me as much.
The score for Rift and SWTOR is the same. I would not give either game an 87 (neither are above an 80 IMO) myself, and I think SWTOR is better (its more FUN) but I cant really see being up and arms that both games have the same score one way or the other.
YMMV
Couldn't have said it better myself, I was really left for words after the SWTOR review and think it was quite a bit off the appropriate "score" for the game. As an MMO it is lacking in several key areas and these should have been highlighted more and scored appropriately. The score an MMO gets is the first impression many will get of the game and will mislead people if not in the appropriate area.
Yeah, as an MMO I can't justify giving SWTOR more than a 7.5, even though it's fun, because I have yet to see proof between myself and my friends that play that grouping is more encouarged than in past MMOs. It's not true. But against these pro reviewers and fanboys, who am I to argue without being called a "hater"?
The only scoring system that makes any sense is "0 or 10". Gamers bifurcate everything anyway...it's just how message boards work. Non-sensational opinions get buried quickly anyway.
Self-pity imprisons us in the walls of our own self-absorption. The whole world shrinks down to the size of our problem, and the more we dwell on it, the smaller we are and the larger the problem seems to grow.
OK, Bill, the main critcism to your scoring is the following: quite often, there is a very big gap between what a reviewer has written and the actual score. Taking once again SWTOR , about which I guess your article was written, here is an example:
In the section "Innovation", the reviewer had said that there weren't really many new things, as this is your standard MMO, even if very polished. Yet, the score was 8. If 8 means "not much innovation", then one may wonder, what the grades from 1 to 7 mean? In effect, it appears that an 8 score means "ok", and then you are left with only 9s and 10s for better performance.
So what's the point of all this? Why use a 10 scale in the first place, if you are only going to use the top 3-4 scores? Wouldn't it be better if you just move to a 5 scale, and save yourself all the trouble? I mean, it's simple math - if you have a scale from 1 to 10, you could presume that scores from 1 to 5 means "not good" in different intensity, and scores from 6 to 10 mean "good", in different scale.
Going back to the "Innovation" example, I would be more accommodating if the reviewer had written that the introduction of fully voiced conversations and group conversation was a truly groundbreaking innovation, which is why the score is 8. However, the reviewer didn't write that, and you kind of wonder, does he have any idea what he is reviewing at all?
So it's not like everyone is trying to be negative here, it's just there is a huge discrepancy between what you are writing and what you are scoring, and I think you need to remedy that. Because, after all, scores may not be more important than the review, but it's what people see first, so it's important the two not to differ too much.
Yea forums tend to promote unjustified remarks, stating opinions as facts, and hyperbole after hyperbole. Only occasionally do I really have a beef with the ratings mmorpg.com gives to the games in their reviews. But most of these are when they review unfinished games, and list off features that aren't even implemented yet (Xsyon I believe is one of these reviews I was completely apalled by). But for fully released games made by companie they seem to do an alright job. If you want truely polarizing reviews go watch zero punctuation.
"They essentially want to say 'Correlation proves Causation' when it's just not true." - Sovrath
Sorry, Bill. You'll always be fighting against the 'No one is allowed to have an opinion unless it matches mine' mentality in this community. People are going to find fault with your numbers, with your words, and will invent fantasies in their heads of seedy backroom deals with the publishers encouraging you to think one way or the other.
LOL! You made me spew my Kix cereal when I read your post. Thanks for the great laugh and the spot on remarks about the seedy backroom deals.
¯\_(ツ)_/¯
Actually tor has alot of depth. i find the people saying otherwise havent actually even played the game.
Im sorry but im having alot of fun in tor the heroics are a blast the flashpoinjts are fun pvp is better for me then any of the pvp i ever did in wow.
The crafting is alot of fun even if it is a money sink. Im sorry but i think mmorpg and others did a good job with there reviews.Tor has alot of content. it has alot fo cool things going for it.
the vo and companion system are innovative. As is the choices u make that are light or dark that influence your story.
So haters are gonna hate but in the end tor is a succesful game so far and in a yr all the haters will be asking how so mnay people stil play it and calling it a flavor of the month game
or saying people only play it because its popular just like they did with wow ignring one big fact. Most players play games to have fun and tor is a blast.
Unlike all the jaded gamers who come here and blast every game other then gw2 Who ask for innovation but really are asking for fps mechanics in an mmo or no quests which really confounds me most of us just want a fun game.
i dont care if a game is innovative all i want is a fun game. And the majority of gamers out there only care if a game is fun not how innovative or how much freedom they have
for the majority of us we dont have alot of time so we want a game that gives us direction. We like thempark games unlike th ose that hate on tor.
Kix cereal?!?! Seriously Bill? I give that cereal a 4/10!
Lol, reviews in themselves are usually good but then they mess up the score.
95% of all computer games get the score between 6/10 and 10/10, if not more. Average should logically be 5.5 which for some reason is no average score but something that completely and utterly sucks.
I usually just subtract 5 and see it as out of 5 instead, it always give a pretty realistic score.
There is no way I always can agree with a review score but that is just fine, I am probably the only one who would put the scores just like I want it anyways, but it is rare that if I look on 5 different reviews and take an average at least don't think it is pretty fair.
First off, I'm not Bill. >.>
That said, I LOVE Kix. Besides, in a mom-house, there's only kid cereal to be found.
Now, back on topic!
¯\_(ツ)_/¯
So after the many recent outcries about "free high-scores for every big companies' title" you are basicly looking for a not-so-obvious way to sell us hype and bias business-journalism, introducing your readers to mediocre games that come with lots of advertsing money?
On my part, i don't need a different rating system, if in the end you don't honestly list the cons about a game anyway. I'm not talking about the usual fluff cons like "- game takes 20GB of hard drive space".
It's not about how you paint and name your rating system, it's about how believeable your ratings comes over.
I remember reading about the review system this site uses afew months/years back. I think i even suggested some sort of 5 Star system to apply instead, but lots of other posters gave suggestions too.
Then didn't you guys already try to use a new "system" to review the games only afew months ago? I remember alot of people flaming the 1st MMO you reviewed with it under the new system. Also, then you lot had an article about SWTOR only a short time ago, where 5 out of six of your staff (not sure of the numbers) must have been on drugs or something and they were throwing around high 8 - 9.5 scores for the game. Heck, I think Bill even gave it a high score aswell! Only one of your staff members seemed to score it right.
Anyway, whatever way you guys decided to rate games in future, you need a simple system that's easy to understand & that has some actual weight to it. At the minute it seems like you look into a hat and just pluck 8.5's out and slap it on these MMO's and then they start to tank afew weeks later.
How is hard to grasp?
Review: "This game is more of the same. X is bad, Y needs work, Z is just like every other game."
Score: 8.8 AMAZING!
The score and the review should match up!
Until you cancel your subscription, you are only helping to continue the cycle of mediocrity.
It appears that the normal 1-10 standardized scale we have all come to live by really doesn't apply for the reviews here. I'm pretty sure if you can render a floating cube and make it show up someone else's screen you have already earned yourself a 4 right there.
Let me just say that I find it very funny that I ask you to put the scores secondary to the words, and the comments quickly became all about the scores.
I get your concerns folks, really I do. But we are trying to do our best service and honest recommendations on every game we review, which means we're writing about the good and the bad equally, which makes the scores seem unjust. That's sort of my point. But rather than us NOT write about the bad, I'm asking you to realize that a score of 8/10 can mean a game's a lot of fun despite its faults.
Additionally, the stance we take on scoring in general is akin to a Grade Scale in any American High School. Anything below 6 is likely considered failing, a 7 is a C, an 8 is a B, and an A (90/100+) is damned near perfect.
Again, you may not agree with this method, but it IS how we do things and we ARE always evaluating things.
Try to be excellent to everyone you meet. You never know what someone else has seen or endured.
My Review Manifesto
Follow me on Twitter if you dare.
Bill,
Just ditch the numbers, honestly. What you're saying is that you are willing to give a game a high score regardless of how bad the bads are, just because you think it's good beyond it's faults. That's fine, but that means the score essentially has little if anything to do with the review, so why bother? The point of contention is that you list some pretty heavy negatives (to some) then follow it with a score that is like saying there weren't any to begin with, and I'm not just talking SWTOR...this kind of thing was done with many games (DCUO comes to mind).
Sometimes your scores are like those drug commercials all over TV. "Will cause heartburn, diarrhea, migrains and anal leakage, but at least you won't get preggers!"
"Forums aren't for intelligent discussion; they're for blow-hards with unwavering opinions."
What?
So there are (1.0-6.0) six kinds of failings but only 7.0-7.9 for average, 8.0-8.9 for good and 9.0-10 for great to perfect?
I get it.
@nblitz
Welcome to the American grading system employed at EVERY school in the country.
¯\_(ツ)_/¯
Welcome to the world of captialism where money effects numbers. or in this case (grading scores).
Im going to have to agree with a number of the people who are saying that they dont get how the "numbers work" as far as gaming site reviews go. Its not a matter of understanding the system that is used but understanding why or how certain games are getting those reviews. Ive found objective player reviews for games much more accurate then gaming sites in recent years. ( as companies dump money into said gaming sites for advertising )
take this one for example : http://www.mmorpg.com/discussion2.cfm/thread/337702/Elikals-SWTOR-review.html
I found it to be a very informative and decent read. It pretty much gives pros and cons in a nut shell with room for people to agree /slightly disagree in some scoring areas.
Bill, I endorse this post.
Cheque is in the mail!
(It's good to review one's reviewing methods every now and then. Keeps things fresh, honest and relevant.)
You aren't marking an English essay like a grade school teacher, you are rating a game. The games you are reviewing are made by professionals, not kids that need an outdated grading system to get passed through the school system like a hot potatoe.
Enter a whole new realm of challenge and adventure.