Time is money. I've been voting with my wallet on this issue, though I may give eso another shot. I suggest they do the same thing internally. Weigh the low cost of hiring professional testers (who are more likely to find and report problems) vs constant buggy reviews that reduce the chance of getting more players.
The solution isn't with the players. We are the customer. We pay for a working product that we deem worthy of our time and money. If someone wants to help great, but it shouldn't be relied upon.
Maybe they could design an extra costume for each dlc and give it to people who completed at least 80% of the dlc's achievements on the pts and provided feedback. Probably not cost effective for them tho.
Earning crowns would work nicely that would give them incentive but make it not just the time played but also the number of bug reports reported and give more for bug reports that are valid and not duplicates.
Well, if the community screams 30% drain poisons are too strong on the PTS and the devs reply by doubling their strength, the community tends to go 'sure, ok, screw you then'.
Absolutely. Potions now at 60% and pages of dissatisfaction.
Back bench for a minute the concern of whether more players should use a PTS - the biggest issue is having feedback from long standing experienced players, often leaders of large guilds, who are putting in extended time on the pts:- then completely ignoring it.
That's why players are then skeptical about using the pts, as they see it as a waste of time. So then they resort back to the game forums again.
IMO The Elder Scrolls Online is not a game like Runescape (Community Driven). We paid for a service and it should be served without any issues. If it was asked to community for new content that is a completely different story. But yeah, If ZoS willing to reward testers that would be a very good idea.
In other hand, ZoS is very small video game studio. I think instead of using PTS method, they should hire more engineers.
Well, if the community screams 30% drain poisons are too strong on the PTS and the devs reply by doubling their strength, the community tends to go 'sure, ok, screw you then'.
Absolutely. Potions now at 60% and pages of dissatisfaction.
Back bench for a minute the concern of whether more players should use a PTS - the biggest issue is having feedback from long standing experienced players, often leaders of large guilds, who are putting in extended time on the pts:- then completely ignoring it.
That's why players are then skeptical about using the pts, as they see it as a waste of time. So then they resort back to the game forums again.
And that's exactly why the invitation only system for test servers works better. Sure, you don't get the massive numbers so the sort of issues that crop up only when you throw large numbers of players at it won't be as easy to identify, but you do get a lot more back and forth with the developers about balance changes and the rationale behind them.
From my experience in those types of private test servers with NDAs in other MMOs I never felt like we weren't being heard. We might have agreed to disagree with the devs in some instances but we all had a say and a better understanding about the changes.
"Social media gives legions of idiots the right to speak when they once only spoke at a bar after a glass of wine, without harming the community ... but now they have the same right to speak as a Nobel Prize winner. It's the invasion of the idiots”
― Umberto Eco
“Microtransactions? In a single player role-playing game? Are you nuts?” ― CD PROJEKT RED
IMO The Elder Scrolls Online is not a game like Runescape (Community Driven). We paid for a service and it should be served without any issues. If it was asked to community for new content that is a completely different story. But yeah, If ZoS willing to reward testers that would be a very good idea.
In other hand, ZoS is very small video game studio. I think instead of using PTS method, they should hire more engineers.
You guys don't seem to get that there are levels of QC. You can hire 100 more engineers and there will still be a need for a final more populated QC step... and no matter what, there will be bugs that will make it to live.
"Social media gives legions of idiots the right to speak when they once only spoke at a bar after a glass of wine, without harming the community ... but now they have the same right to speak as a Nobel Prize winner. It's the invasion of the idiots”
― Umberto Eco
“Microtransactions? In a single player role-playing game? Are you nuts?” ― CD PROJEKT RED
Incentive for testing: for every X hours spent testing (not just logged on to the PTS, but actually testing whatever the devs say needs testing), reduce my monthly sub by Y, or grant me n cash shop tokens.
If the normal sub is $15 a month and I can reduce that by $1/hour testing, I might actually put in 5 - 8 hours testing a month for you.
Maybe players should be able to clone their character to the test server or make a new one which would stay on the test server. This would seem like a fair way to do it since what is on the test server would stay on the test server. Some sort of compensation should also be considered for the players that do use the test server. Anyway Bethesda has always relied on players to fix their single player games so why would an MMO backed by them be any different.
I think the article brings up a good point. Some things with MMO's never change. New ways of getting players interested in testing should be looked into by the industry as a whole. After all, players get a reward for playing PvE and PvP but nothing for testing content.
As to ESO another point is that I've followed the posting in the forums and have seen a lot of complaints about concerns raised in test not being even acknowledged when brought up by the player base to the Devs, and this has gone on since launch. I've also read complaints about patches being tested by the player base not being the same ones that were released. So one point would be players feeling they are being selectively ignored during testing.
"We all do the best we can based on life experience, point of view, and our ability to believe in ourselves." - Naropa "We don't see things as they are, we see them as we are." SR Covey
There is a reason that every game developer should have a dedicated QA team inhouse testing the new features.
These days it sadly is a cost that management see as not being cost effective. It is all about profit and cutting corners to make the investors happy with a bigger return on the investments made.
Sure there is a QA team during the development of the big AAA titles and as soon as the game is released the QA staff is the first to get sacked. And if anyone is still employed as a QA tester it usually is a really small QA team.
And to host public Patch Test Servers is good and all that but to do so you need to have a dedicated team that is able to sort through all the bug reports and feedback from the players and compile said bug reports and feedback in a way the developers understand. That will say pie charts and hard facts about issues at hand.
Dam Ryan ZOS will listen to me just as they listen to you. which means they won't. You named the reasons why I and most don't participate in the PTS. Please wait for the Dark Brotherhood get the quest you are concerned about and if it has a problem send ZOS a bug report like the rest of us. Until then just ... well have a good day.
A game that is not made by the community isn't required to be tested by the community. It's the creators responsibility to see to it that it's properly tested and debugged. That's software dev class 50. (Way before 101.)
It's awesome when the community helps out and tests, but it's not required.
If you let freebie players do it, they probably will.
If you want paying players to do it, you should probably give them a positive incentive.
Another big hint: Let them know when it's going on and what you want them to test, and make it very public! Maybe, a login notice of some kind...
I've been on plenty of games that you never know when something new is being tested because the devs don't bother to tell us directly. Sure, sometimes they post something in the forums, but honestly, most people don't trawl around in the forums looking for tidbits of testing mentions from the devs. They're too busy playing, or talking about how to play better, or bragging about their playing, or whatever, to do that. Any dev that doesn't have his head stuck up his rectum and is woozy from the methane already knows that.
You want them to test?
Tell them very visible when you have a specific test going on, like a new expansion or big update, and what you want them to test.
(If you aren't clear, they'll do whatever they bloody well want to.)
Give them compensation for not playing their characters/game to test for you!
(As to that, base it on time testing, not random drawings, or everyone this week gets an in game tester hoodie for a character, or anything else that doesn't feel like it's making the time worthwhile.)
Dont bother with PTS, thats for them to use. I am not goign to waste my paid gametime testing their game for them, to me its a complete and utter waste, may as well wipe ones ass with paper money and flush it.
As said above if a company wants to encourage volunteers it can encourage them.
Going through inputs is a non-trivial, time consuming, task though.
And it also has to engage with testers - which also takes time. So if the community says X and the developers ignore X or do the opposite then testers have to be told why. Or - as said - they go so what.
It is in no way the player's responsibility for the game to be tested and playable. We do our part by giving them money and playing it. That's as far as it goes, imo.
Yes, players can make it easier, for sure, but they should never be blamed for a patch or game being a buggy mess.
That being said, I am sure there are ways to make it worth people's time to test the game by rewarding them on their live characters or something, and I never see any rewards for testing, really. I still wouldn't take part because I prefer a patch to be fresh to me so I don't get bored right away.
I dont think that it is the communities job, but I do get that MMO companies cannot effectively test the "massively" part of their software without the community. So while I think it may have been phrased poorly by that poor Bioware developer, I understand the intent that Devs can only do so much testing regarding the "massively" side of things.
Just because Zenimax is a multi million gaming company, that doesn't mean they can afford having 100s to 1000s of their staff testing the game and fixing bugs. And even if they could, that would hardly help.
If it was that easy to squash bugs, we'd actually have bug-free games today. Hell even the SP games are so bugged at launch, you'd think they haven't tested them internally at all. MMORPGs are on a whole new level though, for an obvious reason, they are massive.
It's just something we have to deal with, there will always be bugs. Open-world games that give you more freedom are especially susceptible by it, that's because it's just not possible to predict billions of different outcomes that could come from the players themselves. You can't predict their behaviour.
And this is why PTS is so valuable. It allows them to gather information based on player's different behaviours. When there are thousands and thousands of players testing the new content, it helps them at least to see what gets broken where there are too many players around. It's just something QA Team would never be able to do on their own, they don't have the numbers, nor are they there to actually play the game.
The way ZOS handles the PTS was definitively a great idea. Without it the game would have been riddled by bugs for months. This way they can at least fix most of the bugs on the PTS and then release the DLC live. But there is always a room for improvement. Players need a reason to test on the PTS, different kinds of tests, ZOS developers playing with them etc. Also rewarding them with some ingame items seems like a good idea too.
I guess OP is at least right about one thing. Bad news is always louder than good news. The comments here are perfect examples for that. The way I see it, players these days are like spoiled children, taking everything for granted. They think by paying 15 dollars a month, they can ask for everything. What's worse is that ESO technically requires no subscription now, meaning some of these whiners don't even pay.
People have been crying about how evil these multimillion game companies are and how this industry is going to be ruined by them. If anything, these players are somewhat responsible for that.
Player Test Servers are a thing of the past; where players cared about games to spend time there and help. Now, with year long (and more) paid alphas, closed betas and open beta's, aren't the developers getting enough testing out of the player base? The new trend of pay alphas and the mass amount of games to chose from limit the desire to help test a game for the developers.
Was he wrong? Does the community have some responsibility in making sure the updates are fluid and well tested?
Yes, he was wrong. We are customers, not empolyees. We pay money to use a system they provide. And it is not the customers duty to test the product, he pays for!
Test servers are always so under populated when new content is out for testing. I don’t know if it’s due to players not wanting to spoil something, not wanting to go through content just to repeat it when it’s released or they just don’t want to download an entirely new client just to test something that will in no way advance their character.
Both. And: I spend my free time enjoying a game. I don't have enough of this free time to 'work' for a game company as a volunteer.
Meanwhile, people rage for a chance at "testing" WoW: Legion and keys sell for up to 200 USD - 4-5 times the price of the actual expansion :P
You cannot expect proper testing from your playerbase. Some might help if they have forum accounts and so on, but the vast majority will never touch/download a PTR, and if they do it's to test new abilities, buffs/nerfs and how they impact their chars, respec for free if it costs in the game and so on.
The conclusion? Don't be cheap - hire QA personnel.
Oh looky another stealth thread telling us how great ZeniMax is lmfao. Tell ya what OP why don't you and the staff on this site test content for them? God knows you owe them that for all the pontificating you do over such an average mmo.
Its clear that a lot of people here are new to the genre or know next to nothing about how it has changed over the last 18 years. And mind you, this is going to come from someone that cannot stand this game, even a little.
There once was a time when MMORPG companies (Turbine, Sony, ETC) used to go out of their way to make sure that players helped to test the games. They would have premade characters with the proper equipment waiting for you. Give out rewards. Hold special events on the PTS with GMs...they GAVE REASONS to help find bugs.
To those here saying a good company would "enter BS here and find the bugs themselves"...it IS NOT POSSIBLE. Do you even MMO? They cannot even be released bug free after 1 1/2 years of Alpha/Beta testing...there are too many variables in players systems as well as game features/code that could have a flaw in them.
Hell, even single player games made for a console (set hardware) are released with bugs.
Players are the very reason why the genre has been in a downward spiral. To most of them getting into a beta test is a free trail period...they do NO testing at all, they dont try to find bugs and they dont report them when something wrong happens. Some, even go out of their way to hide the bugs they find so they can use them as exploits after release.
The players are responsible...everyone is ALWAYS responsible for their own actions AND inaction's.
"People who tell you youre awesome are useless. No, dangerous.
They are worse than useless because you want to believe them. They will defend you against critiques that are valid. They will seduce you into believing you are done learning, or into thinking that your work is better than it actually is." ~Raph Koster http://www.raphkoster.com/2013/10/14/on-getting-criticism/
Comments
The solution isn't with the players. We are the customer. We pay for a working product that we deem worthy of our time and money. If someone wants to help great, but it shouldn't be relied upon.
Absolutely. Potions now at 60% and pages of dissatisfaction.
Back bench for a minute the concern of whether more players should use a PTS - the biggest issue is having feedback from long standing experienced players, often leaders of large guilds, who are putting in extended time on the pts:- then completely ignoring it.
That's why players are then skeptical about using the pts, as they see it as a waste of time. So then they resort back to the game forums again.
In other hand, ZoS is very small video game studio. I think instead of using PTS method, they should hire more engineers.
From my experience in those types of private test servers with NDAs in other MMOs I never felt like we weren't being heard. We might have agreed to disagree with the devs in some instances but we all had a say and a better understanding about the changes.
“Microtransactions? In a single player role-playing game? Are you nuts?”
― CD PROJEKT RED
“Microtransactions? In a single player role-playing game? Are you nuts?”
― CD PROJEKT RED
If the normal sub is $15 a month and I can reduce that by $1/hour testing, I might actually put in 5 - 8 hours testing a month for you.
As to ESO another point is that I've followed the posting in the forums and have seen a lot of complaints about concerns raised in test not being even acknowledged when brought up by the player base to the Devs, and this has gone on since launch. I've also read complaints about patches being tested by the player base not being the same ones that were released. So one point would be players feeling they are being selectively ignored during testing.
"We all do the best we can based on life experience, point of view, and our ability to believe in ourselves." - Naropa "We don't see things as they are, we see them as we are." SR Covey
These days it sadly is a cost that management see as not being cost effective. It is all about profit and cutting corners to make the investors happy with a bigger return on the investments made.
Sure there is a QA team during the development of the big AAA titles and as soon as the game is released the QA staff is the first to get sacked. And if anyone is still employed as a QA tester it usually is a really small QA team.
And to host public Patch Test Servers is good and all that but to do so you need to have a dedicated team that is able to sort through all the bug reports and feedback from the players and compile said bug reports and feedback in a way the developers understand. That will say pie charts and hard facts about issues at hand.
It's awesome when the community helps out and tests, but it's not required.
If you let freebie players do it, they probably will.
If you want paying players to do it, you should probably give them a positive incentive.
Another big hint: Let them know when it's going on and what you want them to test, and make it very public! Maybe, a login notice of some kind...
I've been on plenty of games that you never know when something new is being tested because the devs don't bother to tell us directly. Sure, sometimes they post something in the forums, but honestly, most people don't trawl around in the forums looking for tidbits of testing mentions from the devs. They're too busy playing, or talking about how to play better, or bragging about their playing, or whatever, to do that. Any dev that doesn't have his head stuck up his rectum and is woozy from the methane already knows that.
You want them to test?
Tell them very visible when you have a specific test going on, like a new expansion or big update, and what you want them to test.
(If you aren't clear, they'll do whatever they bloody well want to.)
Give them compensation for not playing their characters/game to test for you!
(As to that, base it on time testing, not random drawings, or everyone this week gets an in game tester hoodie for a character, or anything else that doesn't feel like it's making the time worthwhile.)
Lost my mind, now trying to lose yours...
As said above if a company wants to encourage volunteers it can encourage them.
Going through inputs is a non-trivial, time consuming, task though.
And it also has to engage with testers - which also takes time. So if the community says X and the developers ignore X or do the opposite then testers have to be told why. Or - as said - they go so what.
Yes, players can make it easier, for sure, but they should never be blamed for a patch or game being a buggy mess.
That being said, I am sure there are ways to make it worth people's time to test the game by rewarding them on their live characters or something, and I never see any rewards for testing, really. I still wouldn't take part because I prefer a patch to be fresh to me so I don't get bored right away.
Take the Magic: The Gathering 'What Color Are You?' Quiz.
I guess OP is at least right about one thing. Bad news is always louder than good news. The comments here are perfect examples for that. The way I see it, players these days are like spoiled children, taking everything for granted. They think by paying 15 dollars a month, they can ask for everything. What's worse is that ESO technically requires no subscription now, meaning some of these whiners don't even pay.
People have been crying about how evil these multimillion game companies are and how this industry is going to be ruined by them. If anything, these players are somewhat responsible for that.
Yes, he was wrong. We are customers, not empolyees. We pay money to use a system they provide. And it is not the customers duty to test the product, he pays for!
Both. And: I spend my free time enjoying a game. I don't have enough of this free time to 'work' for a game company as a volunteer.
You cannot expect proper testing from your playerbase. Some might help if they have forum accounts and so on, but the vast majority will never touch/download a PTR, and if they do it's to test new abilities, buffs/nerfs and how they impact their chars, respec for free if it costs in the game and so on.
The conclusion? Don't be cheap - hire QA personnel.
Interesting ...
You stay sassy!
There once was a time when MMORPG companies (Turbine, Sony, ETC) used to go out of their way to make sure that players helped to test the games. They would have premade characters with the proper equipment waiting for you. Give out rewards. Hold special events on the PTS with GMs...they GAVE REASONS to help find bugs.
To those here saying a good company would "enter BS here and find the bugs themselves"...it IS NOT POSSIBLE. Do you even MMO? They cannot even be released bug free after 1 1/2 years of Alpha/Beta testing...there are too many variables in players systems as well as game features/code that could have a flaw in them.
Hell, even single player games made for a console (set hardware) are released with bugs.
Players are the very reason why the genre has been in a downward spiral. To most of them getting into a beta test is a free trail period...they do NO testing at all, they dont try to find bugs and they dont report them when something wrong happens. Some, even go out of their way to hide the bugs they find so they can use them as exploits after release.
The players are responsible...everyone is ALWAYS responsible for their own actions AND inaction's.
"People who tell you youre awesome are useless. No, dangerous.
They are worse than useless because you want to believe them. They will defend you against critiques that are valid. They will seduce you into believing you are done learning, or into thinking that your work is better than it actually is." ~Raph Koster
http://www.raphkoster.com/2013/10/14/on-getting-criticism/