First off, I completely disagree with the idea that the responsibility to test new content is on the community. That's just insane, especially given the size of Bioware and Zenimax. There's no good reason, outside of wanting to cut costs, that they don't hire QA testers.
That being said, I'd definitely play on the PTS to HELP squash bugs or find issues. But in a game like ESO, that also means ruining the content for myself when it goes live. I'd rather experience it for the first time on my main.
I used to do PTS quite a bit on Trove, because it didn't rely heavily on the experience of the world and story. If it had, I would of avoided the PTS there as well.
So I think that's the biggest issue facing ESO's PTS. Nobody wants to ruin the story for themselves. That's like 90% of what ESO offers.
Well you can get a lot of people to beta test but most don't do any testing as such. They hardly make reports and most just play to check the game out .Then when the game comes out the population willing to play on the test servers for patches and such I presume has become non existent.
I remember playing on the test server in Everquest where you could have a permanent character and there was an actual community and people who only played on that server in spite of the risk that they lose their character.
Nowadays companies just let the patches go out and then make the adjustments when the paying players scream blue murder.
No offense but multi million pound mmo's shouldn't rely on their customers to 'test' things, maybe they should use some of that money and actully invest in a decent size QA team.
There are things you cannot test internally. Having a public test version is common sight in software development...
First off, I completely disagree with the idea that the responsibility to test new content is on the community. That's just insane, especially given the size of Bioware and Zenimax. There's no good reason, outside of wanting to cut costs, that they don't hire QA testers.
Trying learning a few things about software, programming, hardware and the many other things involved in this, then come back and post again.
As I said in my first post, it isnt even possible to work out all of the bugs in a single player game. It isnt possible to bring up a server cluster with with Redhat, which is arguably the most fleshed out version of Linux and have ZERO flaws/issues popping up once users get involved...do you know anything about the other side of this issue? MMOs are massive in design and scope and that is without the 100s of thousands of different hardware configurations players will be using.
Hell man, even software companies like Microsoft, Adobe and countless others have "users" testing their software like Photoshop and Office because having a massive amount of people looking for bugs is NEEDED.
"People who tell you youre awesome are useless. No, dangerous.
They are worse than useless because you want to believe them. They will defend you against critiques that are valid. They will seduce you into believing you are done learning, or into thinking that your work is better than it actually is." ~Raph Koster http://www.raphkoster.com/2013/10/14/on-getting-criticism/
Couple of things. First, if the SWTOR issue was was back from when they were introducing housing, they got plenty of feedback and ignored it. I was one of them. Second, some games have a decent PTS population when something major is going on. When an expansion is on PTS for Rift, there's a fairly decent number of players there. Part of it is because they actively encourage testing. When I've provided feedback there, it gets acknowledged and taken care of. Devs are also active on PTS during those times. I haven't checked out ESO's PTS largely because of the size of the download.
I'm a software developer and I've worked on everything from small websites to large military projects. I've also worked in QA within the gaming industry for a AAA developer.
I haven't played ESO except for beta.
What I can tell you is that the community DOES have a very important part with regards to helping test large games and this is unavoidable unless we want the cost of games to increase further.
The reason for this is because game developers are not in control of where their software ends up or how their software is used. There are millions of possible hardware / software configurations before you even start on differing ways to play.
I'll use a hypothetical example. Lets says ESO takes 300 hours to play through all the scripted content and that there are 1million hardware configurations to test on. That means 300m man-hours to test, at $14ph that is $4.2billion to test. Then you need to retest some of those playthroughs....then you need un-natural testing (collision detection across 100% of game, combat testing, AI testing etc etc)
In short, it is not financially viable for a game company to 100% test their game. The best they can do is 100% test the functionality (black box testing) and then make a best effort at the rest of it. Inevitably though, they'll miss something, somewhere and this is where the community comes in.
It is the developers job to incentivise us to test, however, and maybe this is where Zenimax is failing.
First off, I completely disagree with the idea that the responsibility to test new content is on the community. That's just insane, especially given the size of Bioware and Zenimax. There's no good reason, outside of wanting to cut costs, that they don't hire QA testers.
Trying learning a few things about software, programming, hardware and the many other things involved in this, then come back and post again.
As I said in my first post, it isnt even possible to work out all of the bugs in a single player game. It isnt possible to bring up a server cluster with with Redhat, which is arguably the most fleshed out version of Linux and have ZERO flaws/issues popping up once users get involved...do you know anything about the other side of this issue? MMOs are massive in design and scope and that is without the 100s of thousands of different hardware configurations players will be using.
Hell man, even software companies like Microsoft, Adobe and countless others have "users" testing their software like Photoshop and Office because having a massive amount of people looking for bugs is NEEDED.
And?
At the end of the day it is the responsibility of the company to test the software. Do you think e.g. Airbus and Boeing (to name just one industry) take that attitude?
QA is not just about software either.
And at the end of the day it comes down to a cost-benefit analysis.
There are multiple processes that can be used e.g. six sigma and multiple ways of doing "open testing" with "volunteers". Not just about numbers either - if you get 100k comments btw ask yourself how long will it take to go through them (allow a scant 1 min per comment to triage them say and go from there; gets time consuming real fast.)
People will help. More will help if encouraged. People being "NEEDED" - fine no issue with that. To suggest it is peoples responsibility however - nah. And I do know "a few things".
How about the devs listen to the player feedback when it's given on the PTS? I was in the very first closed Beta of ESO and played it up until a month prior to the announcement it was going B2P. I reported many bugs and problems I encountered during game play and it launched with all the bug and problems I reported anyway. I wasn't the only one either. Some of these were still present when I stopped playing.
Why should I or anyone bother testing anything when Zeni has proven time and time again that unless it's something that completely breaks the game, they aren't going to do anything about it?
Many PC players were already suckers once when we were paying a monthly sub to basically beta test the game for consoles. Now your advocating that people spend whatever time they have doing more testing for consoles, since we know that Zeni doesn't give two sh#ts about the PC crowd.
I'm not an IT Specialist, Game Developer, or Clairvoyant in real life, but like others on here, I play one on the internet.
How about the devs listen to the player feedback when it's given on the PTS?
... what makes you think they don't?
What makes you think they do? They didn't seem to listen very well when the game was in beta and had a terrible launch. As far as I know it's still the same team with the same guy in charge...
I'm not an IT Specialist, Game Developer, or Clairvoyant in real life, but like others on here, I play one on the internet.
Offer free game time in exchange for work done on the test server. All the kids who play 30hrs/week because they don't have jobs will be beating down your door.
well reading everyone reply op, better go back to zenimax and tell them it will not pass
gotta open the pocket a bit, peoples are not yet fool enough, yeah they pay for dlc, cash shop, f2p pay to win, but pays and test error for free not yet !!
What is going on in our gaming world and real world. It is becoming you pay us you help us you get to pay us. Company get's rich yet the community gets less. No more surprises when you load up that game or DLC anymore. We already seen it all.
Starting to feel bad about purchasing the game. I been waiting till DB hits to start playing full time, but I think it was a really bad idea when reading different forums and all the problems with the game and developers. It's weird you never really see a response from the developers on so many threads complaining about the game. Then it is like the last straw when I see a thread like this basically blaming the buyers of the game that it is their fault for not helping them.
Seems to me once we get to the point of blaming the gamer's for not helping make the game better it is time to allow that or those games to just die!
Sometimes it's not always about what you can see or hear but what's under the hood of a game that's most impressive. Between those thousands and thousands of lines of code, magic happens. Sometimes the most amazing feats of gaming wizardry happen without you even noticing.
Well , if it's a Free to play game growing with community sure i can test it . But a company like zenimax is too big for this. They have to trow some money for tests . ( from the money that i have paid for the game )
Worth mentioning that TESO - as it stands - is a polished game; two years ago different story - not unusual well it comes to Zenimax sadly. No one who hasn't tried it should be put off.
Like I said before, it's their game Zeni sets the rules if they really want people in test then let them provide incentives like they do to play live. Simple as that or it looks like they don't really care that much about testing because it's not a priority with the bosses.
It's a service industry and customers are not employees.
"We all do the best we can based on life experience, point of view, and our ability to believe in ourselves." - Naropa "We don't see things as they are, we see them as we are." SR Covey
Here's an idea, hire a @*#&;ing Q&A team you joke-@$$ developers. If they want to be cheap and not hire staff for testing then they need to reward players for testing and i'm not talking about some worthless in-game title either.
Incentive: "Time is Money, friend." X hours of testing (not AFK in a town, but actual activity in the relevant areas: AFK deducted from total time) earns a month sub.
It is a simple fix... Just give players an incentive to play on the PTS> For every minute you play on the PTS, you will get an experience bonus on the live server. Or, for every hour you play on the PTS, offer crown credits.. There are many ways to get people to play on the PTS. If they need more people, they just have to offer a good enough reason to have people join.
Personally, i never play on the test server, because I refuse to spoil the content until the DLC goes live. I like to enjoy the new content slowly and savor the quests, storyline and new lands to explore.
While any company should UAT any release, the issue occur when you have the wrong people UATting as well. These games are so large and cater to so many different groups, there is sure to be something missed in every single release unless the business has the resources necessary to make a huge UAT Tester base. And maybe some do. I would hope Blizzard would have 1000s of people whose job was to just UAT WoW. However, working at a fortune 5 company in the IT field has taught me that team is probably really only 50, with only some of them fully devoted to testing. And then they release the update to those who wanted the update only to hear people cry that their wasnt implemneted correctly. C'est la vie, no?
That said, I dont play on test because of wipes. If my character were guaranteed persistant, I would. Maybe
Look at the number of bugs that were reported in beta , before launch , before launch I say , that made it through (and dear god some still exist).
Then also look at the number of posts over the year plus from people noting problems with an upcoming patch/release on the PTS that still make it through.
Zenimax's problem isn't the number of testers , it's the shit quality of their own coders , or managers not letting them have the time to fix things.
The only thing more testers would do is point out more shit that's broke that they still won't fix.
Forgot to mention this is my earlier post. Yes, PTSs are necessary. I'm going to pick on Rift to explain why. One of the devs put it this way "Players will always come up with something unexpected". This was one of my gripes with GW2. There were bugs players would have caught had there been a test server. Instead, they test in-house, which means people who already know what they should be doing. They don't take into account the unexpected things that a player might do.
Different companies run their PTS differently. Neverwinter and Rift both keep their PTS up at all times more or less, and copying a character over takes minutes. SWTOR, on the other hand, only has their PTS up when they need something tested (and for SoR, it was only for a single server), and character copying takes a couple hours to a couple days.
Gobstopper3D said:
What makes you think they do? They didn't seem to listen very well when the game was in beta and had a terrible launch. As far as I know it's still the same team with the same guy in charge...
...because why would they setup a PTS in the first place? To ignore it afterwards? Unlikely.
Comments
That being said, I'd definitely play on the PTS to HELP squash bugs or find issues. But in a game like ESO, that also means ruining the content for myself when it goes live. I'd rather experience it for the first time on my main.
I used to do PTS quite a bit on Trove, because it didn't rely heavily on the experience of the world and story. If it had, I would of avoided the PTS there as well.
So I think that's the biggest issue facing ESO's PTS. Nobody wants to ruin the story for themselves. That's like 90% of what ESO offers.
I remember playing on the test server in Everquest where you could have a permanent character and there was an actual community and people who only played on that server in spite of the risk that they lose their character.
Nowadays companies just let the patches go out and then make the adjustments when the paying players scream blue murder.
As I said in my first post, it isnt even possible to work out all of the bugs in a single player game. It isnt possible to bring up a server cluster with with Redhat, which is arguably the most fleshed out version of Linux and have ZERO flaws/issues popping up once users get involved...do you know anything about the other side of this issue? MMOs are massive in design and scope and that is without the 100s of thousands of different hardware configurations players will be using.
Hell man, even software companies like Microsoft, Adobe and countless others have "users" testing their software like Photoshop and Office because having a massive amount of people looking for bugs is NEEDED.
"People who tell you youre awesome are useless. No, dangerous.
They are worse than useless because you want to believe them. They will defend you against critiques that are valid. They will seduce you into believing you are done learning, or into thinking that your work is better than it actually is." ~Raph Koster
http://www.raphkoster.com/2013/10/14/on-getting-criticism/
I haven't played ESO except for beta.
What I can tell you is that the community DOES have a very important part with regards to helping test large games and this is unavoidable unless we want the cost of games to increase further.
The reason for this is because game developers are not in control of where their software ends up or how their software is used. There are millions of possible hardware / software configurations before you even start on differing ways to play.
I'll use a hypothetical example. Lets says ESO takes 300 hours to play through all the scripted content and that there are 1million hardware configurations to test on. That means 300m man-hours to test, at $14ph that is $4.2billion to test. Then you need to retest some of those playthroughs....then you need un-natural testing (collision detection across 100% of game, combat testing, AI testing etc etc)
In short, it is not financially viable for a game company to 100% test their game. The best they can do is 100% test the functionality (black box testing) and then make a best effort at the rest of it. Inevitably though, they'll miss something, somewhere and this is where the community comes in.
It is the developers job to incentivise us to test, however, and maybe this is where Zenimax is failing.
At the end of the day it is the responsibility of the company to test the software. Do you think e.g. Airbus and Boeing (to name just one industry) take that attitude?
QA is not just about software either.
And at the end of the day it comes down to a cost-benefit analysis.
There are multiple processes that can be used e.g. six sigma and multiple ways of doing "open testing" with "volunteers". Not just about numbers either - if you get 100k comments btw ask yourself how long will it take to go through them (allow a scant 1 min per comment to triage them say and go from there; gets time consuming real fast.)
People will help. More will help if encouraged. People being "NEEDED" - fine no issue with that. To suggest it is peoples responsibility however - nah. And I do know "a few things".
Why should I or anyone bother testing anything when Zeni has proven time and time again that unless it's something that completely breaks the game, they aren't going to do anything about it?
Many PC players were already suckers once when we were paying a monthly sub to basically beta test the game for consoles. Now your advocating that people spend whatever time they have doing more testing for consoles, since we know that Zeni doesn't give two sh#ts about the PC crowd.
I'm not an IT Specialist, Game Developer, or Clairvoyant in real life, but like others on here, I play one on the internet.
I'm not an IT Specialist, Game Developer, or Clairvoyant in real life, but like others on here, I play one on the internet.
gotta open the pocket a bit, peoples are not yet fool enough, yeah they pay for dlc, cash shop, f2p pay to win, but pays and test error for free not yet !!
try again in 1 years maybe it will work !!
Starting to feel bad about purchasing the game. I been waiting till DB hits to start playing full time, but I think it was a really bad idea when reading different forums and all the problems with the game and developers. It's weird you never really see a response from the developers on so many threads complaining about the game. Then it is like the last straw when I see a thread like this basically blaming the buyers of the game that it is their fault for not helping them.
Seems to me once we get to the point of blaming the gamer's for not helping make the game better it is time to allow that or those games to just die!
Rob Manuel
It's a service industry and customers are not employees.
"We all do the best we can based on life experience, point of view, and our ability to believe in ourselves." - Naropa "We don't see things as they are, we see them as we are." SR Covey
Personally, i never play on the test server, because I refuse to spoil the content until the DLC goes live. I like to enjoy the new content slowly and savor the quests, storyline and new lands to explore.
That said, I dont play on test because of wipes. If my character were guaranteed persistant, I would. Maybe
That's just, like, my opinion, man.
Then also look at the number of posts over the year plus from people noting problems with an upcoming patch/release on the PTS that still make it through.
Zenimax's problem isn't the number of testers , it's the shit quality of their own coders , or managers not letting them have the time to fix things.
The only thing more testers would do is point out more shit that's broke that they still won't fix.
Different companies run their PTS differently. Neverwinter and Rift both keep their PTS up at all times more or less, and copying a character over takes minutes. SWTOR, on the other hand, only has their PTS up when they need something tested (and for SoR, it was only for a single server), and character copying takes a couple hours to a couple days.