Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

[Column] General: Beta Do It Right

SBFordSBFord Former Associate EditorMember LegendaryPosts: 33,129

In the "old days" (relatively speaking, of course), beta tests in games were literally to help developers find and squash bugs. Any more, beta tests seem to mainly be about generating hype. In today's Devil's Advocate, we take a look at beta testing and offer a few thoughts. See what we've got to say before chronicling your beta experiences in the comments.

While general play is alright for an MMO beta test, you can do so much more if you’re willing to engage in erratic in-game behavior.  The premise of a beta test is to find ways to break the game, so that the issue you’ve found can be remedied prior to release. I am, of course,  assuming the developer isn’t going to saddle you with a horribly unfinished mess and then leave you to stumble madly in ignorance. 

Read more of Victor Barreiro Jr.'s The Devil's Advocate: Beta Do It Right.

image


¯\_(ツ)_/¯ 


«1

Comments

  • BattlerockBattlerock Member CommonPosts: 1,393
    I was just thinking about beta as hype generators the other day. It feels like more people are playing. Betas then actual full games. Personally and as you make mention I play beta just to see if I might like the game. These games that just stay in beta and now we have pay for beta, I just roll my eyes. Neverwinter and firefall beta unlimited and never ending to the point im tired of playing before it even releases or something bettwr comes along like Destiny so I wont even look at firefall now
  • NibsNibs Member UncommonPosts: 287

    The problem is that once you've been in a few betas you realise that your feedback is almost certainly going to be ignored.

    The devs know better. The testers are just plain wrong (or ignored). Players don't know what they really want. Any one of a million other nonsense responses that I've seen devs post on beta forums.

    Those 50 bugs you reported - still there at release. In some cases they're still there months into release.

    As a tester, sooner or later, you realise that the devs are too far down whatever path they've taken to turn back now.

    UNLESS you get invited into the very early stages of beta, preferably into alpha, your opinion is practically worthless. You are basically there to stress test the system. At least that's the feeling I've been left with.

    Alphas tend to be populated with 'friends & family'. People who quite often aren't even the target audience and so any feedback that they give may as well be ignored. Unfortunately devs don't want to be seen to ignore their friends & family (who does?) and so the only non-dev people that get into the 'dev process' early enough to actually make a difference are quite possibly the worst people to be put in such a situation: Friends and family that seemingly know nothing about games. By the time most people get into a beta all the design decisions are so set in stone that they are not going change.

    I test business software as part of my job. I know how mind numbingly dull true testing is. I know who to find, replicate and document bugs. But when I've managed to get into game betas, even at fairly early stages, I make reports. I wait. I know fixes don't happen over night. I know the devs have 2, 3 or more builds already squirrelled away before my bugs even have a chance of being squashed. So I wait. 8 weeks, 12 weeks later and they're still there. Sometimes they've even spread.

    You offer (I think) well rounded suggestions on how to improve sub-system X or activity Y. You get mostly agreement on the beta forums. The devs do the opposite because some super secret 'focus group' disagreed with you and all the other testers.

    Honestly, what's the point?

    So now, the devs don't care what I have to say, I don't care enough to bother saying it in the first place.

  • RyowulfRyowulf Member UncommonPosts: 664
    Of all the betas I have been in, I could count on one hand the ones that changed greatly from when I first entered till they went live.

    Unless you are in alpha or really early closed beta I don't think the game is going to change all that much from its core. SO heck yeah you should judge whether you will be pre-ordering a game based on its beta.

    On the other hand never say never.  Games can change for the better.  Heck going live is just more beta for the first 4-6 months anyway, but be a smart consumer. Don't rush to buy, but don't try to destroy the game just because you weren't happy with it in beta.

    About an NA normally I agree that you should keep quite, but what about when a game asks for a pre-order?  What about a game like WARZ that was making claims about their game that were all hype?  As someone playing a game is it right to say nothing and let people be fooled and their money more or less stolen?
  • PyrateLVPyrateLV Member CommonPosts: 1,096
    Originally posted by SBFord

    In the "old days" (relatively speaking, of course), beta tests in games were literally to help developers find and squash bugs. Any more, beta tests seem to mainly be about generating hype.

    I couldnt agree with this statement more

    Tried: EQ2 - AC - EU - HZ - TR - MxO - TTO - WURM - SL - VG:SoH - PotBS - PS - AoC - WAR - DDO - SWTOR
    Played: UO - EQ1 - AO - DAoC - NC - CoH/CoV - SWG - WoW - EVE - AA - LotRO - DFO - STO - FE - MO - RIFT
    Playing: Skyrim
    Following: The Repopulation
    I want a Virtual World, not just a Game.
    ITS TOO HARD! - Matt Firor (ZeniMax)

  • asmkm22asmkm22 Member Posts: 1,788

    I disagree with Beta's not being the final product.  Based on experience, practically no significant  changes are made to the game during Beta testing.  You can have the entire beta community telling the dev's that something isn't good, or that a missing feature *needs* to be added, and they'll just release the game anyway.

    You make me like charity

  • marganculosmarganculos Member UncommonPosts: 334
    Originally posted by asmkm22

    I disagree with Beta's not being the final product.  Based on experience, practically no significant  changes are made to the game during Beta testing.  You can have the entire beta community telling the dev's that something isn't good, or that a missing feature *needs* to be added, and they'll just release the game anyway.

    +1, so true... i don't remember last beta which was significantly different than released products... and im betatesting since ....  umm i even remember ...  September 1997 .... last day of the original beta test of UO!

  • GrinnzGrinnz Member UncommonPosts: 312
    No one finishes games anymore at least in a f2p sense, they just are in perpetual beta stage....
  • zellmerzellmer Member UncommonPosts: 442

    So many direct ports of korean games and the like end up being EXACTLY the same, beta or not.

     

     

  • sketocafesketocafe Member UncommonPosts: 950

    Seems to me that growing perception isn't that far off the mark. Have companies not been using betas to fuel hype in recent years?

     

  • FalcomithFalcomith Member UncommonPosts: 830

    I have been beta testing games, mostly mmo's, for almost 10 years. And in the last 3 to 5 years the true testing is done in the alpha betas in which the developers learn and listen to the testers.

    Once you hit closed beta its more of the developer testing hardware and server capabilities on there end and the testers are just there to have a population to help stress the server. On occasion if enough people run into that game breaking bug that was missed in alpha, it gets looked into.

    Open beta today is just a way for them to help hype the game through community advertising and get those last minute pre-orders as NDA is usually lifted at that point.

    Dont get me wrong, not all developers follow the formula above. But more and more as time goes on look for that quick buck in which usually the game suffers because of it. But then there is also the Publishers who are to blame since they hold the funding for the developers in there hand. But thats another story.

     

  • WightyWighty Member UncommonPosts: 699
    Originally posted by PyrateLV
    Originally posted by SBFord

    In the "old days" (relatively speaking, of course), beta tests in games were literally to help developers find and squash bugs. Any more, beta tests seem to mainly be about generating hype.

    I couldnt agree with this statement more

    I second that... Despite rabid fanboism that always pulls the "but it's beta" card the truth is (specially for the "limited betas" you are playing a release candidate and there isn't going to be much more changed except for maybe some stabillity optimization.

     

    All this hype crap has to stop...

    What are your other Hobbies?

    Gaming is Dirt Cheap compared to this...

  • nennafirnennafir Member UncommonPosts: 313

    I've been in at least a dozen MMO betas, and probably closer to 2 dozen.

    The state of the game in beta is the state of the game at release.  I am honestly surprised you listed this as a myth.

    The game will be buggy in beta, and the devs will fix some of the bugs, but then at release they will at the last minute modify something or other and there will be the general server stress.

    So, after having been in beta for all of these games AND continued actually to play the games (since I am bored and what else am I going to do) I have to say:  State of the game at beta (either open or closed) = state of the game at release (more or less.)

  • ariasaitchoariasaitcho Member UncommonPosts: 112

    From the F2P perspective:

    Open and closed Betas are more for stress testing servers than for "bug quashing". Yes closed betas are more for bug quashing but I've been in many closed and bugs that were there in closed are still there a year (or more) after release; and are sometimes considered "features"! lol

     

    I've also been in one alpha, I was honored and that game has yet to come to beta yet. The devs are seriously reworking the game, changing some aspects of game play. I understand that a major overhaul takes time.

     

    Lastly, +1 on the bit about non disclosure. It's about integrety. If you seriously think the game is junk based on how bad the beta was, say so. But giving out details of what was wrong and how to do replecate bug X somewhere other than the official forum is wrong.

    image
  • skamperskamper Member UncommonPosts: 252
    You seriously blame players? Developers are pushing "beta" as a pre release demo now.
  • MueslinatorMueslinator Member Posts: 78

    Betas used as hype machines? I would tend to agree. I know a few 'Betas' I participated in, and it felt like I was punching clouds. Almost all of the feedback of the testers (judging from forum posts and 'stories swapped') seemed to be ignored for months, testers belittled, and you know... devs just didn't seem to give a shoot that there were thousands of unpaid players testing their game out, trying to be valuable beta testers. Even very glaring bugs were not fixed. Yeah, I get production cycles and that some things take time. But when a bug persists that locks everyone in the starting zone into release... that's plain lazy.

     

    Thankfully I've had other Betas as well, where feedback was indeed heard, where the devs even said "mechanic X has often been mentioned by testers. We're going to test it this build - please leave your feedback if the new or the old system worked better, and why".

    You felt like your time spent at Beta was not time spent in vain, but that you really did your part to improve the game.

     

    When I beta test, I work. Sure, it's fun work (which is why I do it on my own time), but I am not gaming in that time. I am testing. And I am sinking a lot of man-hours into that testing. And yeah, for me as voluntary and unpaid Beta tester it is important to know that I am not just burning time when I fill out bug reports, feedback questionnaires or write forum posts.

     

    And that is why I have come to disdain 'PR Betas'. Because they are nothing to do with testing, only with generating buzz. I don't do buzz. We have editors for that.

  • cyriciancyrician Member UncommonPosts: 189

    Well  how about charging people to play beta like sooooooooo many devs are doing these days 69$ to get into the beta for neverwinter noo way in hell.

     

    That being said I did buy a founders pack for both defience and firefall. Loving firefall at the moment can play when I want !!! And it only cost me 19$. 

     

    But I agree for the most part beta,s have lost there way and it's because people and company's are greedy and the company's will charge asking as the people pay.

     

    Current games;
    Star treck online
    Rift
    Eve online
    Firefall

  • jtcgsjtcgs Member Posts: 1,777
    Originally posted by cyrician

    Well  how about charging people to play beta like sooooooooo many devs are doing these days 69$ to get into the beta for neverwinter noo way in hell. 

    That being said I did buy a founders pack for both defience and firefall. Loving firefall at the moment can play when I want !!! And it only cost me 19$. 

     So...you hate that they did it with Neverwinter but not Defience? Wut? Dont know about anyone else but I paid with Neverwinter for the Drow, Mount and shop currency, beta access made NO difference at all.

    “I hope we shall crush...in its birth the aristocracy of our moneyed corporations, which dare already to challenge our government to a trial of strength and bid defiance to the laws of our country." ~Thomes Jefferson

  • troublmakertroublmaker Member Posts: 337
    In my experience developers generally don't care about any bugs you find and they remain in the finished product.  Oddly enough the only devs I've ever had introduce any of my suggestions was Trion Worlds of which I alphaed Rift and End of Nations (last summer).  Other than that most of what you get in open beta is what will be available at launch.
  • jdlamson75jdlamson75 Member UncommonPosts: 1,010

    Victor:

     

    Most of the betas that mass audiences are getting access to are in the final stages, and become available to let folks get a glimpse, fix a few bugs, and stress test.  By this stage, games aren't going to change that much, so what you're getting basically IS the finished product.  It's unfortunate that those who stop playing because the game isn't fun annoy you. 

     

    I, for one, won't keep playing a released game if I'm not having fun with it.  Why continue in a beta if it's not fun?

  • MaxJacMaxJac Member Posts: 185
    Beta is so close to release that MMO's don't have time to make fundamental changes prior to launch. The beta as it is now is what you can expect at launch. The issue is that real testing with players needs to happen at an early stage. Let them keep their beta as a glorified marketing tactic. I suggest involving a selection of players as early on as alpha testing.
  • MickleMickle Member UncommonPosts: 127
    There is no such thing as a Beta anymore.  They go from Alpha to Stress Test and hope to fix the bugs after release.  Then they find new problems when they go live and never get back to fixing the bugs.  Most games out today still have bugs that were found during Alpha.  You can read the forums as see that.
  • HedeonHedeon Member UncommonPosts: 997

    very few of the games Ive "beta tested" have been actual beta´s, but somehow 5 years ago people would get annoyed when pointing it out, but have seen alot say this and that game sux during beta and because of that wont ever buy the game, and I believe this to be the main reason general players wont be allowed to test games anymore before it is 95%+ done....

    would even go as far as claim RIFT intentionally left useless systems in the game, to let people say they wanted it changed, but then that is the place Ive been the most amazed how pigheaded people can be though, in RIFT "beta" there were huuge flaws afew months before release, but people would actually defend them and claim they wanted them there, still the developers corrected them, and people afterwards can claim TRION listened to their beta testers and you could actually see the huge improvements afterwards.

    think the last game Ive been in a real beta for, without having to pay, were the chronicles of spellborn...ofc there were FFXIV as well, but dont think they actually had thought they needed to change anything, just needed to test the servers during heavy use, which ofc is important too

  • sanshi44sanshi44 Member UncommonPosts: 1,187
    One thing the devs do that they realy shouldnt is they set a release date and then go into beta, This means they only have X amount of time period to squash all the bugs and fix what needs to be fixed however 95% of the time there more bugs than they could posible fix in that time limit so they need to either release the game on time with bugs or push the date back, and pushing the date back doesnt usualy work to well so they just release it. Devs need to start to do a decent beta test before they even put a release date on the game so they can give themselfs a more reasonable amount of time to fix bugs/fixes.
  • WickedjellyWickedjelly Member Posts: 4,990

    Considering companies push their betas now as marketing tools it isn't a real surprise players are following this trend.

    You market something that way and use it as a mechanism to try and get people to preorder your game this is what you get. Not real fucking surprising nor can I blame some for feeling that way.

    1. For god's sake mmo gamers, enough with the analogies. They're unnecessary and your comparisons are terrible, dissimilar, and illogical.

    2. To posters feeling the need to state how f2p really isn't f2p: Players understand the concept. You aren't privy to some secret the rest are missing. You're embarrassing yourself.

    3. Yes, Cpt. Obvious, we're not industry experts. Now run along and let the big people use the forums for their purpose.

  • dmm02dmm02 Member UncommonPosts: 40

    simple fix : Alpha test > Beta test > Foundation Release (for those that pre-order and yada yada)

    Companies need to differentiate beta from foundation and use the Beta stage for what it is

Sign In or Register to comment.