Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Fuzzy Avatars Solved! Please re-upload your avatar if it was fuzzy!

Beta testing again just small part of the world?

ElikalElikal ValhallaPosts: 8,063Member

Mind ya, it is just hearsay. AND it is stuff the developers said, like the beta test involves testing selected parts. After SWTOR beta I am VERY skeptical against this sort of testing. It is a bit a red flag to me, if a game is not tested as a whole, as it is, but chunk by chunk, and only selected stuff. Actually I think MMOs would serve best if they had really open, vast betas with REAL feedback. And not those pre-made questionaires.

Just some impression. But I get the feeling them companies only beta test selected parts and only when most stuff is decided. Hurm...

People don't ask questions to get answers - they ask questions to show how smart they are. - Dogbert

Comments

  • LoktofeitLoktofeit Stone Mountain, GAPosts: 13,643Member Uncommon
    Originally posted by Elikal

    Mind ya, it is just hearsay. AND it is stuff the developers said, like the beta test involves testing selected parts. After SWTOR beta I am VERY skeptical against this sort of testing. It is a bit a red flag to me, if a game is not tested as a whole, as it is, but chunk by chunk, and only selected stuff. Actually I think MMOs would serve best if they had really open, vast betas with REAL feedback. And not those pre-made questionaires.

    Just some impression. But I get the feeling them companies only beta test selected parts and only when most stuff is decided. Hurm...

    It all depends on what they are testing.

    There isn't a "right" or "wrong" way to play, if you want to use a screwdriver to put nails into wood, have at it, simply don't complain when the guy next to you with the hammer is doing it much better and easier. - Allein
    "Graphics are often supplied by Engines that (some) MMORPG's are built in" - Spuffyre

  • asrlohzasrlohz UddevallaPosts: 645Member

    That's mainly what beta tests are good for. If you focus all beta testers on a step by step basis more bugs and weaknesses will be revealed and fixed. You are thinking about open beta tests which allows you to play through the whole game (mostly) if you pre order it. That's hardly even beta testing. It's marketing.

    image
  • Whiskey_SamWhiskey_Sam Lynchburg, VAPosts: 294Member Uncommon
    Originally posted by Elikal

    Mind ya, it is just hearsay. AND it is stuff the developers said, like the beta test involves testing selected parts. After SWTOR beta I am VERY skeptical against this sort of testing. It is a bit a red flag to me, if a game is not tested as a whole, as it is, but chunk by chunk, and only selected stuff. Actually I think MMOs would serve best if they had really open, vast betas with REAL feedback. And not those pre-made questionaires.

    Just some impression. But I get the feeling them companies only beta test selected parts and only when most stuff is decided. Hurm...

    I've been doing software testing for a living for 15 years, and there are valid reasons to do some of the testing part by part.  But you are to some degree correct that there has to also be a portion of testing that is comprehensive to make sure all of those parts don't blow up when they're run together.

    ___________________________
    Have flask; will travel.

  • svannsvann san jose, CAPosts: 1,638Member Uncommon

    "Hearsay....skeptical...."

     

    As well you should be.

  • SovrathSovrath Boston Area, MAPosts: 18,453Member Uncommon
    Originally posted by Elikal

    Mind ya, it is just hearsay. AND it is stuff the developers said, like the beta test involves testing selected parts. After SWTOR beta I am VERY skeptical against this sort of testing. It is a bit a red flag to me, if a game is not tested as a whole, as it is, but chunk by chunk, and only selected stuff. Actually I think MMOs would serve best if they had really open, vast betas with REAL feedback. And not those pre-made questionaires.

    Just some impression. But I get the feeling them companies only beta test selected parts and only when most stuff is decided. Hurm...

    If the company and testers treat it as a marketing event then "yes" I wold be sceptical. If they are actually treating it as a beta test then I would say a focused approach to certain areas seems preferable.

  • keithiankeithian Los Angeles, CAPosts: 3,042Member Uncommon
    Originally posted by Sovrath
    Originally posted by Elikal

    Mind ya, it is just hearsay. AND it is stuff the developers said, like the beta test involves testing selected parts. After SWTOR beta I am VERY skeptical against this sort of testing. It is a bit a red flag to me, if a game is not tested as a whole, as it is, but chunk by chunk, and only selected stuff. Actually I think MMOs would serve best if they had really open, vast betas with REAL feedback. And not those pre-made questionaires.

    Just some impression. But I get the feeling them companies only beta test selected parts and only when most stuff is decided. Hurm...

    If the company and testers treat it as a marketing event then "yes" I wold be sceptical. If they are actually treating it as a beta test then I would say a focused approach to certain areas seems preferable.

    I agree, as long as they eventually test every area in the Closed Beta which I have no reason to believe they won't by the time Open Beta roles around. Elikal may be referring to Patch Notes he may have seen during the Stress Test or leaked videos only showing the same areas over and over again. I can't comment due to the NDA lol.

    There Is Always Hope!

  • GorweGorwe ZagrebPosts: 2,472Member Uncommon
    What I am worried about is that their "stress" tests...aren't. They are more like extended beta than anything. If you want to stress the servers, just open the Flood Gates and whack at it! Not by doing these selective test nonsense.

    When game devs are acting like Inquisitors, it immediately raises that red flag for me. Look, I learned from that trainwreck of Mythic's(WAR).
  • SovrathSovrath Boston Area, MAPosts: 18,453Member Uncommon
    Originally posted by Gorwe
    What I am worried about is that their "stress" tests...aren't. They are more like extended beta than anything. If you want to stress the servers, just open the Flood Gates and whack at it! Not by doing these selective test nonsense.

     

    Can someone who is knowledgeable about this make a comment?

     

    My layman's, completely non-technical guess is that different areas are going to take different stress loads, er "differently".

    by building  up areas, examining the date and slowly adding more people, one might understand not only how particular areas handle population but there could be a possible large consideration when all these areas are running at once. Something that can be looked at when the minute details of each area are known.

    However, I'd love to hear an actual educated and experienced opinion on this.

  • IselinIselin Vancouver, BCPosts: 5,609Member Uncommon

    There are all kinds of different levels of testing. The problem you're having is trying to find a single meaning for a vague general term. "Beta Stress Testing" can be a myriad of different types of tests from individual zone capacity, to data base performance, server performance, etc. It can mean a piece is being stressed or all is.

    The type of whole meal deal stress testing you're talking about typically happens very, very late in the development cycle in the last couple of months before release.

    There are also different ongoing levels of testing happening simultaneously. As far as I know, the waves of weekend testers that have been invited so far have not tested the AvA in Cyrodiil. But some testers have been there testing the capacity of the engine to handle large numbers and some also have access to the whole enchilada that has been developed to date... someone is doing that beta testing, but it's not the hordes that are allowed in on weekends every now and then.

    Once upon a time if you were in the beta you were in all of the beta every day, all of the time for as long as it took. That still happens but in addition to that, there are other larger groups of special purpose testers that are allowed in occasionally.

    The problem is semantics: all of those different things (plus additional pure marketing events most companies do to generate buzz after the NDA is lifted)  are called "beta."

  • YilelienYilelien Vacaville, CAPosts: 322Member
    Most beta testers. Arnt doing any testing. They just want to play the game for free. By keeping the test to a certain area or class's etc they are able to get alot better feed back. And actually test.
  • Yyrkoon_PoMYyrkoon_PoM Reseda, CAPosts: 150Member
    Originally posted by Yilelien
    Most beta testers. Arnt doing any testing. They just want to play the game for free. By keeping the test to a certain area or class's etc they are able to get alot better feed back. And actually test.

    I agree 100%.

     

    I would guess that 8 out of 10 beta testers never submit a bug report.  As stated by a few other posters, alot of beta testing is meant to stress different things at different times so that programmers and QA engineers can look for things like race conditions, zone failures, CTDs or other single points of failure with live data in real time.

     

    Funniest thing I have seen during a open beta test was a team of bots testing their automated tools out.

  • GorweGorwe ZagrebPosts: 2,472Member Uncommon
    Oh I just prefer to whack two flies with one strike. To market the game as well as stress test it. Simply fabulous eh?

    And yes, I agree with Iselin 100%. The problem is indeed in semantics because everything from the bugtesting to the prelaunch open servers is called "beta" these days. To add salt to the wound, "stress testing" is also as ambiguous as stress testing everything from a single dungeon/RvR map to stress testing the whole server. I really hate these murky details.
  • GrummusGrummus Nantucket, RIPosts: 125Member
    Chunk testing, or forcing beta players into content that's of unusual quality compared to the rest is how Funcom conned everyone into buying Age of Conan.
  • SiugSiug TallinnPosts: 1,236Member Uncommon
    After getting burnt too many times I think that the best remedy is not to preorder anything anymore. Just wait a month or two and by then you know exactly what the game really is. 
Sign In or Register to comment.