Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
Live Forum Q&A Ryan Elam - PlanetSide 2
PS2_RElam
PlanetSide 2 Technical DirectorMember Posts: 11
Hello folks!
My name is Ryan Elam, and I am the Technical Director for PlanetSide 2.
I've been coding for 35 years, and making games for most of that time. I worked on one of the first multiplayer RPGs ever, and later on games such as EverQuest, Vanguard, FreeRealms, Clone Wars, and helped turn Planetside Next into Planetside 2. My greatest skill is the ability to luck into an amazing team of engineers, and then watch them make improbable things happen in an impossible timeframe.
PlanetSide 2 is an extremely ambitious MMOFPS and an awesome challenge to work on. Daily we are met with unique problems to solve and I go to bed most nights proud of what we've been able to accomplish, the other nights I don't go to bed. *grin*
I'm here today to talk about our most recent mission, Operation: Make Faster Game (OMFG). We have known since before Planetside 2 shipped that we had some things we could do to improve the frame rate, but they were going to be a significant time sink and therefore difficult to do in a live environment. A few months ago, we got the call from smed to put the whole team towards doing whatever it took to get performance up, and since then that's what we've been doing. We JUST put the first phase of this onto our Public Test Server (PTS) and a vast majority of players are seeing signifant increases in FPS!
I will say that I'd rather not try and answer the question "How many FPS will I get with my SmartThinker PiCore CPU with a GlowForce HeatMaker 9000 GPU?" I know, in the end, that's the most important thing, your personal performance... Unfortunately there are not only a nearly infinite number of combinations of hardware out there, but there is also many different scenarios in-game with which to measure your performance. THAT said, this is an AMA, so don't be shy!
Finally, to bring this longwinded introduction to a close, my sincere and humble thanks to the MMORPG community for their hard work and for allowing me to jump on and answer you guys' questions!
So, for the next hour or so, in the appropriate vernacular, "Shoot".
Comments
I did not have a question since I do not play PS2 but I would like to welcome you to the forums. I'm glad we are getting more developer love these days with these AMAs!
Hey credit to the MMORPG staff! As busy as things are getting the OMFG:P1 patch to Live, when they came to me asking for my time... and they showed me the picture they had of me and the alpaca... I couldn't say no!
Is this where we post questions?
I want to ask about networking. From the start of the game it has been possible to cut off your upload temporarily and disappear from other people's clients. You could for example be in a spawn room, shut off your upload, and then run to the enemy Sunderer and re-enable your upload. You then catch back up with the server in your new location, and to the enemies you just "appear out of thin air next to their Sunderer". Drop a few tank mine's as soon as your incoming packets catch up with your client, throw a grenade, and then BOOM the enemy Sunderer is dead with no chance for the enemy to defend it.
Another example is packet re-ordering or delaying. Or even packet filtering. You can for example drop certain packets that allow's you to get a few seconds ahead of where other players think you are, without making you "disappear" on their clients. So you can for example shoot someone a few seconds before you appear on their client even though you do appear, you have actually already shot them so you basically insta gib them. Or they receive damage and see you shooting them at the same time as you appear to them.
You can also go even further with it and kill players while in a desynched state while never appearing to them and they will drop dead as soon as the packets are sent back in "insta gib" fashion.
Noone has ever been banned for doing this despite numerous reports.
What is the problem SOE is facing in their network - client server model that is stopping these issues from being fixed?
Another issue about networking is that the game has some bugs where it forces router or internet restarts, no other game I have played has created problems like this.
My questions mostly revolve around the roadmap and subscriptions.
1 when are we likely to see the new swamp continent?
2 when are we likely to see the joining of the 4 continents. Also how will this work, will it be simmilar to the bridge battles in original planetside?
3 any plans to add naval warfare? I think this was in the roadmap, but can't remember.
4 are you considering some more extras for subscribers to encourage subscriptions? Some free cosmetic items every month, or subscriber based titles or something?
5 with EQN coming up, will there be anything extra or some sort of discount for people that want to play both planetside 2 and everquest next?
Not very related but, I have had this question for a while and thought I should try anyway to ask....do you think PS2 will ever introduce more face/hair options for your character? I know me and some others can't get totally connected to our players because everyone looks the same that way. I know we're in the minority. No worries if can't answer!
Thanks:)
What sort of limitations or obstacles are you guys encountering to add sea warfare into the game?
And if it isn't planned, or isn't on the team's radar. Why is that?
Also, why do the menus and general UI load very slowly? Even on the Optimization PTS patch.
And thank you for your hard work. I should order a bunch of cupcakes again for the offices(If you got one during PS2 launch).
-=/I make those video things at: www.youtube.com/ZoranTheBear\=-
We have an entire team devoted to anti-hacking, and they ban a large number of people every day. We don't spend significant effort explaining how we catch these folks, for the same reasons that banks don't show you how their security system works. Some of what you said is merely rumor, as our packet structure doesn't allow for some of what you said to happen. Other parts are very easily detected and those doing it are being banned daily, sometimes within seconds of doing it.
As far as needing to reboot your router, I can't discuss specifically what is happening in those cases, it would be a very long post indeed. But we do have ongoing issues with overloading bandwidth and it's another set of optimizations we want to work on, and that is a likely case for a lot of folks who get "long-lasting lag". I also owe you guys a method of visualizing how your network is currently performing, so you can "see" when things are bad, rather than just having to guess. *grin* Lots to do!
Trust me, you're welcome. MMO developers like to work in Hard Mode.
1) Soon. We really want you to get it, but it's not ready yet.
2) We're still working out specifics, stay tuned. Lots of exciting changes in the next six months!
3) Most of us love the idea of expanding our warfare into new dimensions. We just have to work out the logistics and priorities. Sea is not permanently off the table.
4) We eat, breath, and sleep PS2. We never stop considering.
5) I can't speak to specific discounts, but we've had cross-promotions in the past. I'm personally very excited for what EQN is going to add to the game space!
T-Ray would love to have more customization options! It comes down to resources. We've put so much into the game and every time a new customization exists, it has to be loaded for everyone who can see you. For now, the priority is on things that can be seen for a further distance, like camos and decals and customized armor and helmets and etc. etc. So art would love to add more, I'm just stingy. Still, these games last decades, so very little is off the table permanently.
TL;DR Yes, we will likely someday introduce more options in the future.
Our main limitation to doing sea warfare is time. Imagine there is a list about 400 miles long of things to do, and only enough time to do ten miles per year. We have to prioritize!
We have some UI optimizations in the works to help with UI loading time, but they're also a significant update to get done. Stay tuned!
Hi Ryan,
Have you guys/gals given any thought to creating a mentoring program for new players? I know much was added since I first set foot on Auraxis, (VR Training etc.), but I wonder if a mentoring program would help both new players and vets who might like a change of pace?
Thanks.
How many more O:MFG phases do you expect?
Roughly how long will these phases take to finish?
What kind of fps increases do you expect from them if you compare them to phase 1(just to give me a rough idea)?
What CPUs will they affect the most?
What kind of changes besides the continent revamps will there be in the next phases and what performance problems will they address?
Are any new vehicles (preferably aircraft) coming out in the near future?
Thanks for answering our questions and good luck with the optimization!
Hey Ryan,
Thanks for taking the time to do this!
My main question is regarding server performance and how, if at all, optimization efforts are being made in that regard.
I'm really looking forward to all the optimization, but as someone who gets acceptable FPS (usually 60~ on average) already, my largest frustrations lie in server performance when too many players are in one place. This is most often the case on Indar, especially during alerts, and you'll see a range of symptoms from Medkits not working properly to all forms of healing/repairing/cloaking/jump jets/special abilities acting wonky and not being consumed or regenerated properly, finally to hit registration suffering because of it.
As a competitive/serious player, this is unfortunately unacceptable and I'm left to try to find fights on Esamir/Amerish that often don't even exist. It is especially frustrating because it is not something we as the end user can remedy by purchasing better hardware for ourselves. Will the efforts being made by "OMFG" alleviate the issues I mentioned in the above paragraph as well? If not, are there efforts being made in this regard?
Thanks so much for everything you guys do at SOE, and for listening to your community so well. Keep up the great work.
While we didn't sit down in black robes around a table made of goat horns and say "There Shalt Be Eleven Phases of the OMFG!" in a booming James Earl Jones voice, I'd venture to guess that we will have one more phase after this one to cover some other major things we just simply didn't want you to have to wait for, and then we will likely have one more after that to cover any clean up. After the third phase, there may be more just depending on what percentage of our users are getting a better game experience.
We already have some coders working on Phase 2 while others finish up Phase 1, and I expect it to take no less than another month. A lot of the work comes from introducing changes, then measuring the results of those changes, then figuring out the next change... so it is often as much R&D as it is actual code writing. This makes it difficult to determine exactly how long it will take to get done.
Phase 1 will definitely effect 4 or more core systems. This is simply because we wanted to hit the largest percentage of our player base quickly, and a large percentage of our players have quad-core or greater. Phase 2 will help out those same people, but will introduce some more benefits for those with only two cores as well. After that, we'll see.
We originally expected Phase 1 and Phase 2 to give a roughly 20-30% improvement in frame rate, each. The coders, artists, and designers have worked so hard on Phase 1 that we're seeing a nearly 40% improvement, on average. That may mean that Phase 2 is only 20%, we'll have to see as we get closer.
As far as what problems we're addressing in Phase 2, one of the top issues is visibility -- this is the code which actually determines what you see on your client in any given rendered frame. Right now in some circumstances it takes roughly 33% of your frame time. Now we can't optimize that down to 0%, but we can get it much better.
Now with new vehicles, I tell you we all love new vehicles! They're just really, really hard to make and tune. Trying to make something like the harasser feel fun and useful, it's possibly one of our toughest challenges. I know they are always coming up with new ideas and badly wish to get more vehicles in the game, but I don't believe that there is anything coming in the near future.
And finally, you are very welcome.
We constantly work on server optimizations. PlanetSide 2 is far and away more optimized on the server than any MMO I've worked on to date, but as you well know we really ask a lot of it! Sometimes we have the servers getting kicked in the jimmy by an Alert, like you said, or a player event, and we monitor all the traffic that goes on during one of those, then we try and adjust the code to fix the issues that we find. One of the things that we need to give YOU is the ability to actually see if it is that we're pushing too much data for your internet connection, or the server itself is backed up, or someone stepped on the internet pipe. We just don't have that information readily available to you yet. It's coming, though.
And thank you for your questions. We are always trying to improve our communications with our community. Without community, I'd only have Higby to talk to!
Too early to say at this stage sandman529. Most of our physics work is in software. Only specific particle effects take advantage of nVidia hardware. So there is nothing that would prevent us from running just as well on any equivalent AMD processor. That said, we never stop looking at things to improve.
I appreciate your reply! I assume you're over time here so I don't expect a follow up, and that's ok. Just wanted to say that I'm glad to hear it, though I am also referring to a very specific point where medkits start breaking and it seems like it crosses over a certain threshold. I was hoping to hear that, even though server performance is always being optimized, we may hope to see the dreaded medkit bug (and other associated issues) get fixed with one of these coming optimization phases.
Thanks again.
What is the target goal for optimization? Is it a % fps increase, or keeping the fps in a more stable range? I have heard people with powerful rigs running on High have dips to 30 fps. I know optization will be an on going process, but at what point to you feel satisfied you have done all you can?
In truely massive battles of many hundreds of players in a concentrated spot, the player draw distance is reduced for performance reasons, correct? Will this distance be increased after all these optizations? I like my epicly massive battles.
Has Player Studio had any impact on performance? Do you anticipate with the flow of a lot of new player made content the game will still run smoothly?
What major improvements will PS2 receive once you have squeezed all you can out of the "Make Faster Game" phase? New vehicles? Empire home planets?
Other INCREDIBLYAWESOME things that will start the performance balancing act over again?
Sorry if any of these are repeats. Thanks for taking time to answer these!
OMFG will not likely fix "the medkit bug", but we will be working on it over time. We actually fixed a major server performance issue as recently as GU15. Server optimizations are ongoing, but as you guessed most of those resources are on client optimizations for OMFG at the moment.
Fair enough, glad to hear it either way -- and thanks again!
If you are still hanging around to answer more questions, are you able to comment on whether we will begin to see more content patches after phase 1 of OMFG, or will it not be until the entirety of OMFG is completed (i.e. months down the road) before we begin to see content updates such as GU16, etc. again?
Just curious if the proposed schedule looks something like... OMFG Phase 1 -> Next Content Updates (ESF/Infil/Hossin/etc. AND OMFG Phase 2 being worked on in tandem, or if it will be OMFG Phase 1 -> OMFG Phase 2 -> and so on, before any major content is patched.
It's a question I get a lot from my stream viewers, so thought I'd see if I could try to get even a vague answer. Thanks again
My original goal for performance in Planetside 2 was that each player should be capable of reaching 60fps in "normal" battles and at least 30fps in "huge" battles, just changing their options to fit their rig. Higher-end PCs on High. Lower-end PCs on Low. We really want you in those "huge" battles, so 30fps has become kind of a "minimum" because an FPS game is just better at 60fps.
Our goal now is to get as high a percentage of our player base to 60fps in large battles as we possibly can. We will never attain perfection, but we'll get much closer than we are now.
If all but a tiny percentage of players get a frame rate that is fun, I believe we will all be satisfied that OMFG is done.
As far as player draw distance in large battles, we don't reduce the draw distance so much as "prioritize" the people you know about. This in effect lowers draw distance, because if you're in a 500+ man battle you won't see all 500, just those closer to you.
Player Studio has not had any negative impact on performace. We're very mindful of every asset that goes into the game.
For future improvements, check the Roadmap on Planetside2.com (obvious plug).
We are constantly evaluating what should come when, and we prioritize FUN over FAST. Hrm using your example, I think the most LIKELY scenario would be OMFGP1->OMFGP2->Content->OMFGP3.
We are currently looking at what future resources we need to finish OMFG and as devs roll of performance, many of them roll back onto features.
Eh, not the greatest answer murda, but an honest one.
Ryan.
My question is in regards to GPU driven PhysX effects. In a nut shell, why do they cause crashes so often? Is it a problem on SOE's side, issues with Nvidia? I LOVE to use them as they add so much immersion (and get me shot because I can't see through the sparks) but I can't use them without constant crashing.
A follow up to that would be, will we see any of the new PhysX based effects/systems in Planetside 2 that Nvidia recently talked about at their "Way it's meant to be played" conference (Such as FLEX, GI and Flameworks)??
Greetings Ryan, from the Myriad Apocalypse Project of Woodman!
I tried the O:MFG patch on the test server the other day. The improvement was the largest I've seen since PS2 beta - good job!
One of my Outfit friends made the point, about something I'd been thinking about too, that what exactly is the effect of all the cosmetic customizations on performance.
In an earlier reply to this thread you acknowledged there is a performance impact. So how about a toggle option in the graphics settings?
Toggle cosmetics for enemy on/off - because after all, to me they are just another roadkill for my Sunderer and I don't care what they look like.
A toggle for friendlies other then my Platoon, Squad, Outfit, Friends list - because after all, they are just a stream of XP for my deployed Sunderer AMS and I don't have a dress code for who deploys.
Thanks for reading!
TerraNovaGaming.Community
not sure if Q&A is still open but ill ask anyway.
Is there any plans on adding different modes / objective types to the persistent war in PS2? Relying only on capturing points on 3 huge continents can be frustrating. A game of this magnitude can benefit from various modes a lot more than the rest of the shooters out there.