MMO Development Lesson #18
Quality Assurance is not a four letter word. Test is, but that is beside the point. The point is, QA is vital to the success of your MMO. These games are simply too big for you to catch everything on the first pass. Never underestimate the power and passion of QA testers–and don’t treat them like crap just because they aren’t “on the dev team.” I prefer to consider anyone who makes a game better part of the dev team. Utilize and respect your QA team, and do the very same for those dedicated Testers who are part of your community.
Sometimes one is lucky to have QA testers at all! But I heartily second your advice not to treat QA like crap. That way lies badness. In fact, the worst release debacles I’ve ever seen have come from places where people got all prissy and exclusive with regard to who was “on the dev team” and who wasn’t.
We were actually called “Quality Assurance/Dev Support”, which was nice. There weren’t really any walls between any of the groups, and every developer was open to constructive feedback.
They were also very up front about the job: “You are the last few to join the team, and when the game ships in [X] months, you will most likely be let go. Do you still want the job?” And when it was time to go, they told us two weeks in advance, wrote me an awesome letter of recommendation, and encouraged us to spend at least an hour a day looking for jobs. They made the best out of a difficult situation.
One of the things I loved about working at Turbine was the fact that the QA department kicked ass, and had a lot of power to say, “No, this isn’t ready. Fix it.” I think part of the reason was that QA did not report to the same VP as development did, so there was no development lead who could easily override QA.
It baffles me how they say Vanguard had one QA guy. *ONE* QA guy. It’s just astounding.
Oh sweet donkey yes.
And it’s not just gaming. I’ve worked at a few “mainstream” companies that don’t get it either. Companies that build large, complicated software systems with hundreds of components written in multiple languages interacting in ways that are untrackable beyond “you changed your library and now that thing over there you didn’t know existed is breaking”, and whose attitude towards having an organized testing infrastructure is “just don’t write buggy code in the first place”.
Then they wonder why they always, but always have an emergency patch a month after they ship a new version.
*headdesk*
Having the dev team and the QA team under 2 seperate department heads is definitely a good thing.
I only work on simple VLT games but 2 years ago when the company re-organized and made this type of change, things have been for the better.
It takes away the power factor of a VP over the QA manager and makes it an even fight between VP vs. VP.
As for Vanguard. I think they were trying to use the power of the publishers QA team. But even with that, more than one guy is needed internally.
Robertson - yeah, I read that after I had posted it. That’s still too small a number, even if they’re supposed to be a liason between the exterior QA department and the interior developers. MMOs are big things, and Vanguard was even bigger!
Now I do think that the QA department should grow as a game gets older. MouseJunior gives a good example why - no matter how well you code, if you’re looking at code that’s been tapped repeatedly for five years it’s just not going to look good. Inevitably shortcuts will have to be made in that time to get everything to work, and the people who were working on it probably aren’t anymore so when it’s time to make that Big Change that should be isolated to this one area it ends up affecting a dozen other things that end up being hair-rending experiences. And that makes everyone a Sad Panda.
I am always astounded to hear stories from other companies. If I had a dev of any type that tried to fight a QA request that something wasn’t fun, or didn’t play well, or even was broken I’d can that dev so fast he’d need an opener to get out of his chair.
I guess I’m lucky in a way that i work at a small company. I lead design, art, prod and code discussions but I’m not the boss. I also run the bug/feature/task system for all the teams and coordinate things sent to test with our volunteer testers.
I can’t imagine having an issue with someone if I sent something back to them from test to say it didn’t match the spec or had a bug. I don’t even accept “works for me” as a close ticket option. It goes to test (me) for verification.
I guess I’m also lucky in that if a dev hasn’t tested it himself and checked it in, he gets the evil eye if something stupid comes out of test. I mean seriously, you wrote it and couldn’t take 5 minutes to see that its at least passingly functional? Back to the pond with you junior.
I would hate to work in an environment where QA “needed” to be a separate entity to get its job done. Hell, I’d hate to work with anyone who looked at QA as an impediment to their project.These guys are more important than designers or devs. Bottom up people. If no one buys it because its crap you don’t need VPs, designers or prima donnas of any craft.
I like that Dev Support moniker as well. If I ever have more than my prods and my outstanding volunteer beta team doing the work, I’m going to give them that name.