Of course, automated software can only test certain things, and there is no substitute for the creativity and judgment of a human tester. Besides, an automated system can't tell you whether or not a game is fun to play. For that, MGS employs separate usability testing groups, sometimes very early in the game's development, to test out new ideas. Other usability tests take place much later in the development cycle, with outside playtesters who can examine a game from a fresh perspective. "These guys are the ones living the popular conception of game testing; play games for a living, get your chance to let the developers know what would be great, and what isn't working," says Lamb.
At Nintendo, contract testers can also report problems with gameplay, although it isn't clear whether or not the developers do anything with those reports. Feedback on the playability of a game is reserved for a select group of full-time NOA employees, or so the rumor goes. Davis wasn't exactly sure; Nintendo is deliberately tight-lipped about the subject. But some of the Nintendo perks are not as closely guarded, like the ability to shop at the heavily discounted company store. For a gaming enthusiast working as a tester, perks like this are part of what make the whole experience worthwhile.
Putting It All Together
So which is it: "dream job" or "button-mashing monkey"? At Nintendo and Microsoft, testing is a little bit of both, and the perks of getting to play unreleased games must be measured against the drawbacks of playing unfinished, potentially broken builds for days, weeks, and months. Game industry jobs are highly sought after, even for experienced testers coming from the business software world. As with any other job, however, the corporate culture makes a difference.
At Nintendo, some of it has to do with their traditional corporate structure. Company-wide directives work their way down from the top management in Japan to contract testers, who are given strict work schedules and testing assignments. Nintendo is all about raw manpower; testers are a valuable resource in ensuring that a game gets enough time in front of human eyes and hands.
Entry-level contractors at Microsoft share an experience similar to that of their Nintendo counterparts, but the paths diverge for senior testers. Nintendo moves their senior testers into management roles, while senior Microsoft testers move up in the engineering career ladder. Experienced testers are tasked with the development of automated testing tools to handle the more repetitive aspects of game testing. For them, testing is a technical problem to be solved through engineering rather than longer hours in front of a console.
From a career perspective, DeHoogh tells me "it's pretty common practice to do a year or so at Nintendo and then move up to Microsoft. They do pay more and Microsoft takes care of its employees more." Plus, he mentions that the food is better.
Some contract testers eventually leave altogether for greener pastures, giving up the thrill of game testing for a larger paycheck and the benefits that come with a full-time position. Others stay and advance along the tester career track. After all, game testing has its own set of unique perks. Despite the low pay and the repetitive work, there is still something glorious about the opportunity to watch games grow and change while getting paid to play them. As Lamb puts it, "After six years I'm still dreamy eyed over making a living while also making games, and a large bank account never motivated me in anything."
Alan Au is a freelance writer, academic, and games industry advocate. When he's not trying to circumvent bugs that the testers missed, he spends his time exploring the connection between games, education and health.