Escapist EditorialsPutting Gamers to Work: DARPA's Grand DesignEscapist Editorials - RSS 2.0
DARPA hopes to make crowdsourcing an exercise in entertainment.
We at The Escapist love the Defense Advanced Research Projects Agency (DARPA) . Since its inception in 1958, DARPA has been actively developing truly innovative technology and pursuing cutting-edge scientific research. We've naturally covered a lot of DARPA's goings on over the years because of its fondness for robots and similarly amazing technology. Just last year it funded Boston Dynamics' development of the brick-throwing robodog and "WildCat," which makes the Four Horsemen seem quaint, as well as making strides towards the inevitable future of combat, jet-mounted lasers. (PEW PEW!) DARPA is basically XCOM's R&D team.
Most recently, DARPA turned its Sauron-like gaze on gamers, sponsoring the development of several browser-based titles intended to test military software vulnerabilities. As part of its "Crowd Sourced Formal Verification" program, DARPA worked with a handful of game developers to create entertaining ways to solve real life software testing problems. I had an opportunity to discuss the project with Aaron Cammarata, the Chief Creative Officer at one such developer, voidAlpha, which was responsible for StormBound. If you'd like the whole story, check out the full interview here.
The obvious question, of course, is how in the world you end up developing games for DARPA, which is best known for things like the aforementioned weaponized robopuppy. (Okay, maybe "weaponized" is a strong word, but those concrete blocks could do some serious damage!) It turns out that Cammarata had actually worked with DARPA before on an unrelated video game project. After hearing about the project, "... honestly my first response was to go look up Formal Verification. I have a degree in CS, so I had a vague idea in mind, but trying to fit that together with 'video game' was making my head explode," Cammarata said. Bringing a single-slide presentation to a "Crowd-Sourced Formal Verification Proposer's Day" in San Francisco, Cammarata pitched working with the studio to "a room full of really scary-smart people."
I was also curious as to where the notion behind StormBound came from. Crowdsourcing has been around, sure, but it's traditionally been thousands of people poring over satellite images looking for something, rather than each person playing their own personal game (Unless you consider "I Spy" a form of high entertainment, I suppose). DARPA, it turns out, decided to innovate crowdsourcing on its own. "The project's parameters were set by DARPA's goals, and we did our best to provide a game that would meet those needs," said Cammarata, going on to ask "If these kinds of problems can be crowdsourced, what else might we solve this way?" He even went so far as to suggest that this is "the birth of a whole new industry," calling StormBound "a game that serves a functional purpose." It goes beyond the now-outmoded idea of "gamification," which typically aimed to make tedious tasks slightly less tedious - or even addictive and attractive - by introducing reward mechanics. StormBound and its ilk actually present a fun and challenging game experience, and applies the user input to solving legitimate Computer Science problems.
Sure, games have been considered for use in military training, successfully used in rehabilitation efforts, and even used to supplement traditional education, but, at least to my knowledge, nobody's ever mapped solutions of dynamic puzzles in games to real-world software testing. Will it be a billion dollar industry? That's all going to depend on adoption, I expect. If DARPA can make the next Candy Crush, it will invariably be testing a lot of software.
StormBound starts with a tutorial which explains the basic concepts for solving the puzzles. As soon as you're out of the tutorial, however, you'll start working on real puzzles, offered up by the testing servers. Cammarata would love to be able to offer "live data" even earlier on in the process - in the tutorial itself, perhaps - but "The problem is that it's a catch-22," he explains, "this isn't like a Sudoku or crossword, because we don't know the answer to a level before we give it to you. That's why we need you to play it!" I was also curious about the division of development time between game and verification. I was impressed to learn that it took only 18 months to go from concept to execution. "A lot of that time was building the bridge between gaming and verification." That almost goes without saying, given the fact that verification requires a highly-specialized skillset, and voidAlpha was essentially playing Gutenberg here, allowing literally anybody to participate in the process. "That's what makes it a capital-R, DARPA-hard Research problem," Cammarata said.
If you're curious about how the game-to-verification integration works, you should check out Jef Bell's explanation. I won't pretend to understand it entirely, but the basic idea is that the game will serve you a bit of software data, presented as a puzzle. You "analyze" that data using the tools provided in the game. "The spells you cast in StormBound describe the patterns," Bell explains. By combining the spells (in Formal Verification they're called assertions) used to clear a level, it creates a way to prove the truth of Boolean (true or false) properties of the software.
While I can't necessarily explain it all in plain English, nor can I really understand the nitty-gritty details all that well, one thing is clear; this concept has the potential to change the world of formal verification for software makers. In its present form, it's not going to be the next Plants vs. Zombies or Peggle, but as the burgeoning CSFV industry moves forward, refining the ideas and creating even more compelling stories and mechanics behind the verification-driven games, it may be a simple matter for commercial enterprises to start utilizing the practice to better consumer products as well. Can you imagine a world where Windows isn't riddled with vulnerabilities? Neither can I, but given the track record, I suspect this is our only hope.