A partnership between games developer Ubisoft and IBM is experimenting with speech recognition to enhance one of the first social virtual reality experiences: ‘Star Trek: Bridge Crew’.
As of today, not only can players interact with each other on the virtual bridge; when you play as the captain, you can give verbal commands to non-player crew members.
Speaking to The Wall Street Journal, creative director of the game, David Votypka says IBM Watson is very different to the older kind of voice recognition technology, which relied on specific keywords that the computer would recognise:
“You don’t have to specifically say “raise shields” and that’s the only phrase you can use. You can say like “we need shields up now” and Watson will be like, “ok, the player is trying to tell me to raise the shields” so it gives the players a lot of natural language and a lot of natural options and it’s probably the most advanced voice recognition technology that’s been used in games so far.”
Unusually, ‘Star Trek: Bridge Crew’ is a multi-platform VR experience—so it can be played on PSVR, Rift or, as seen here, on the HTC Vive.
Players take on one of four roles; the captain, the engineer, the helmsman or the tactical officer.
But, the voice recognition functionality works using IBM’s supercomputer Watson, which interacts and understands the world in a way more similar to humans.
To break it down to it’s simplest form, you need to picture Watson as a kind of central AI brain—with ‘Bridge Crew’ engineers using Watson’s voice and conversation service.
Essentially this means that when a player speaks a voice command, it gets captured and then translated to text.
The text is then sent up to the IBM servers where Watson lives.
Watson then decodes the text, understands it, and passes it to its conversation engine.
“So you train Watson and it offers you certain functionality but the more you train it, the smarter it gets. Watson can actually analyse and kind of parse and process, and make assumptions about “oh, you said these words, so I think this is actually what you’re trying to say.” So the result is a very natural language phrasing that the players can use.”
While the game was released at the end of May, beta-testing for the version using speech recognition functionality has only just begun.
Other companies using speech technology include Microsoft, Amazon, Google and Apple.