Natural conversation with game characters has become possible thanks to AI

Modbox creator combined voice recognition with Windows, artificial intelligence OpenAI GPT-3the natural speech synthesis of Replica and VR, creating a unique demo: probably one of the first virtual characters with conversational AI.

Modbox is a sandbox for creating multiplayer games on SteamVR. Recently, its developer has been researching the latest solutions in the field of speech-driven machine learning and AI. These are the OpenAI GPT-3 language model and the Replica natural speech synthesizer. The effect is simply stunning and very innovative. In existing video games and VR experiences, you either can’t talk to NPCs at all, or you can only navigate through pre-written dialogue trees. The dialogues are scripted and recorded by the actors, and the game only plays audio files. In this case, it’s completely different. The virtual character receives any question posed by the player and answers it. Unfortunately, there is a significant delay between the question and the answer, because the GPT-3 and the Replica they work in the cloud. Future models running locally on the device may overcome this drawback.

How is it possible? Until recently, having a direct and natural conversation with virtual characters and getting convincing answers no matter what we ask for has until recently been in the realm of fantasy. This is only changing with recent advances in machine learning that finally make this idea possible. The GPT-3 model, which is the result of years of evolution, is responsible for this. Let us give you a little history. In 2017, Google’s AI department introduced a new approach to language models called Transformer. State-of-the-art machine learning models have used the concept of attention before to achieve better results, but the Transformer model was built entirely around it. A year later, a startup OpenAI, backed by Elon Muskapplied the Transformer approach to a new general language model called Generative Pre-Training (GPT) and found that it was able to predict the next word in multiple sentences and answer multiple-choice questions.

In 2019, OpenAI expanded the model more than 10 times, releasing GPT-2. It turned out that this scaling greatly increased the capabilities of the system. With a few prompts, the model was already able to write entire essays on almost any topic. In some cases, it was indistinguishable from a human. Due to the potential ramifications of OpenAI, he initially chose not to release it, which in turn sparked media attention and speculation about the social implications of the advanced language models. GPT-2 had 1.5 billion parameters, but in June 2020 OpenAI scaled the idea to 175 billion in GPT-3, the operation of which is shown in this demo. GPT-3 results are almost always indistinguishable from human reports.

Technically, GPT-3 is gone true “understanding”although the definition of the word is a matter of debate – mostly philosophical. Sometimes the effects of the model can seem absurd or even harmful (such as praising suicide), so the researchers had to impose certain restrictions, such as a “common sense” mechanism. That AI should be ethical from the start has long been supported by Microsoft, and this is also the case with GPT-3, which can only reach generally available consumer products with appropriate blockages.

Microsoft, which has invested a billion dollars in OpenAI, has Proprietary source code and commercial use rights for GPT-3it is therefore unlikely that this feature will be permanently added to Modbox. Either way, the demo above is perhaps the best glimpse into the future of interactive game characters so far. Future language models could change the very nature of game design and allow the introduction of entirely new genres.

Did you like this article? Share with others!

Leave a Comment