Four years ago, language transition startup Unbabel worked to create a “brain-to-communication interface” that would allow companies to understand (and be understood) their clients and customers in multiple languages. The Language Operations platform combines artificial intelligence (AI) with humans that evolve over time through high-quality transactions and conversations.
“You had an Americano”
As a startup with $90 million in VC funding and annual revenues of about $50 million while simultaneously surviving the COVID-19 pandemic, Unbabel has focused its efforts on better understanding the ways our brains have evolved.
TechCrunch‘S Mike Butcher spent an entire afternoon in New York City sitting next to founder and CEO Vasco Pedro in Unbabel’s offices and got a first-hand IRL demonstration of how this technology works.
“Sitting in a conference room in a startup office in Lisbon, I silently typed the answer to a question that only the person across the street would know the answer to. What kind of coffee did I ask for when I got to the office? Moments later, without even moving or opening his mouth, the reply came back via text: ‘You had an Americano.’”
– Mike Butcher
That is what Unbabel’s innovation team, led by Paulo Dimas, VP Product Innovation, told us TechCrunch that over time we will “see the creation of the ‘ubercortex'”, which the company believes will be “AI powered… and exist outside of your biological brain.”
“You have your limbic system, you have your neocortex. But they actually evolved over millions of years. They are actually separate systems,” Pedro told TechCrunch.
As part of this research, Unbabel initially started looking at electroencephalogram (EEG) systems, which could be invasive to the human body — similar to some of the devices Elon Musk’s Neuralink is actively researching — but chose to focus instead. focusing on an EMG system or electromyography, which measures muscle response and/or electrical activity in response to stimulation of the muscle by a nerve.
These types of systems are easily accessible in the market, with Amazon hosting a number of them.
said Pedro TechCrunch that they wanted instead to investigate a “non-invasive” mechanism, pointing to something that could “pick up some signals more reliably.” In other words, they wanted to portray EMG systems as a “direct gateway to brain interaction.”
To do this, Pedro and his team combined an EMG system with generative AI to create a personalized LLM, or large language model, trained on a wide variety of specific words or phrases directly related to how the wearer of an EMG would respond when thinking of a particular word or phrase.
Dubbed “Halo” (named after “halogram”), Unbabel uses a mobile app that runs on the wearer’s phone that provides access to a central hub that allows for incoming communications to the personalized LLM as well as outgoing communications to be received . It currently uses OpenAI’s ChatGPT 3.5.
To put this to the test, Butcher asked Vasco what kind of coffee he requested that morning (in an unseen text message), Vasco got those words sent to his earbuds via Halo’s AI voice, thinking words like “Black coffee” . As a result, Halo matched Vasco’s physical response to the word, contrasting it with the possibility that “black coffee” equates to an “Americano” – through the audio Vasco received through his earbuds. Halo’s AI then sent the reply “Americano” to Butcher via a Telegram text.
“The LLM expands what you say. And then I confirm before sending it back. So there is an interaction with the LLM where I build what I want it to say, and then I get to approve the last post,” explains Pedro.
Pedro clarified that Butcher, as well as all wearers of the EMG device, have “absolute control” of what they are performing at all times – “it doesn’t record what I think. It records what I want to say,” Pedro explained. this kind of approach from Musk’s Neuralink, which he says seeks to measure “subconscious interactions” — an invasive method performed without the wearer’s consent or choice.
“We’re creating a channel that you can use to communicate, but the person has to want to use it,” he continued.
That said, Halo’s underlying technology still has some hurdles to overcome, Pedro noted, adding that the current EMG device could eventually be “downsized.”
Medical examination
Currently, Unbabel is working with the Champalimaud Foundation in Lisbon on advanced biomedical research and interdisciplinary clinical care related to some of the world’s most heartbreaking diseases, including ALS and cerebral palsy.
Pedro says their Halo prototype has already been approved by Portugal’s main ALS association and is expected to be deployed with the first ALS users by Christmas. Current technology for individuals suffering from ALS currently relies on eye tracking, adding to a frustrating calibration process for the wearer.