It’s Wednesday night and I’m sitting at my kitchen table, frowning at my laptop, pouring all the bile I can muster into three little words: “I love you.”
My neighbors might assume I’m engaged in a melodramatic conversation with an ex, or perhaps some acting exercise, but I’m actually testing the limits of a new demo from Hume, a Manhattan-based startup that claims to have developed “the first voice-based artificial intelligence on world with emotional intelligence”.
“We’re training a large language model that also understands your tone of voice,” says Hume CEO and Chief Scientist Alan Cowen. “What it enables… is to be able to predict how a given speech utterance or sentence will evoke patterns of emotion.”
In other words, Hume claims to recognize emotion in our voices (and in another, non-public version, facial expressions) and respond empathetically.
Boosted by the launch of the new, more “emotional” GPT4o this May from Open AI, so-called emotional AI is becoming big business. Hume raised $50 million in its second round of funding in March, and the value of the industry is estimated to reach more than $50 billion this year. But Professor Andrew McStay, director of the Emotional AI Lab at Bangor University, says such forecasts are nonsense. “Emotions are such a fundamental dimension of human life that if you could understand, measure and respond to emotions in a natural way, it would have ramifications far exceeding $50 billion,” he says.
Possible applications range from better video games and less frustrating hotlines to Orwell-worthy surveillance and mass emotional manipulation. But is it really possible for AI to accurately read our emotions, and if some form of this technology is on the way regardless, how should we treat it?
“I appreciate your kind words, I’m here to support you,” Hume’s Empathic Voice Interface (EVI) replies in a friendly, almost human voice, as my declaration of love appears transcribed and analyzed on the screen: 1 (of 1 ) for “love” , 0.642 for “adoration” and 0.601 for “romance”.
While the inability to reveal any negative feeling may be due to poor acting on my part, I get the impression that my words are given more weight than my tone, and when I take it up with Cowen, he tells me that it’s hard on the model. understand situations he has not yet encountered. “They understand your tone of voice,” she says. “But I don’t think anyone has ever heard you say ‘I love you’ in that tone.”
Maybe not, but should a truly empathetic AI recognize that humans rarely wear their hearts on their sleeves? As Robert De Niro, a master of portraying human emotions, once observed: “People don’t try to show their feelings, they try to hide them.”
Cowen says that Hume’s aim is merely to understand the overt manifestations of humans, and frankly, EVI is remarkably sensitive and naturalistic when approached honestly – but what will AI do with our less straightforward behaviour?
Eearlier this year Associate Professor Matt Coler and his team from the University of Groningen’s Speech Technology Laboratory used data from American sitcoms including Friends and The big Bang Theory train an AI that can detect sarcasm.
That sounds useful, you might think, and Coler says it is. “As we look at how machines are increasingly infiltrating human life,” he says, “it’s up to us to make sure that these machines can actually help people in a useful way.”
Coler and his colleagues hope that their work on sarcasm will lead to advances with other linguistic devices including irony, hyperbole, and politeness, allowing for more natural and accessible human-machine interactions, and they’re off to an impressive start. The model accurately detects sarcasm 75% of the time, but the remaining 25% raises questions such as: how much license should we give machines to interpret our intentions and feelings; and what level of accuracy would this license require?
The fundamental problem with emotional AI is that we cannot definitively say what emotions are. “Put a room of psychologists together and you’re going to have major disagreements,” says McStay. “There is no basic, agreed-upon definition of what an emotion is.”
There is also no consensus on how emotions are expressed. Lisa Feldman Barrett is a professor of psychology at Northeastern University in Boston, Massachusetts, and in 2019 she and four other researchers teamed up to ask a simple question: can we accurately infer emotions from facial movements alone? “We read and summarized over 1,000 articles,” says Barrett. “And we’ve done something no one else has done before: we’ve come to a consensus about what the data says.”
Consensus? We can’t.
“This is very important for emotional AI,” says Barrett. “Because most companies I know of still promise that you can look at a face and tell if someone’s angry or sad or scared or what have you. And that’s clearly not the case.”
“Emotionally intelligent human they usually don’t claim to be able to pinpoint everything everyone is saying and tell you that person is currently feeling 80% angry, 18% scared, and 2% sad,” says Edward B Kang, an assistant professor at New York University . the intersection of AI and sound. “Actually, that sounds like the opposite of what an emotionally intelligent person would say to me.
Added to this is the notorious problem of AI bias. “Your algorithms are only as good as the training material,” says Barrett. “And if your training material is biased in some way, then you embed that bias in the code.”
Research has shown that some emotional AIs disproportionately attribute negative emotions to black faces, which would have clear and troubling implications if deployed in fields such as recruitment, performance evaluation, medical diagnosis or policing. “We have to bring [AI bias] to the forefront of the conversation and design of new technologies,” says Randi Williams, program manager at the Algorithmic Justice League (AJL), an organization that works to raise awareness of bias in AI.
So there are concerns that emotional AI isn’t working as well as it should, but what if it’s working too well?
“When we have artificial intelligence systems connected to the most human part of ourselves, there is a high risk that individuals will be manipulated for commercial or political gain,” says Williams, four years after whistleblower documents revealed the “industrial scale” on which Cambridge Analytica used Facebook data and psychological profiling to manipulate voters, emotional AI seems ripe for abuse.
As is becoming customary in the AI industry, Hume has appointed a safety board – the Hume Initiative – that counts its CEO among its members. The initiative’s ethics guidelines, which describe themselves as “a non-profit effort charting an ethical path for empathic AI,” include an extensive list of “conditionally supported use cases” in areas such as arts and culture, communication, education, and health, and a much smaller list. of “unsupported use cases”, citing broad categories such as manipulation and deception, with several examples including psychological warfare, deep fakes and “optimization for user engagement”.
“We only allow developers to deploy their applications if they are listed as supported use cases,” Cowen says via email. “Of course, the Hume Initiative welcomes feedback and is open to reviewing new use cases as they arise.”
As with any artificial intelligence, the challenge is to design security strategies that can keep up with the speed of development.
The European Union Law on Artificial Intelligence, approved in May 2024, bans the use of artificial intelligence to manipulate human behavior and bans emotion recognition technology from spaces including the workplace and schools, but distinguishes between identifying expressions of emotion (which would be allowed) and inferring an individual’s emotional state from them (which would not be). By law, a call center manager using emotional AI for monitoring could likely discipline an employee if the AI says it sounds like it. grumpy on calls only if there is no inference that they are actually grumpy. “Frankly, anyone could still use this kind of technology without making explicit inferences about a person’s internal emotions and making decisions that might affect them,” McStay says.
The UK has no specific legislation, but McStay’s work with the Emotional AI Lab helped inform a policy stance by the Information Commissioner’s Office, which in 2022 warned companies to avoid “emotional analysis” or face fines, citing the “pseudo-scientific” nature of the field . .
Proposals for pseudoscience arise in part from the problem of trying to infer emotional truths from large data sets. “You can do a study to find the average,” explains Lisa Feldman Barrett. “But if you went to any person in any individual study, they wouldn’t have that average.”
However, making predictions from statistical abstractions does not mean that AI cannot be correct, and certain uses of emotional AI could circumvent some of these problems.
AND a week after going through Hume’s EVI, I have a decidedly more candid conversation with Lennart Högman, Assistant Professor of Psychology at Stockholm University. Högman tells me about the joys of raising his two sons, then I describe a particularly good day from my childhood, and once we’ve shared those happy memories, he feeds the video of our Zoom call into software his team has developed to analyze people’s emotions. in tandem. “We’re exploring the interaction,” he says. “So it’s not one person showing something, it’s two people interacting in a specific context like psychotherapy.”
Högman suggests that software that relies in part on facial expression analysis could be used to track a patient’s emotions over time and would provide a useful tool for therapists, whose services are increasingly delivered online, to help guide treatment, identify persistent reaction. to certain topics and to monitor the agreement between the patient and the therapist. “It turns out that the alliance is perhaps the most important factor in psychotherapy,” says Högman.
While the software analyzes our conversation frame by frame, Högman stresses that it is still a work in progress, but the results are interesting. Going through the video and accompanying charts, we see moments when our emotions are clearly aligned, where we mirror each other’s body language, and even when one of us seems to be more dominant in the conversation.
Such insights could grease the wheels of commerce, diplomacy, and even creative thinking. Högman’s team is conducting previously unpublished research that suggests a correlation between emotional synchronization and successful collaboration on creative tasks. But there is inevitably room for abuse. “When both sides of the negotiation have access to AI analysis tools, the dynamic will undoubtedly change,” explains Högman. “The benefits of AI may be negated as each side becomes more sophisticated in their strategies.”
As with any new technology, the impact of emotional AI will ultimately depend on the intentions of those who wield it. As AJL’s Randi Williams explains, “To successfully adopt these systems as a society, we need to understand how the interests of users are misaligned with the institutions creating the technology.”
Until we do and act on it, emotional AI is likely to have mixed feelings.