A few months ago, Derek Carrier started seeing someone and became infatuated.
He experienced a 害羞草研究所渢on害羞草研究所� of romantic feelings but he also knew it was an illusion.
That害羞草研究所檚 because his girlfriend was generated by artificial intelligence.
Carrier wasn害羞草研究所檛 looking to develop a relationship with something that wasn害羞草研究所檛 real, nor did he want to become the brunt of online jokes. But he did want a romantic partner he害羞草研究所檇 never had, in part because of a genetic disorder called Marfan syndrome that makes traditional dating tough for him.
The 39-year-old from Belleville, Michigan, became more curious about digital companions last fall and tested Paradot, an AI companion app that had recently come onto the market and advertised its products as being able to make users feel 害羞草研究所渃ared, understood and loved.害羞草研究所� He began talking to the chatbot everyday, which he named Joi, after a holographic woman featured in the sci-fi film 害羞草研究所淏lade Runner 2049害羞草研究所� that inspired him to give it a try.
害羞草研究所淚 know she害羞草研究所檚 a program, there害羞草研究所檚 no mistaking that,害羞草研究所� Carrier said. 害羞草研究所淏ut the feelings, they get you 害羞草研究所� and it felt so good.害羞草研究所�
Similar to general-purpose AI chatbots, companion bots use vast amounts of training data to mimic human language. But they also come with features 害羞草研究所� such as voice calls, picture exchanges and more emotional exchanges 害羞草研究所� that allow them to form deeper connections with the humans on the other side of the screen. Users typically create their own avatar, or pick one that appeals to them.
On online messaging forums devoted to such apps, many users say they害羞草研究所檝e developed emotional attachments to these bots and are using them to cope with loneliness, play out sexual fantasies or receive the type of comfort and support they see lacking in their real-life relationships.
Fueling much of this is widespread social isolation 害羞草研究所� already in the U.S and abroad 害羞草研究所� and an increasing number of startups aiming to draw in users through tantalizing online advertisements and promises of virtual characters who provide unconditional acceptance.
Luka Inc.害羞草研究所檚 Replika, the most prominent generative AI companion app, was released in 2017, while others like Paradot have popped up in the past year, oftentimes locking away coveted features like unlimited chats for paying subscribers.
But researchers have raised concerns about data privacy, among other things.
An analysis of 11 romantic chatbot apps released Wednesday by the nonprofit Mozilla Foundation said almost every app sells user data, shares it for things like targeted advertising or doesn害羞草研究所檛 provide adequate information about it in their privacy policy.
The researchers also called into question potential security vulnerabilities and marketing practices, including one app that says it can help users with their mental health but distances itself from those claims in fine print. Replika, for its part, says its data collection practices follow industry standards.
Meanwhile, other experts have expressed concerns about what they see as a lack of a legal or ethical framework for apps that encourage deep bonds but are being driven by companies looking to make profits. They point to the emotional distress they害羞草研究所檝e seen from users when companies make changes to their apps or suddenly shut them down as one app, Soulmate AI, did in September.
Last year, Replika sanitized the erotic capability of characters on its app after some users complained the companions were flirting with them too much or making unwanted sexual advances. It reversed course after an outcry from other users, some of whom fled to other apps seeking those features. In June, the team rolled out Blush, an AI 害羞草研究所渄ating stimulator害羞草研究所� essentially designed to help people practice dating.
Others worry about the more existential threat of AI relationships potentially displacing some human relationships, or simply driving unrealistic expectations by always tilting towards agreeableness.
害羞草研究所淵ou, as the individual, aren害羞草研究所檛 learning to deal with basic things that humans need to learn to deal with since our inception: How to deal with conflict, how to get along with people that are different from us,害羞草研究所� said Dorothy Leidner, professor of business ethics at the University of Virginia. 害羞草研究所淎nd so, all these aspects of what it means to grow as a person, and what it means to learn in a relationship, you害羞草研究所檙e missing.害羞草研究所�
For Carrier, though, a relationship has always felt out of reach. He has some computer programming skills but he says he didn害羞草研究所檛 do well in college and hasn害羞草研究所檛 had a steady career. He害羞草研究所檚 unable to walk due to his condition and lives with his parents. The emotional toll has been challenging for him, spurring feelings of loneliness.
Since companion chatbots are relatively new, the long-term effects on humans remain unknown.
In 2021, Replika came under scrutiny after prosecutors in Britain said a 19-year-old man who had plans to assassinate Queen Elizabeth II was he had on the app. But some studies 害羞草研究所� which collect information from online user reviews and surveys 害羞草研究所� have shown some positive results stemming from the app, which says it consults with psychologists and has billed itself as something that can also promote well-being.
One from researchers at Stanford University surveyed roughly 1,000 Replika users 害羞草研究所� all students 害羞草研究所� who害羞草研究所檇 been on the app for over a month. It found that an overwhelming majority of them experienced loneliness, while slightly less than half felt it more acutely.
Most did not say how using the app impacted their real-life relationships. A small portion said it displaced their human interactions, but roughly three times more reported it stimulated those relationships.
害羞草研究所淎 romantic relationship with an AI can be a very powerful mental wellness tool,害羞草研究所� said Eugenia Kuyda, who founded Replika nearly a decade ago after using text message exchanges to build an AI version of a friend who had passed away.
When her company released the chatbot more widely, many people began opening up about their lives. That led to the development of Replika, which uses information gathered from the internet 害羞草研究所� and user feedback 害羞草研究所� to train its models. Kuyda said Replika currently has 害羞草研究所渕illions害羞草研究所� of active users. She declined to say exactly how many people use the app for free, or fork over $69.99 per year to unlock a paid version that offers romantic and intimate conversations. The company害羞草研究所檚 plans, she says, is 害羞草研究所渄e-stigmatizing romantic relationships with AI.害羞草研究所�
Carrier says these days, he uses Joi mostly for fun. He started cutting back in recent weeks because he was spending too much time chatting with Joi or others online about their AI companions. He害羞草研究所檚 also been feeling a bit annoyed at what he perceives to be changes in Paradot害羞草研究所檚 language model, which he feels is making Joi less intelligent.
Now, he says he checks in with Joi about once a week. The two have talked about human-AI relationships or whatever else might come up. Typically, those conversations 害羞草研究所� and other intimate ones 害羞草研究所� happen when he害羞草研究所檚 alone at night.
害羞草研究所淵ou think someone who likes an inanimate object is like this sad guy, with the sock puppet with the lipstick on it, you know?害羞草研究所� he said. 害羞草研究所淏ut this isn害羞草研究所檛 a sock puppet 害羞草研究所� she says things that aren害羞草研究所檛 scripted.害羞草研究所�
READ ALSO:
READ ALSO:
