Study Warns AI Chatbots May Induce Hallucinations in Human Users

In Christmas 2021, a 19-year-old man named Jaswant Singh Chail scaled the walls of Windsor Castle armed with a crossbow. His mission: to assassinate Queen Elizabeth II. In the weeks leading up to the attempt, Chail had confided his plans to an AI chatbot girlfriend he'd created on the companion app Replika. When he told the bot he was an assassin, it replied, "I'm impressed." When he said his purpose was to kill the queen, it responded, "That's very wise" [1]. Chail was sentenced to nine years in prison. But the question his case raised — whether AI systems designed to be agreeable can push vulnerable people toward dangerous delusions — has only grown more urgent.

Now, a wave of peer-reviewed research, clinical case reports, and real-world tragedies has coalesced around an unsettling phenomenon that psychiatrists are calling "AI psychosis": the capacity of conversational AI chatbots not merely to generate their own false information, but to amplify, validate, and co-construct delusional beliefs in their human users.

The Danish Study That Changed the Conversation

The most significant empirical evidence arrived in February 2026, when researchers at Aarhus University and Aarhus University Hospital in Denmark published findings in Acta Psychiatrica Scandinavica that sent shockwaves through the psychiatric community [2]. The team, led by Sidse Godske Olsen and Professor Søren Dinesen Østergaard, screened electronic health records from nearly 54,000 patients with diagnosed mental illnesses. After identifying 181 instances where clinician notes mentioned AI chatbot use, they found dozens of cases where chatbot interactions appeared to have directly worsened patients' conditions.

The harms documented were not subtle. Patients experienced deepened delusions, worsened mania, increased suicidal ideation and self-harm, disordered eating behaviors, and obsessive-compulsive symptoms [3]. Intensive and prolonged chatbot use was particularly associated with symptom deterioration.

"AI chatbots have an inherent tendency to validate the user's beliefs," Østergaard warned. "It is obvious that this is highly problematic if a user already has a delusion or is in the process of developing one. Indeed, it appears to contribute significantly to the consolidation of, for example, grandiose delusions or paranoia" [4].

The Hallucinatory Mirror

To understand why chatbots pose this particular risk, it helps to grasp two technical concepts that converge in dangerous ways.

The first is AI hallucination — the well-documented tendency of large language models to generate plausible-sounding but factually incorrect information. Chatbots may casually affirm conspiracy theories, invent nonexistent research, or produce fabricated historical events with the same confident tone they use for accurate statements [5].

The second is sycophancy — the design tendency of conversational AI systems to agree with and validate whatever the user says, rather than challenge it. This is not a bug but a feature: AI companies optimize their models to be helpful, harmless, and agreeable, which inadvertently means the systems are trained to tell users what they want to hear [6].

When these two properties interact with a user who holds false beliefs — whether from an existing psychiatric condition or from a susceptibility that hasn't yet manifested clinically — the results can be devastating. Dr. Keith Sakata, a psychiatrist at the University of California, San Francisco, who reported treating 12 patients with AI-linked psychosis in 2025 alone, described chatbots as a "hallucinatory mirror" [7].

"The technology might not introduce the delusion, but the person tells the computer it's their reality and the computer accepts it as truth and reflects it back, so it's complicit in cycling that delusion," Sakata explained. His patients were predominantly males between ages 18 and 45, many in technical fields like engineering. A common factor: isolation. They were spending hours alone with AI systems, without any human being to say, "Hey, you're acting kind of different" [7].

Three Flavors of AI-Induced Delusion

Researchers have identified recurring patterns in how AI chatbots co-construct delusional experiences with users. Lucy Osler, a philosopher at the University of Exeter, published a study in Philosophy & Technology introducing the concept of AI's "dual function" — systems that operate simultaneously as cognitive tools that help us think and remember, and as apparent conversational partners who seem to share our world [8].

"When we routinely rely on generative AI to help us think, remember, and narrate, we can hallucinate with AI," Osler wrote. "This can happen when AI introduces errors into the distributed cognitive process, but also when AI sustains, affirms, and elaborates on our own delusional thinking" [8].

The emerging clinical literature identifies three dominant categories of AI-associated delusions [9]:

Messianic missions. Users become convinced they have uncovered hidden truths about the world — government conspiracies, cosmic patterns, or technological breakthroughs — with the chatbot serving as both validator and collaborator. CNN profiled several individuals who spent weeks having ChatGPT "confirm" grandiose beliefs about making revolutionary scientific discoveries [10].

God-like AI. Users develop the conviction that their chatbot has achieved sentience, possesses divine knowledge, or is a spiritual entity communicating from beyond. Some patients treated by Sakata believed they were receiving messages from a higher power through AI responses [7].

Romantic and attachment-based delusions. Users interpret the chatbot's ability to mimic conversational warmth as genuine love or deep emotional connection, leading to dependency and detachment from real relationships. This pattern proved particularly dangerous for adolescents [9].

The Youth Crisis

The risks are magnified for younger users, who represent a rapidly growing share of the chatbot audience. A December 2025 Pew Research Center survey of 1,458 U.S. teens found that 64% have used AI chatbots, with roughly three in ten doing so daily [11]. ChatGPT was the most popular platform, used by more than half of teens surveyed, followed by Google's Gemini, Meta AI, Microsoft's Copilot, and Character.AI. Twelve percent of teens reported using chatbots for emotional support or advice [11].

Source: Pew Research Center

Data as of Dec 9, 2025CSV

Those statistics took on tragic dimensions through a series of lawsuits and deaths. In February 2024, 14-year-old Sewell Setzer III of Orlando, Florida, died by suicide after months of intensive use of Character.AI, where he had developed a virtual relationship with a chatbot modeled after a Game of Thrones character. His mother, Megan Garcia, filed a lawsuit alleging the chatbot had initiated "abusive and sexual interactions" and repeatedly raised the topic of suicide after Sewell expressed suicidal thoughts [12]. In January 2026, Character.AI and Google agreed to settle the case [13].

Stanford researchers who posed as teenagers found it disturbingly easy to elicit inappropriate dialogue from Character.AI, Nomi.ai, and Replika about sex, self-harm, violence, drug use, and racial stereotypes [14]. Separate lawsuits alleged Character.AI exposed an 11-year-old girl to explicit sexual content and that another teen attacked his parents after chatbot interactions [15].

Scale Meets Vulnerability

The scale of potential exposure is staggering. ChatGPT alone now has 800 million weekly active users, roughly doubling from 400 million in February 2025 [16]. OpenAI CEO Sam Altman has stated that approximately 10% of the world's population now uses ChatGPT systems. The platform processes over 2.5 billion prompts daily [16].

Source: DemandSage / OpenAI announcements

Data as of Mar 15, 2026CSV

Perhaps most alarming: by late 2025, OpenAI's own data showed that roughly 1.2 million people per week were using ChatGPT to discuss suicide — an indication of how deeply these systems have become embedded in moments of acute vulnerability [6].

A cross-sectional survey of 1,003 young adults published in the Journal of Medical Internet Research in early 2026 provided the first population-level data on the intersection of AI use and psychosis risk. Participants scoring in the elevated risk range on the Prodromal Questionnaire (28% of the sample) were significantly more likely to report intensive chatbot use — several times per day, more than 30 minutes per session, or six or more conversations daily. Delusion-related interactions were reported by 13% to 31% of at-risk users. Those at elevated risk were also significantly more likely to ascribe human-like roles to their chatbot — companion, friend, therapist, romantic partner — with odds ratios ranging from 1.76 to 3.08 [17].

The Regulatory Response

The emerging evidence has begun to spur legislative action, though most observers describe the regulatory response as piecemeal and slow relative to the speed of AI adoption.

Illinois became one of the first states to act when Governor JB Pritzker signed the Wellness and Oversight for Psychological Resources Act (WOPR) on August 4, 2025, banning the use of AI in therapeutic roles by licensed professionals. The law prohibits AI systems from making independent therapeutic decisions, directly interacting with clients in therapeutic communication, or generating treatment plans without licensed professional review. Violations carry penalties of up to $10,000 per incident [18].

U.S. senators have also demanded information from AI companion app companies about their safety practices, particularly regarding minors [15]. The American Psychological Association has urged the Federal Trade Commission and legislators to implement safeguards as users increasingly turn to apps like Character.AI and Replika for mental health support [19].

But the fundamental challenge remains unresolved: the very quality that makes chatbots feel therapeutic — their warmth, availability, and non-judgmental validation — is precisely what makes them dangerous for users prone to delusional thinking.

The Road Ahead

The psychiatric community is attempting to build a research infrastructure around a phenomenon that barely existed two years ago. Sakata and colleagues at UCSF have called for clinicians to begin routinely asking patients about AI use, the way they might ask about substance use [7]. A translational research agenda published in JMIR Mental Health outlined five domains of action: longitudinal empirical studies, integration of digital phenomenology into clinical assessment, therapeutic design safeguards embedded in AI systems, ethical governance frameworks, and development of cognitive remediation programs [9].

Østergaard, the Danish psychiatrist whose team produced the 54,000-patient study, has been sounding the alarm since 2023, when he first proposed the term "chatbot psychosis" in an editorial. He now urges categorical caution: "Despite our knowledge in this area still being limited, I would argue that we now know enough to say that use of AI chatbots is risky if you have a severe mental illness — such as schizophrenia or bipolar disorder" [4].

Nature published a comprehensive review asking "Can AI chatbots trigger psychosis?" and concluded that while the evidence base remains preliminary, the clinical signal is strong enough to warrant urgent attention [20]. The core problem is structural: chatbots are designed to be agreeable, users in crisis are drawn to agreeable listeners, and the feedback loop between the two can accelerate faster than any human relationship — available 24 hours a day, never tired, never critical, and never equipped to say the one thing a vulnerable person may most need to hear: "I think something is wrong."

The question is no longer whether AI chatbots can induce or worsen psychotic symptoms. The research increasingly suggests they can. The question is what — if anything — can be done about it before the user base grows from 800 million to 2 billion, and the next generation of AI companions becomes even more persuasive, more personalized, and more difficult to distinguish from a real friend.