University of Ottawa researchers developing safer AI therapy tool to counter chatbot risks

University of Ottawa researchers are developing UbiMyTherapist, an AI mental health assistant designed to be safer than using ChatGPT as a therapist. The tool will run on everyday devices like smartwatches and smartphones, continuously monitoring a user's emotional state through physiological data like heart rate and sleep patterns.

Karim Alghoul, a Faculty of Engineering professor at uOttawa, leads the team. Rather than trying to stop people from using large language models like ChatGPT for mental health support, the approach layers protective technology on top. "Everyone is using LLMs anyway," Alghoul said. "So instead of just kind of forcing people to not use it, what if we put some layers of protection?"

The tool uses what's called Retrieval-Augmented Generation (RAG) technology, which grounds the AI's responses to a specific clinical dataset. This prevents the AI from generating false information—a problem known as "hallucination"—and reduces the risk of what researchers call "AI psychosis" caused by unreliable chatbot responses. The "Ubi" in the name refers to "ubiquitous," reflecting the continuous nature of data collection from wearables.

Although still in prototype phase, Alghoul warns the app isn't suitable for everyone. He would not recommend it for people with severe mental health disorders, but sees it as helpful for managing anxiety and mild depression. The product will eventually be downloadable on wearable devices, though no launch date has been announced.