Engineers at the University of Waterloo have discovered a new way to program robots to help people with dementia locate medicines, glasses, phones and other items they need but have lost.
And while the initial focus is on helping a specific group of people, the technology could one day be used by anyone who has searched high and low for something they’ve misplaced.
“The long-term impact of this is really exciting,” said Dr. Ali Ayoub, a postdoctoral fellow in electrical and computer engineering. “The user can engage not only with a companion bot but also with a personal companion bot which can give them more autonomy.”
Ayoub and three colleagues were astonished by the rapid rise in the number of people suffering from dementia, a condition that restricts brain function, causing confusion, memory loss, and disability. Many of these individuals repeatedly forget the location of everyday objects, reducing their quality of life and placing additional burdens on caregivers.
Engineers believe that a companion robot with its own episodic memory could be a game-changer in such situations. They succeeded in using artificial intelligence to create a new kind of artificial memory.
The research team started with an animated Fetch robot that contained a camera to perceive the world around it.
Then, using an object detection algorithm, they programmed the robot to detect, track and keep a memory record of specific objects in its camera view through the stored video. With the robot’s ability to distinguish one object from another, it can record the time and date the objects entered or left.
The researchers then developed a graphical interface to enable users to choose the objects they want to track and, after typing in the names of the objects, search for them on a smartphone or computer app. Once this happens, the bot can indicate when and where it last noticed a particular object.
Tests have shown that the system is very accurate. And while some individuals with dementia may find the technology daunting, Ayoub said caregivers can easily use it.
Going forward, the researchers will conduct user studies with people without disabilities, and then people with dementia.
Paper on the project Where is my phone? Towards the development of an episodic memory model for companion robots to track users’ salient objects presented at the recent 2023 ACM/IEEE International Conference on Human-Robot Interaction.