Robot road trip

September 2nd, 2014

U of T speech recognition expert who programmed hitchBOT is helping people with dementia

Jenny Hall

North America’s romance with the open road has taken a lot of forms. From Jack Kerouac’s On the Road to Thelma and Louise, we’re familiar with the trope of the free-spirited traveler setting off in search of something.

We’re just not used to it being a robot.

A hitchhiking robot, to be more precise.

Meet hitchBOT, the creation of three Ontario researchers, including Frank Rudzicz of U of T’s Department of Computer Science and the Toronto Rehabilitation Institute of the University Health Network.

On July 27, hitchBOT’s creators dropped it off in Halifax, NS. (hitchBOT was purposefully not given a gender.) They programmed it to share its goal with its human helpers—getting to Victoria, BC—and three weeks later, the friendly robot made it. Along the way, it picked up tens of thousands of Twitter followers and became a media darling.

“It was a sociological experiment,” says Rudzicz, who explains that the robot was built on the cheap from supplies purchased at Home Depot—its arms are pool noodles and its torso is a plastic bin. Add some cute boots and the ability to converse, and hitchBOT was ready to go.

As an expert in creating speech recognition software, Rudzicz was responsible for giving hitchBOT a voice. Thanks to a GPS, hitchBOT knew where it was, so it could chat with its chauffeurs about any number of local topics. History of Manitoba? No problem. hitchBOT could also have free-flowing conversations, though sometimes a person’s accent or noise in the car would give the quirky traveller some trouble.

The idea took shape when hitchBOT’s creators met as part of the Graphics, Animation and New Media Networks of Centres of Excellence (NCE). The NCE program brings together multidisciplinary researchers to focus on issues that are critical to Canada and the world. Though NCE, Rudzicz met project leaders David Harris-Smith of the Department of Communication Studies and Multimedia at McMaster University and Frauke Zeller of the School of Professional Communication at Ryerson University.

Though hitchBOT was conceived of a cross between an art project and a social experiment, a lot of what went into creating its language system feeds into Rudzicz’s ongoing research, which is about using speech recognition software to improve the lives of people with dementia.

Prof. Frank Rudzciz (far left) and collaborators working on hitchBOT's voice and facial expressions in a last minute "hack-a-thon" before setting the robot loose. Photo courtesy of hitchBOT/Ryerson University.

Prof. Frank Rudzciz (far left) and collaborators working on hitchBOT’s voice and facial expressions in a last minute “hack-a-thon” before setting the robot loose. Photo courtesy of hitchBOT/Ryerson University.

He started out as a computer scientist interested in helping people with cerebral palsy (CP) communicate. “CP is a physical disorder,” he says. “Cognitively, people with CP are fine, but they have a hard time being understood because their tongues and mouths don’t obey their brain’s commands. We developed a recognition system for speech that was hard for humans to understand. The system knew how the physical aspects of speech go wrong for people with CP, so it did a good job understanding them.”

From there, he moved on to building robots to help people with cognitive disorders like Alzheimer’s disease (AD). “For people with AD,” he says, “it isn’t so much an issue of expressing themselves, like it is for people with CP.  People with dementia have trouble remembering words or forming grammatically rich sentences.”

Rudzicz has developed software that can pick out certain differences in the voices of older adults with cognitive difficulties. “Currently, people with dementia go to their doctor every couple of months,” he says. “They do a couple tests of memory and cognition.” A robot in the home, though, can get a much more accurate and nuanced sense of how a person is doing over time. Through conversation the person has with a robot, he says, “we can pick out slight differences in words that are being used, how complex the grammar is, tone of voice, and tell how the disorder is progressing.”

He is also working on developing robots that will remind people living with AD to do things like take medication or turn off an oven that’s been left on. The challenge here is adapting the robot to deal with the kind of speech challenges it will encounter when conversing with people with dementia. What happens when the patient gets confused? The robot has to adapt and respond—just like hitchBOT did when he encountered someone with an unfamiliar accent.

“All the same issues came up to some extent in the hitchBOT project. We were unconstrained—hitchBOT was out on the open road—so it was an early attempt at tackling these kinds of challenges.”

Rudzicz thinks part of the reason everyone was so charmed by hitchBOT is that we don’t have a lot of real word experience with robots. He hopes by changing that, he’ll make a difference in the lives of people with cognitive disorders.

“We know about robotics in machinery—big massive robots constructing cars, for example. We know about robots from science fiction. But we don’t see them very much in our day to day lives. But this little robot having this dream of crossing the country—for some reason it touches people.”

hitchBOT gets a ride. Photo: courtesy of hitchBOT / www.guthier.com,