King's College London

Academics, graduates and undergraduates from the Department of Informatics took home gold last month at a leading robotics competition in Milton Keynes exploring human-robot social interaction and the deployment of autonomous robots in the real world; being featured in BBC One's The One Show.

Increasingly, human-robot social interaction has become a key element of robotics research as the potential of autonomous robots in settings like care homes and hospitality has been explored to deal with the impact of an aging population and demographic change.

Led by Senior Lecturer in Computer Science Matteo Leonetti in collaboration with Lecturer in Autonomous Systems Gerard Canal, the team consisted of PhDs Peter Tisnikar, Zoe Evans, Jared Swift and undergraduate students Nicole Lehchevska, Pawel Makles and Yousef Altaher. 

Competing in the ‘Coffee Shop’ and ‘Elevator’ “episodes” of the Milton Keynes Smart City competition against teams from across Europe, the group programmed robot TIAGo to take a lift and orders at a mock café to test how it would autonomously interact with humans. This included tasks like ensuring items were delivered, tables were kept clean, and customers were approached when convenient to mimic the social conventions of good customer service.

To do this the group developed software for autonomous behaviour by integrating and building on several areas of artificial intelligence, such as computer vision, object detection, and speech recognition. By implementing these systems on TIAGo, the robot could perceive the world and react to external stimuli, like a voice request asking for coffee.

Integration of these systems and deployment in real-world scenarios can be difficult however, presenting a number of challenges to be overcome. As PhD student Zoe Evans explains, “When you deploy in the real world, it’s not guaranteed that multiple software packages like speech recognition and computer vision will work optimally on the same piece of hardware. Furthermore, common systems like the object detection system YOLO that tend to work well for general objects have a tougher time identifying specific items that you’re likely to encounter in a café, like coffee cups and packs of biscuits.

“Throughout the course of the competition, sunlight from windows often interfered with the infrared detection systems underpinning TIAGo’s vision and members of the general public didn’t speak into the robot’s microphone clearly when issuing voice commands. This was far from a pristine testing environment and to deal with these challenges you need to learn from your surroundings and locally optimise your system, and that’s a major skill.

“There is a huge difference between what works in simulation in the lab and what works in the real world. Building a mobile, talking system that can navigate the unexpected challenges posed by a real-world environment is a great learning experience and is very gratifying.”

The ability to apply theoretical lessons in the lab into the real world also positions students well for a career in industry, with industry robotics players such as Ocado sponsoring the event.

Dr Matteo Leonetti explains, “This competition goes far beyond what we’re able to teach in the classroom, and the open-ended collaboration it invites is a mirror to the problem-solving, project-based work seen in industry.

“When students leave university, there are few of them that can fully program an operational robot. The level of practice our students received here goes far beyond what their peers across the country receive, and that level of experience in hands-on deployment will stand them in good stead when seeking employment in the sector. I have brought students to this event for the past six years and plan to develop the team further within our Computer Science and Artificial Intelligence programmes at King’s.”

The future of the work presented at Milton Keynes will focus on how robots can collaborate with humans and adapt to specific individuals' behaviour through techniques like reinforcement learning. Examples like how robots can adapt to support people with different levels of partial sightedness will be a key focus for Dr Leonetti and the team going forward as they examine how robots can support, rather than replace, humans.