Alexa Prize TaskBot Win
We’re excited to announce that Glasgow School of Computing Science researchers, Team GRILL Team wins the first Alexa Taskbot Challenge. The details of the finals are included in Amazon’s finals announcement. Our winning system is called GRILLBot, a “multi-modal task-oriented digital assistant to guide users through complex real-world tasks”. It helps users with cooking and everyday tasks in the home..
You can read our system description in the Amazon Alexa Taskbot Prize proceedings. The details of the team’s system research paper is GRILLBot: A flexible conversational agent for solving complex real-world tasks.
The Alexa prize GRILL team consists of School of Computing Science graduate students Sophie Fischer, Carlos Gemmell, Iain Mackie, Paul Owoicho, and Federico Rossetto. Dr. Jeff Dalton is the team’s faculty advisor.
Left to right: Iain Mackie, Federico Rossetto (pictured on the tablet), Paul Owoicho (standing), Carlos Gemmell, Jeff Dalton (standing), and Sophie Fischer
Users of Alexa in the US can interact with the systems by saying, “Alexa, open GRILL Bot” and all the finalist teams by saying “Alexa, let’s work together”.
“GRILL Lab’s ongoing research on conversational information seeking and multimodal dialogue systems make it an ideal fit. The competition is exciting for us because it goes beyond simple information tasks to help users with real-world tasks. As passionate home cooks and amateur chefs, we find the challenge of helping people in the kitchen through an AI system exciting. In short, it is the perfect opportunity to put our research into practice and make a real difference in people’s lives.” said faculty advisor Jeff Dalton.
“The experience of taking the ambitious Alexa Prize TaskBot goals from ideas and research to prototype and launching on a tight timeline was thrilling and humbling,” said Federico Rosetto.
The team found the challenge very exciting, “The experience was a rollercoaster and challenged the team across multiple dimensions. The experience of taking the ambitious taskbot goals from ideas and research to prototype, and through to launching on a tight timeline was thrilling and humbling. We found meeting the evolving needs of real users very motivating, but also hard as the competition and user expectations got harder. Taking our research from the lab to launching a system that interacts with real users is very rewarding.”
“We learned that users learn and enjoy a task more when there are rich image and video elements,” said Iain Mackie. To do that, the students relied on image and video augmentations, applying multimedia elements that could be reusable across different tasks.
GRILLBot is just the first step toward a new generation of virtual assistants. “Taskbots have the potential to help people and make a significant difference in all aspects of life. Taskbots and interaction in the real world push the boundaries of virtual assistants to become more knowledgeable, support richer forms of interaction, adapt to a changing environment, and be empathetic. We’re a long way from that vision and we need to make significant advances in interactive language technologies.”
Team leader Carlos Gemmell said, “The future of taskbots is exciting with richer forms of communication and modality from video, rich sensors, and even augmented reality. We envision taskbots that can learn new skills to adapt to complex tasks in new environments by autonomously acquiring knowledge and adjusting their communication patterns with individuals. We find it exciting because realizing the vision requires fundamental advances in core research areas of machine intelligence across areas.”
“Ultimately, the goal of taskbots is one shared by the research vision from our research lab: to enable people to understand the world and make people’s lives better,” said Carlos Gemmell, GRILLBot’s project leader.
Stay tuned for more exciting work from the team!
The GRILL is hiring (independent from the Alexa team) and has multiple open funded PhD and Postdoc opportunities for researchers in neural models for conversational AI. Please reach out or email Dr. Dalton directly.
#JoinTheKitchen