Advanced Robotics: Conversational Agents in Healthcare
In recent years, robotics has advanced in leaps and bounds. The term advanced robotics now refers to technologies that interact with the real-world environment to solve real-world problems. Applications of advanced robots, particularly in healthcare, are transforming patient care and operational efficiency.
The Role of Virtual Assistants
Many robotic applications require a virtual assistant to interact with the robot. A virtual assistant is a software agent that performs tasks or services for an individual based on commands or questions. It simulates a conversation, delivering voice or text-based information to a user. This enables interaction with artificial agents through natural speech.
A Conversational Agent for Hospital Use
This project involves creating a customized conversational agent that operates within a hospital, assisting patients and enhancing their experience. The system uses a Google Home Mini smart speaker, chosen for its powerful conversational engine. Although the system is compatible with Alexa, this article focuses on the Google Home Mini implementation.
Project Capabilities
With this system, users can wake up the assistant by saying a customized keyword, initiating a conversation. For example:
«Hello Victory, where is patient Juan?»
Response: «Patient Juan is in room 25.»
The robot can also initiate conversations independently, for instance, greeting users or responding to specific events.
What is ROS?
The Robot Operating System (ROS) is an open-source framework that allows developers to build and reuse code across robotic applications. ROS operates via nodes that communicate by publishing and subscribing to topics or using services for requests and responses.
Communication in ROS
- Messages: Publishers send messages that subscribers receive.
- Services: Nodes send a request and receive a response.
Hardware Components
Raspberry Pi 4
The Raspberry Pi 4, running the Debian Buster 10 OS, provides the computational power and GPIO pins for external interactions. Its low cost and flexibility make it ideal for this project.
Matrix Voice
The Matrix Voice development board includes an 8-microphone array, 18 RGB LEDs, and an FPGA. It connects directly to the Raspberry Pi, enabling voice-driven behavior and IoT integrations.
Software Capabilities
Custom Wake Word Detection
Using Rhasspy, an open-source voice assistant service, the system detects custom wake words via Pocketsphinx. This lightweight tool enables offline configuration and testing through a web interface.
System Features
- Play Noise: Prevents accidental wake-ups by Google Home Mini.
- Wake Up Google Home Mini: Uses speakers to pronounce the keyword.
- Start Conversations: Initiates dialogue autonomously.
State Machine Implementation
The state machine, developed in C++ using Qt, transitions through states like emitting noise, waiting for wake words, and initiating conversations. Signals trigger state changes based on specific events.
Node-RED
Node-RED facilitates graphical programming of hardware and online services by connecting predefined nodes. Below is a part of the flowchart used in this project:
Dialogflow Integration
Dialogflow simplifies the creation of conversational interfaces. Intents like «which patient is in room X?» are mapped to responses processed in Node-RED.
Current Project Status
The technical components are complete, and the system’s hardware assembly is ready:
The next step is to design a 3D part to integrate the system seamlessly into the robot.
A Collaborative Initiative
This project is a collaboration between Scalian Spain and the University of Málaga, led by Juan Antonio Ramírez and supervised by Francisco Javier Camacho Bermúdez and Alejandro Hidalgo Paniagua.