Overview / Usage

Rudra is a fully voice-operated, completely hands-free system for the elderly, so all they have to do is talk to Rudra to get a job done. The voice assistant has been specially curated according to the needs of the elderly. It performs timely checkups on them, allows them to create reminders for themselves, listen to stories, call up any family members, and have a sympathetic discussion on any topic they desire with the robot. All this is accomplished by employing the following:

1. NLP: Natural language processing helps in discerning the tone and context of any spoken sentence, allowing Rudra to have more sympathetic reactions to it. DialogFlow has been used for this purpose.
2. Expression-wise response: Rudra continuously monitors the elderly's expressions to further improve the user's experience. This also helps in monitoring their mental health regularly.
3. Query response: Several APIs have been used for providing the best one-shop-stop solution for any miscellaneous queries the user might have. The APIs are:
   1. Dialogflow
   2. Firebase Realtime Database
   3. Google Calendar
   4. News
   5. Weather
4. Meal suggestions and reminders: Physical well-being is equally as important as mental health, therefore, Rudra is designed to timely remind the elderly to drink water, have food, and take any medicines if needed.
5. Action Recognition: Computer vision is used to make sure that the elderly have drunk water, had food, and slept properly. The data is only updated in the backend once Rudra detects the above-stated actions and then confirms them with the user to avoid misinterpretation.

Methodology / Approach

Voice Assistance

• Python & Dialogflow
• Sympathetic interaction
• Response according to expression detection and action recognition
• Reminders (meal, water, medicine, etc.)
• Controlling Rudra using voice commands

Human Following

• ESP32, Camera, Compass
• Detects the direction of the recognized face and follows the human
• The advantage in providing all-time surveillance for quick emergency response

Companion App

• Flutter
• For relatives/ caregivers
• Remote monitoring of health and wellbeing
• Remote control accessible

Emergency Monitoring and Reaction

• Camera
• Immediate notification sent to the phone app

Technologies Used

Raspberry Pi 4

Motor Driver

BO Motors + wheel

11.1V 3S 2200mAh LiPo Battery

USB Camera

Micro Servos

Ultrasonic Proximity Sensors

Flutter, Python, Dialogflow, Face Recognition, Voice Assistant, Firebase