- Projects 18
- Followers 11
OpenGesture
Accelerating African Sign Language using OpenGesture ...learn more
Project status: Published/In Market
RealSense™, Internet of Things, Artificial Intelligence
Intel Technologies
DevCloud,
OpenVINO,
Intel powered laptop,
oneAPI
Overview / Usage
While deaf people from different language communities can communicate with each other without difficulty in SASL (South African Sign Language), they cannot understand "sign language" interpreters unless they have been schooled in the manually coded language used by the interpreter. With the growing adoption of assistive technologies, deaf and visually impaired people need to be able to communicate naturally with their network, regardless of whether the second person has expertise on sign language, especially under cases of video consultations with their health practitioner, Educators or be it friends & family . We propose a deep neural network for the prediction of South African Sign Language(SASL) and gesture recognition to direct standard English text translation in natural video sequences using CPU. To effectively handle complex evolution of pixels in videos, we propose to decompose the motion and content, two key components generating dynamics in videos.
Methodology / Approach
Data Centric AI
_It is crucial to have as much, high quality and accurate data as possible for improved model accuracy and performance, therefore training data consists two types of hand color and depth images. __The OpenGesture Dataset was collected using Intel Realsense D435 by the ABI Data Engineering Team Africa._The dataset has ten digits (0-9) sign language classes. Each gesture is repeated 30 times by two independent sample agents.
Model Centric AI
We apply transfer learning technique of which is a model that has already been trained on a related task and reusing it in a new model. OpenGesture demonstrates how to build a Keras model for classifying ten signs of gestures by using a pre -trained TF2 SavedModel from TensorFlow Hub for image feature extraction, trained on the much larger and more general ImageNet dataset. There are multiple possible models to try (more are provided on the OpenGesture Colab).
Technologies Used
TensorFlow Hub
Repository
https://github.com/AfricaMachineIntelligence/OpenGesture
Collaborators