OpenGesture Skill
Hard to believe we started and completed the development of the OpenGesture Skill in 30 days. It’s been busy! I certainly feel OpenGesture Skill is working well, and as intended. “If I have seen further than others, it is by standing upon the shoulders of giants” Sir Isaac Newton.
In this blog, I’ll explain how I used PCSkills Framework library and Intel OpenVINO Toolkit to integrate Amazon's Alexa and hand gestures from OpenGesture.
1. Alexa Skills Kit
First, in order to communicate with Alexa, I developed custom Intent called OpenGestureIntent. Then added sample utterances and custom slots for the OpenGesture application type, date and location. The usability of the OpenGesture skill directly depends on how well the sample utterances and custom slot values represent the real-world.
2. Amazon Web Services (AWS)
I used the same Lambda function from the PCSkills Framework, since using a Lambda function for the OpenGesture alexa voice service eliminates some of the complexity around setting up and managing endpoint. Furthermore I used the AWS Cognito, and AWS Simple Queue Service (SQS) to help with sending requests and user pool identities.
3. Microsoft Visual Studio 2017
I followed PCSkills Framework source code and API's to help create a hybrid model of cloud and PC. The PC Search Skill code helped a lot to figure out some of the integration problems for the custom slots, and performing on PC actions.
4. Intel OpenVINO Toolkit
Here I used OpenCV as the link between Unity3D and OpenVINO inference Engine to help deploy the hand gesture neural networks.
The inference engine backend is used by default since OpenCV 3.4.2 (OpenVINO 2018.R2) when OpenCV is built with the Inference engine support. Also, the Inference engine backend is the only available option (also enabled by default) when the loaded model is represented in OpenVINO™ Model Optimizer format.
I have also made the OpenGesture Application available for anyone interested in contributing or reimplementing the OpenGesture Skill project.