Overview / Usage

Moving a mouse pointer on a screen usually requires physical contact of the fingers on a screen, mouse or trackpad. However, there may be times when physical contact is not possible, such as the fear of contacting a virus via surfaces. Hence, the need for an alternative to control the mouse pointer without physical contact.

This application uses the eye gaze from the face adjacent to a screen or video input to move the mouse pointer of a computer.

Methodology / Approach

Four different computer vision models are used - face detection model, landmark detection model, head-pose estimation model, and gaze estimation model.

The face-detection model detects the face from a frame. The detected face is then cropped and sent to the landmark detection model and head-pose model. The landmark detection model is used to identify the left and right eyes from the cropped face. While, the head-pose model is used to estimate the yaw, pitch and roll of the face. The results from both the landmark detection model and head-pose model is then fed to the gaze-estimation model, which the produces an x and y coordinate estimation of the eye-gaze on the screen.

The output from the eye gaze is then fed to a mouse controller. The mouse controller then moves the mouse pointer to the given coordinates on the screen.

Technologies Used

OpenVINO

Documents and Presentations

Repository

https://github.com/ajudges/mouseController