DevMesh

Recognition Assistance for Visually Impaired in Workplace using Computer Vision

Nandini Ramanan

Nandini Ramanan

Dallas, Texas

1 0
  • 0 Collaborators

In order to make OFFICE a better place for VISUALLY IMPAIRED, who experience serious difficulties in the workplace due to their reduced perception of the environment. Laying down a foundation of two embarrassing situations that each of the interviewees faced, with UBIQUITOUS COMPUTING, I have made an effort towards improving their perception of the surrounding reality. I built an assistive application for visually impaired to help with face recognition and activity recognition. We reconstruct face-images in different angles, using one front image captured with a real-time camera mounted on users. Used deep learning network called Overfeat for feature extraction. Supported with Mobile application to control the camera and a bone conduction device for feedback to the user based on the predictions ...learn more

Project status: Concept

Artificial Intelligence

Code Samples [1]

Overview / Usage

An assistive application for visually impaired in an office location should perform the following;

  1. Help identify people in office.
  2. Help avoid awkward situations, by informing them if a question was directed to them.
  3. Mobile application to control the camera.
  4. Concise/ clear audio feedback to the user.
  5. Ensure privacy protocols are maintained in the application

Methodology / Approach

I began the work by listing the problems that visually impaired people face at their workplace. These included diverse scenarios that I thought were a major concern according to the initial research. This was followed by multiple brainstorming sessions that helped me to narrow down to two specific problem scenarios. These problem scenarios led me to several ideas that could be feasible solutions. I conducted four interviews, out of which, three were with the completely visually impaired people and one with the partially visually impaired person. I learned about the problems faced by the visually impaired through the interviews. The most common problems were related to knowing who is approaching them, whether the person talking near the visually impaired is making a conversation with them or someone else and who are the people around them. An analysis was done for each idea against the feedback from interviews with the target population which led to a refined needs assessment.
With the help of the experts and brainstorming, I narrowed down the problem space to two most specific ideas of identifying individuals in the office community and responding to the conversations that are directed towards them. This problem is of great interest to me because it would help the visually impaired individual build good relationships with their peers, and at the same time build their confidence. An in-depth validation of these designs was done using storyboards and personae. Additionally, I conducted a survey to further refine and improvise designs. To summarize, based on the data I collected from the interviews and a survey, I clearly defined problem space and finalized the design.

Technologies Used

  1. Overfeat CNN
  2. Android studio
  3. AWS
  4. Python
  5. OpenCV

Repository

https://github.com/nandhiniramanan5/Activity-recognition-using-CV.git

Comments (0)