Human Action Recognition from Limited training data
- 0 Collaborators
The goal of this project is to show that without using high dimensional more costly data, we can achieve comparable, even better performance with intelligent computing and data augmentation. Throughout this project we only utilized RGB data modality, which is most commonly obtainable video data, whereas most state-of-the-art methods utilize RGB-D data to extract salient features with higher precision. Yet, after adopting dynamic frame dropout, gradient injection and data augmentation along with our proposed deep LSTM framework, we achieved better accuracy. ...learn more
Project status: Under Development
Intel Technologies
Intel Opt ML/DL Framework
Overview / Usage
Human activity recognition based on video streams has received numerous attentions in recent years. Due to lack of depth information, RGB video based activity recognition performs poorly compared to RGB-D video based solutions. On the other hand, acquiring depth information, inertia etc. is costly and requires special equipment, whereas RGB video streams are available in ordinary cameras. Hence, our goal is to investigate whether similar or even higher accuracy can be achieved with RGB-only modality. In this regard, we propose a novel framework that couples skeleton data extracted from RGB video and deep Bidirectional Long Short Term Memory (BLSTM) model for activity recognition. A big challenge of training such a deep network is the limited training data, and exploring RGB-only stream significantly exaggerates the difficulty.
Methodology / Approach
We propose a set of algorithmic techniques to train this model effectively, e.g., data augmentation, dynamic frame dropout and gradient injection. The experiments demonstrate that our RGB-only solution surpasses the state-of-the-art approaches that all exploit RGB-D video streams by a notable margin. This makes our solution widely deployable with ordinary cameras.
Technologies Used
Python, Tensorflow, Keras, Caffe.