Overview / Usage

A potential application of action localization can be tracking and localizing athletes in various sports. A real-world event where this would be a useful application is the 2020 Olympics in Tokyo, Japan. An example use case can be Gymnastics Artistic where athletes perform different moves in different sequences. If we can use pose-estimation features + RGB features (maybe even optical flow) and design a model, we can achieve efficient action localization and classify different moves.

Methodology / Approach

A possible approach is using a spatiotemporal deformable parts model. In this, we can extract HOG3D features, generate filters, build up the feature pyramid, then classify the combined segments as an action. This approach is inspired by the work by: http://www.cs.ucf.edu/~ytian/sdpm.html. Another approach is to propose action tubelets spaced across different frames. Then, classify upon the action as a whole. This method is inspired by: https://arxiv.org/pdf/1705.01861.pdf. The primary framework to use would be PyTorch or Tensorflow. A training, validation, and testing dataset has to be curated by hand for best results.

Technologies Used

PyTorch, OpenCV, PIL, NumPy, Pandas, Intel AI DevCloud