Overview / Usage

Video generation from an image is quite a new problem in computer vision. The problem is ill-posed, since there may be infinitely many options for the next frame which is to be generated. Since it is relatively new, not many researchers have attempted this problem. In video generation from images, our task is to generate video from a sketch. Now, a sketch is a type of data which is very sparse, it is not a natural image nor it has some nice properties a natural image has, such as gradients in texture, colors, etc. The goal is to put “life” into user-drawn static sketches. The product is still in its research phase. We explore various machine learning and computer vision algorithms such as Optical Flow, Keypoints matching, Convolutional Neural Network, Generative Adversarial Network, Recurrent Neural Network.

Methodology / Approach

We use various task-specific deep learning neural networks such as SketchRNN and Style transfer and some general ideas and algorithms as convolutional neural networks, recurrent neural networks, and general purpose image-to-image translation network (pix2pix). We also used key points matching algorithms.

Technologies Used

Intel DevCloud, Intel ColFax Cluster, Tesnorflow, Keras, PyTorch.