Overview / Usage

problem extraction of Text from Image

Methodology / Approach

Firstly, the image is fed to CNN to extract image features. The next step is to apply Recurrent Neural Network to these features followed by the special decoding algorithm. This decoding algorithm takes LSTM outputs from each time step and produces the final labeling
The detailed architecture will be the following. FC — fully connected layer, SM — softmax layer.
Next, we do reshape operation. After that, we obtain the sequence of 8 vectors of 16 elements. After that, we feed these 8 vectors to the LSTM network and get its output — also the vectors of 16 elements. Then we apply a fully connected layer followed by a softmax layer and get the vector of 6 elements. This vector contains a probability distribution of observing alphabet symbols at each LSTM step.

The number of CNN output vectors can reach 32, 64 or more. The choice will depend on the specific task. Also in production, it is better to use multilayered bidirectional LSTM.

After the model training, we apply it on images from test set and get really high accuracy. We also visualize probability distributions from each RNN step as a matrix.

Technologies Used

1] Deep Learning
2] machine Learning
3] python