Overview / Usage

A challenging artificial intelligence problem require to generate text descriptions from the given image data. it requires both methods from computer vision to understand the content of the image and a language model from the field of natural language processing to turn the understanding of the image into words in the right order.

Given an Image - Model will detect the textual description about the image using computer vision and natural language understanding.

Methodology / Approach

Getting Data - Flickr8k dataset

Preprocess the Image Data, using VGG Network to extract the features from the Image Data

Preprocess the Text Data using NLP text preprocessing (remove punctuations, convert to lowe case, remove all words with numbers in them, Remove all words that are one character or less in length (e.g. ‘a’))

Vectorize text data

build the feature extractor model(cnn) + build sequence model(lstm model)

save the model with low validation loss

evaulate the test image using best saved model.

​

​

Technologies Used

keras

nlp

opencv

Repository

https://github.com/Amir22010/NLP_Deep_Learning/tree/master/Image_Captioning_NLP_VISION