Overview / Usage

A Social Media Listening team at a company has to manually validate heart failure related conversations fetched from the social listening tool which scans twitter, Facebook, forums, blogs etc. Such conversations are posted by multiple stakeholders like patients, doctors, media houses, general public, etc. The team needs to identify the patient conversations, so as to dig deeper into them and identify the patient needs. The data science team wants to automate this process by building intelligent algorithms to predict patient conversations.

The task is to build an Intelligent pipeline that can segregate patient conversations from the rest of the group given historically tagged patient data.

Detection of conversation can be modeled as a supervised learning problem utilizing the classical NLP techniques. The final machine learning model can be exposed as REST API using which the media handling team can easily detect the patient conversations.

Methodology / Approach

• Thoroughly wrangled the data for feeding it to machine learning models. This includes rigorous data cleaning and data preprocessing (used Count Vectorizer and TF-IDF).
• Thoroughly investigated the dataset to understand it better so as to further understand what models might work well.
• Came up with two machine learning models - Naive Bayes and Neural Network. The later, of course, outperformed the former. The final model was robust enough to detect patient conversations.

Technologies Used

• Python (lnaguage)
• TensorFlow, scikit-learn, numpy, pandas, matplotlib (main libraries)

Repository

https://github.com/sayakpaul/Patients-Conversation-Detector