Overview / Usage

Problem Definition
Hashtags provide users with a tagging mechanism to help organize, group, and create visibility for their posts. This is a simple idea but can be challenging for the user in practice which leads to infrequent usage. Hashtag recommendation comes with numerous challenges including processing huge volumes of streaming data and content which is small and noisy. We will use preprocessing methods to reduce noise in the data and determine an effective method of hashtag recommendation to get better recommendations.

Objective
To build a Twitter Hashtag Recommendation System. The System will recommend the user other hashtags based on his/her given input.

Domain : Twitter Tweets retrieved with the Twitter API
Purpose : Recommend similar hashtags to users
Personalization Level : Generic
Interfaces : Explicit Input, Recommended Output

Methodology / Approach

Getting Data
The recommender uses the Twitter API to obtain the raw tweets for preprocessing. To obtain a large volume of data, the Streaming API is used. This tutorial provides a very basic introduction to getting tweets using a Python library called Python Twitter Tools to connect to Twitter API and downloading the data from Twitter.

Preprocessing
Cleaning the tweets before processing is done using various steps.

This infographic provides some of the basic details about cleaning tweet text.
Most of the cleaning is done using Regular Expressions in Python 2/3 and R. This tutorial from Tutorials Point may be helpful.
The stop words to remove from the tweets may be found here.

Technologies Used

The basic requirements of the project are as follows (Note: The project was developed with the following specifications and may also work with earlier versions):

Python v2.7
twitter
HTMLParser
Tkinter module
R v3.2.2