Fine Tuning a BERT Model for Sentiment Extraction using Intel OneAPI Toolkit (Daal4Py, OneDAL) and Intel Optimised PyTorch

Risab Biswas

Jalpaiguri, West Bengal

1 4

Fine Tuned a BERT Sentiment Model using Intel OneAPI ToolKit and PyTorch. The training was done both on a NUC and Intel DevCloud. The model is trained on the IMDB Dataset of 50k reviews. The model accuracy came to 92.2% and the inference shows some great results. The model is served as a web app. ...learn more

Project status: Under Development

oneAPI, Artificial Intelligence

Groups
DeepLearning, Artificial Intelligence India

Intel Technologies
oneAPI, DPC++, Intel NUC, Intel Opt ML/DL Framework, Intel Python, DevCloud

Links [1]

Overview / Usage

Sentiment Extraction is one of the widely adopted use-cases in the Industry. It's a very good starting point for NLP research enthusiasts.

You'll learn how to do the required text preprocessing (special tokens, padding, and attention masks) and build a Sentiment Classifier using the amazing Transformers library by Hugging Face!

This will give you a great understanding of Intel's OneAPI Advanced Analytics toolkit and how to leverage the power of Intel DevCloud.

Dataset- https://rb.gy/mri6k4

Methodology / Approach

Following are the Steps -

Intuitively understand what BERT is!
Setting up environment on Intel DevCloud
Preprocess text data for BERT and build PyTorch Dataset (tokenization, attention masks, and padding)
Use Transfer Learning to build Sentiment Classifier using the Transformers library by Hugging Face
Evaluate the model on test data
Predict sentiment on raw text
Serving the solution as a web app using Flask