TexGen
- 0 Collaborators
TexGen is a transformer based model that used the OpenAI GPT-2 under the hood to predict the next word in a sentence based in the previous word. This can be used to generate sentences or even articles, almost indistinguishable from those written by humans. ...learn more
Project status: Published/In Market
Intel Technologies
Intel Python
Overview / Usage
TexGen uses OpenAI GPT-2 to create a model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training.
Methodology / Approach
TexGen uses dominant sequence transduction models that are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. It connects the encoder and decoder through an attention mechanism.
Technologies Used
The model is based on the concept of Transformer Model based on the Google Paper "Attention is all you need".
Libraries Required:
tensorflow-gpu==1.12.0
fire>=0.1.3
regex==2017.4.5
requests==2.21.0
tqdm==4.31.1
This project is completed with the "345M" model with the base.