- Projects 1
- Followers 1
Gavin AI
Gavin AI is a project, created by Scot_Survivor (Joshua Shiells) & ShmarvDogg, which aims to have Englsih human like conversations through the use of AI and ML. Gavin works on the Transformer architecture however, Performer & FNet architectures are being investigated for better scaling. ...learn more
Project status: Under Development
oneAPI, HPC, Artificial Intelligence
Groups
Student Developers for oneAPI
Intel Technologies
oneAPI,
DPC++,
Intel Python,
Intel CPU,
Intel vTune
Overview / Usage
Overview
Gavin is a conversational AI which is developed and trained by Scot_Survivor (Joshua Shiells) & ShmarvDogg. Which is then interfaced through Discord.
What do we do
Scot_Survivor, works on the implementation of the AI & ML architecture. Cleaning & sourcing datasets as well as overseeing the training of Gavin.
ShmarvDogg work's on the backend of Gavin specifically on data preprocessing and areas that require high performance.
Gavin is an ever evolving AI which we often make changes to investigating new and experimental ML model architectures and reading papers in the field of conversational AI to learn about and implement newer methods and technologies to make him as coherent & functional as we can with the resources available to us.
Git link
The repo tree can be found here: https://github.com/Gavin-Development
Methodology / Approach
We use TensorFlow, Numpy with Python in combination with custom made modules developed in C++ using pybind 11 to develop and train Gavin.
The frontend is comprised of a script to interface with a discord bot to allow us to test him as well as give the world easy to use access.
All of these different projects can be found under:
https://github.com/Gavin-Development
Papers & Sources
- Attention is all you need: https://arxiv.org/abs/1706.03762
- Rethinking attention with Performers: https://arxiv.org/abs/2009.14794
- FNet: Mixing Tokens with Fourier Transformations: https://arxiv.org/abs/2105.03824
- Dataset Reddit Data: https://files.pushshift.io/reddit/comments/
Technologies Used
Powerful consumer grade GPUs. For the training of large & advanced models such as Gavin.
High capacity & speed RAM, memory is used for many things, within Gavin its critical to have high capacity memory to ensure out dataset can be loaded for training. While high speed is needed in order to have the throughput needed to not slow down our overall training.
Fast multicore CPUs for data processing. All of Gavin's data processing is done on the CPU, by using threading & multiprocessing its possible to stretch the work load across CPU cores, allowing for massive improvements in speed.
Currently we're investigating the use of intel's oneAPI at the moment, for use in our data pipeline. When it comes to AI/ML it is vital that a data pipeline is both robust and fast enough to keep up with the GPU. In the event there is an I/O bottleneck, training can effectively come to a halt.
Repository
https://github.com/Gavin-Development
Collaborators