- Projects 3
- Followers 0
Jonas Mayer
Munich, Bavaria
Stereo Style Transfer AI
See the world as a 3D painting ...learn more
Project status: Under Development
Virtual Reality, Artificial Intelligence
Intel Technologies
Intel Opt ML/DL Framework
Overview / Usage
What if Virtual Reality glasses could transform your environment into a three-dimensional work of art in realtime? What if every detail of the world would be shown in the style of a painting from Van Gogh? One of the many interesting developments in the field of Deep Learning is the so called "Style Transfer". It describes a possibility to create a patchwork (or pastiche) from two images. While one of these images defines the the artistic style of the result picture, the other one is used for extracting the image content. The Hardware Hacking Team from TNG Technology Consulting managed to build an application using OpenCV and Tensorflow to realize such goggles. When you see the world through these glasses, your environment will be displayed in the styles of famous painters like Claude Monet or Pablo Picasso.
Methodology / Approach
The world is recorded by two cameras which are integrated into VR glasses. The recording laptop is then sending the images from those cameras to a powerful workstation computer. There the images are distortion corrected using OpenCV and then stylized using a convolutional neural net and Tensorflow. The stylized content is streamed back to the recording laptop, where it is shown in the Windows Mixed Reality VR goggles. This can all be done in real time with around 100ms delay.
Training of the neural net is done up-front in the cloud using a style image. The neural net is then trained to stylize arbitrary images.
Technologies Used
Windows Mixed Reality,
Tensorflow,
OpenCV,
Gstreamer,
OpenGL
Collaborators