Overview / Usage

Using AI/ML to assist the creation of art and design has become increasingly important for software features, professional artists and hobbyists. Plenty of such examples made by Snapchat, Adobe Photoshop and Nvidia etc.

Background effects

Background removal or blurring can help with preserving privacy or expressing the participant’s emotion. This is an important feature that you may find in conferencing applications such as Google Hangout or Microsoft Teams.

Style transfer

Some other common scenarios include the transformation of the image style, which is super important in defining a piece of artwork or the artist.

While neural style transfer can create artistic effects, it’s not able to mimic the exact style of a particular artist or type of art. Another drawback of neural style transfer is that when applying the transformation to some objects such as human faces or portraits, the output image may not look so appealing although artistic.

Therefore, stylizing only the segmented background results in a much more elegant image for a photo portrait. This also mimics a common scenario in photography or graphics design changing the artworks’ background texture and or style.

Methodology / Approach

We chained the two models to produce a stylized image as follows:

1. Take a photo or choose an image from the gallery as the input.
2. Apply the segmentation model to the input image.
3. Apply the style transfer model to the image background (content image)
4. Blend the masked out foreground image with the stylized background to produce the final image.

Technologies Used

* TensorFlow Lite

* Android/Kotlin

Repository

https://github.com/margaretmz/segmentation-style-transfer