GauGAN-gifs

Welcome to the GitHub Repository for my ARS 390: AI and Art Final Project. I hope you enjoy what you see (:

Arist Statement: For this project, I wanted to focus on the spontaneous beauty that comes from using Nvidia’s GauGAN application. I really wanted to focus on the bridge (or to some, the gap) between AI and Art and I was immediately drawn to this platform when it was introduced to us. When I focused on this medium for my midterm project, I quickly learned the tedious process involved with producing a cohesive gif using the GauGAN platform. At first it was incredibly frustrating, but the deeper I dove into it, the more I was intrigued. It is so satisfying to see the original block-color segmentation map come to life through the GAN. It is especially rewarding seeing an entire scene to pan through. I hope you enjoy the whimsical results from my experimentation.

Artistic Goals: I wanted to create gifs which give a sense of realism, while still indicating that they aren't quite real. I really love the output from the GauGAN platform as it creates a believable yet still somewhat questionable visual experience when turned into a gif. I think the resulting work really captures the overlap between AI and Art. I love how many differnt approaches can be taken with this concept. The platform gives a lot of space for creativity in different ways than the standard.

Technical Goals: After working on my midterms project, I wanted to never look at another GauGAN again because of how meticulous the whole process was. I tested mutliple methods to design the little worlds in order to create a gif which panned smoothly. At first, after a few trial and errors of getting the correct GauGAN color palette onto Photoshop, I drew a larger scene which I then placed into a 512x512 canvas. I placed the starting corner on the canvas, and then saved about 50 slightly shifted or zoomed images of the drawing. This method mostly worked, except that you can't know what cohesivity to expect of the output until you have run all of the inputs through the GAN. With this method, it was also difficult to recall the corresponding color to what you wanted to draw in Photoshop. The next method that I tried was done entirely on the GauGAN website. This made it easier to include features that I wanted since I was working directly on the platform. Howevere, instead of actually being able to shift components in the scene, I had to hand draw them moved over a little bit for every frame. The third method was a combination of both, where the scene would be started on the GauGAN website and then download and uploaded to Phothsop to then be shifted and added on to. All three methods provided various succesful gifs which are shown below.

Future Goals: Some potential for furthering this experimentation with the GauGAN could involve trying to draw an existing image using the GauGAN palette as the segmentation map, and then feeding this original image as the style GAN, and seeing how similar the outputted image is to the original image. An extension of this could be to build up a big enough dataset of these pairs to train a Teachable Machine to see if it could identify between an actual image and a GauGAN generated one.

References:

Nvidia GauGAN website: http://nvidia-research-mingyuliu.com/gaugan/

Inspiration from: https://youtu.be/ZXFmZsv0Ddw

RESULTS:

GauGAN gif #1 (original project) - drive