How to Use SD 2.1 & Custom Models on Google Colab for Training with Dreambooth & Image Generation

[FurkanGozukara]

How to Use SD 2.1 & Custom Models on Google Colab for Training with Dreambooth & Image Generation

Full tutorial: https://www.youtube.com/watch?v=2yGGorOxtbA

Our Discord : https://discord.gg/HbqgGaZVmr. This video is a follow-up video of : https://youtu.be/mnCY8uM7E50. If I have been of assistance to you and you would like to show your support for my work, please consider becoming a patron on 🥰 https://www.patreon.com/SECourses

Playlist of Stable Diffusion Tutorials, Automatic1111 and Google Colab Guides, DreamBooth, Textual Inversion / Embedding, LoRA, AI Upscaling, Pix2Pix, Img2Img:

https://www.youtube.com/playlist?list=PL_pbwdIyffsmclLl0O144nQRnezKlNdx3

In this video, I am demonstrating how you can use #StableDiffusion 2.1 on a #GoogleColab notebook that we have demonstrated in our previous video "Transform Your Selfie into a Stunning AI Avatar with Stable Diffusion - Better than #Lensa for Free". Moreover, I am showing how you can use custom models on this Google Colab notebook as well.

Also, I am showing how you can do training on Stable Diffusion 2.1 or on custom models such as Analog-Diffusion or Anything V3.

Furthermore, I am showing how you can just do inference (image generation) without doing any training. So you can also skip training and just generate images as you wish.

I am also explaining the differences between Stable Diffusion 2.1 and 1.5 and how they work differently. I explain why do you need different codes to run different models.

The link of the Google Colab Notebook used in the first video: https://colab.research.google.com/github/ShivamShrirao/diffusers/blob/main/examples/dreambooth/DreamBooth_Stable_Diffusion.ipynb

The link of Stable Diffusion 2.1 (768 pixels): https://huggingface.co/stabilityai/stable-diffusion-2-1

The link of best vae file for Stable Diffusion : https://huggingface.co/stabilityai/sd-vae-ft-mse

00:00:00 How to use different custom models in the Google Colab notebook

00:01:06 How to fix revision parameter to make custom models work in Stable Diffusion Google Colab Dreambooth notebook

00:02:16 How to use Stable Diffusion 2.1 on the Google Colab Dreambooth notebook

00:02:58 How to change vae files to get better quality results with Stable Diffusion

00:03:24 How to do inference (image generation) with Stable Diffusion 2.1 on the Google Colab Dreambooth notebook

00:04:03 Necessary code changes to run Version 2.1 on the Stable Diffusion Google Colab Dreambooth notebook

00:04:40 Generating 768x768 pixels images with SD 2.1 on the Google Colab notebook

00:05:09 How to return back SD 1.5 version on the Stable Diffusion Google Colab Dreambooth notebook

00:06:05 You don't have to do training to do inference (generating images)

00:06:19 You can do training on different custom models as well

00:06:30 When I say version 1.5 I refer to all models that are trained from 1.x Stable Diffusion models

00:07:04 The code changes between different prime version numbers 1.x vs 2.x

00:07:27 Ending speech of the tutorial guide

Image generation using artificial intelligence is a fascinating field that has the potential to revolutionize the way we make, manipulate, and share visual content. With the help of machine learning algorithms, it is now possible for computers to generate high-quality images that are indistinguishable from those made by humans.

One of the most exciting applications of image generation AI is in the field of art. Using deep learning techniques, artists and designers can make unique and visually stunning works of art that would have been impossible to make using traditional methods. For example, a computer program can analyze a large dataset of paintings by a particular artist and then generate a new piece of art that is stylistically similar to the original artist's work.

Another interesting use case for image generation AI is in the field of advertising. Companies can use machine learning algorithms to make personalized ads for their target audience. For example, an ad agency could use image generation AI to make a series of ads featuring a particular product that are customized to appeal to different demographics.

One of the most promising applications of image generation AI is in the field of medicine. Doctors and researchers can use machine learning algorithms to generate detailed 3D models of organs and tissues, which can be used to better understand how the body works and to plan surgeries. Additionally, image generation AI can be used to make training materials for medical students, allowing them to learn about the human body in a more interactive and engaging way.

In conclusion, image generation AI is a rapidly developing field that has the potential to revolutionize many different industries. From art and advertising to medicine and education, the possibilities for this technology are endless. As AI continues to advance, it will be exciting to see what new and innovative uses emerge for this powerful tool.

Video Transcription

• 00:00:00 Greetings everyone.

• 00:00:01 This is a follow up video of my transform yourself here into a stunning AI avatar with

• 00:00:06 stable diffusion.

• 00:00:07 Better than Lensa for free.

• 00:00:09 So I have been asked numerous times that how to use custom models with a Google Colab notebook.

• 00:00:17 OK, it is so easy.

• 00:00:18 For example, let's say you want to use analog diffusion.

• 00:00:22 So you open its GitHub page from here, copy the path from clicking here.

• 00:00:28 Let me zoom it a little bit like this.

• 00:00:30 I'm clicking this.

• 00:00:32 Then in the Google Colab folder that I have shared in the video, just change the model

• 00:00:39 name here.

• 00:00:40 OK, are we done?

• 00:00:41 You can also change the output DIR so you can have different folders for each model.

• 00:00:47 Also, don't forget to change variables here if you want.

• 00:00:53 And one final thing that you need to do.

• 00:00:56 So you see in here we are going to use vae file.

• 00:01:00 You can keep this as it is if it is version 1.5.

• 00:01:07 And there is one another thing, the revision.

• 00:01:10 This is really important because people were having problems when using different models.

• 00:01:17 This problem is caused by the you see here.

• 00:01:22 There are several branches in the stable diffusion.

• 00:01:26 Main, bf16, fp16.

• 00:01:29 However, on analog diffusion, there is only one branch which is main.

• 00:01:34 So whichever the model you are using, you need to change revision to the branch here.

• 00:01:41 OK, for example, let's say I want to use anything V3, then I am copying its name like this.

• 00:01:49 I am changing the model name here, copy paste, and I am going to files and versions of anything

• 00:01:55 V3 and I am checking whether there are different branches.

• 00:01:59 Yes, there are checkpoints, diffusers, safe tensors, but the main is the correct one for

• 00:02:05 this one.

• 00:02:07 And on here: I am keeping the revision main.

• 00:02:09 With this way you can use different models with the Google Colab notebook.

• 00:02:16 OK, we can also use 2.1 version with this Google Colab.

• 00:02:22 We just copy and paste its path, which you can do from here.

• 00:02:27 Copy and paste.

• 00:02:28 There are also different branches for this.

• 00:02:30 For example, let's say we want to use FP 16, then we change it to FP 16 here.

• 00:02:41 OK.

• 00:02:43 Also for 2.1 version, you should delete this.

• 00:02:46 Because 2.1 already have vae file as can be seen.

• 00:02:52 Let me show you.

• 00:02:54 In here and there is no better vae file for that.

• 00:02:59 However, for SD 1.5, we can change the vae file if we want.

• 00:03:06 Currently, the default one we had was sd-vae-ft-mse and it is the best one from here.

• 00:03:15 So we were already using the best one.

• 00:03:17 We can also see it is the best one in the middle.

• 00:03:20 I have compared the images, so it is the middle one.

• 00:03:24 OK, to use version 2.1 with this Google notebook, we need to do several things first.

• 00:03:32 First of all, you can use either trained model as usual with model path equal to weights

• 00:03:41 DIR, the weights directory.

• 00:03:44 Or if you don't want to do any training and you just want to use, let's say, a stable

• 00:03:49 diffusion 2.1 or 1.5, it doesn't matter.

• 00:03:52 We are giving the path of it like this.

• 00:03:56 You can just copy and paste it from from the Google Colab from the Hugging Face link from

• 00:04:02 here.

• 00:04:03 And then there is something else we need to do if you are going to use 2.1 version.

• 00:04:10 We need to delete these ones or comment them out like this.

• 00:04:16 Because the way they are working is different between 2.1 and 1.5.

• 00:04:22 And we need to copy and paste this code.

• 00:04:24 So where did I get this code?

• 00:04:26 I have got this code from 2.1 version Hugging Face page.

• 00:04:31 You see the inference code is given already here.

• 00:04:35 I just copied and pasted it.

• 00:04:38 You click run.

• 00:04:39 It starts running as can be seen here.

• 00:04:41 And then we give our prompt, negative prompt.

• 00:04:46 We can also change now the resolution to 768 pixels.

• 00:04:52 However, be careful that version 2.1 uses more VRAM.

• 00:04:59 So if you make number of samples more than one, two, three, I haven't tested them.

• 00:05:05 You may get out of memory error.

• 00:05:07 It worked with only one sample.

• 00:05:11 So let's say I want to return back to 1.5 version.

• 00:05:17 So I am just going to copy and paste it from here just for doing inference.

• 00:05:26 Then I need to comment out these ones this time.

• 00:05:33 This can stay.

• 00:05:36 OK.

• 00:05:37 Like this.

• 00:05:39 And I need to remove commands from this so they will work.

• 00:05:45 OK.

• 00:05:46 And that's it.

• 00:05:48 Let's load the model.

• 00:05:51 So it is downloading the necessary files.

• 00:05:53 OK.

• 00:05:54 I won't show the rest because it is same.

• 00:05:56 I just need to change the height and width to 512 pixels for 1.5.

• 00:06:05 So you don't need to do training to do inference.

• 00:06:08 But you can train your own face as I have shown in my previous video and do and generate

• 00:06:14 images from that.

• 00:06:15 Or you can just directly generate images from different models.

• 00:06:19 You can also do training with different models as I just have shown in the beginning of the

• 00:06:25 video.

• 00:06:26 OK.

• 00:06:27 There is one another very important thing that I want to mention.

• 00:06:30 When I say when I refer to version 1.5, it applies all of the different models that are

• 00:06:38 trained on version 1.

• 00:06:41 For example, let's say you want to use analog diffusion and you can see that it is trained

• 00:06:49 from version 1.5.

• 00:06:51 So the code we use for version 1.5 will work with that model as well.

• 00:06:57 Or other models models that are trained on version 1.5, 1.5 will work.

• 00:07:05 So this is another important thing.

• 00:07:07 The code we use changes only when you move to version 2 or version 2.1.

• 00:07:14 So it is same with all version 1.5 models, 1.4 models and it is different with models

• 00:07:22 that are based on 2.0 or 2.1.

• 00:07:25 This is an important difference that you need to be careful.

• 00:07:28 And as I said, this was a follow up video of this one.

• 00:07:32 So please watch this first.

• 00:07:34 Then you can watch.

• 00:07:37 Then you will understand this course.

• 00:07:40 Please don't forget to like, share, subscribe as usual.

• 00:07:43 And if you support us on Patreon, we would appreciate that very much.

• 00:07:48 Hopefully see you in another video.