Stable Diffusion: How to Use VAE
Table of Contents
AI startup Stability AI has been a leading force in the world of generative AI with its diffusion model Stable Diffusion. Each update has seen the artificial intelligence software get a stronger grasp of generating realistic human features. VAE is a limited update for the Stable Diffusion, available in only the 1.4 and 1.5 models. It aims to help users create an improved version of their image rendering of human eyes and text accuracy. This article will let you know exactly what VAE is, and how to use VAE step-by-step.
What is Stable Diffusion VAE?
VAE stands for Variational Autoencoder, and in essence they are neural networks trained to replicate their input data to their output. They are often used for the purpose of dimensionality reduction. However, VAE takes further measures to be accurate by adding in a regularization process during training to prevent overfitting. This ensures that the latent space, the intermediate representation of the data, possesses all the desirable properties that help to make generative processes work more efficiently.
Essentially, VAEs are powerful tools that work alongside Stable Diffusion models to up the quality of the AI-generated images. An enhanced VAE improves the decoding process, leading to faster image recovery with sharper fine details with better rendering of human features such as hands and faces.
Stable Diffusion already has a built-in VAE, but you can and external VAEs that might give you better results. It’s good to know that VAEs can actually come coded in different formats, including vae.pt and .ckpt formats, as well as a newer format that’s called .safetensors.
How to Use VAE?
To use the VAE, you first need to download it into your user interface. Currently, there are two improved versions available on Hugging Face.
You can get these files by using the below download link for both EMA and MSE:
Step
Installation
- The VAE installation procedure implies the AUTOMATIC1111 GUI.
- Put the downloaded files into the directory as stable. Diffusion.WebUI/models/VAE
Step
Usage
- To use the VAE in your AUTOMATIC1111 GUI model, click Settings and then the Stable Diffusion section on the left.
- Find the Section known as SD VAE.
- In the following menu, select any VAE file you wish to use.
- Now click the Apply Settings tab on the top, and it will apply the settings successfully after the loading.
If you cannot find the settings, click on the Show All Pages option on the left side. It will show all the settings on a single page. Now use CTRL+F to find the settings you want.
Is VAE Necessary to Install for Stable Diffusion?
No, installing VAE files is unnecessary, as the default VAE is already available with the V1 and V2 models. When people talk about installing the VAE, they refer to downloading and installing the enhanced versions of the VAE alongside the Stable Diffusion checkpoint models for better results. If you’re feeling unsure on where to begin with Stable Diffusion AI, you can try reading our guide on how to use Stable Diffusion to get you started.
Conclusion
The above short steps were meant to educate you about downloading and installing the improved VAE versions. StabilityAI released them specifically for the 1.4 and 1.5 models to provide further noticeable improvements to rendering faces and eyes in an image.
However, it is not compulsory to use an extra variable auto encoder, and it depends entirely upon the user’s will to download and use them. Using them does not harm or disintegrate your image, as they are optimized for realistic models. If this article has got you interested in the technology behind these text-to-image generative AI’s, we’ve outlined exactly how Stable Diffusion works.
About the Author