|

How To Speed Up Stable Diffusion (9 Methods That Work)

One of the significant advantages of using Stable Diffusion locally on your computer is that you can generate images without any limits or restrictions for free. 

But running Stable Diffusion locally also means that you need to have a powerful enough computer that can handle image generation. The better your computer is, the faster image generation will be in Stable Diffusion. 

However, If you don’t have a powerful enough computer and struggle with generating images faster in Stable Diffusion, this article is for you. 

In this guide, I’ll show you how to speed up Stable Diffusion so you don’t have to wait long for image generation. 

I’ll go through different methods and tips you can apply to speed up Stable Diffusion on your computer. 

That being said, let’s get started. 

How To Speed Up Stable Diffusion

If you want to make Stable Diffusion faster, follow the methods listed below that will help you a lot in improving image generation times. 

Use xFormers

Stable Diffusion comes with an option to enable cross-attention optimization which is an effective way to speed up Stable Diffusion. 

Many cross-attention optimization techniques include Doggettx, sdp-no-mem, Sub-quadratic attention, and more. 

One such cross-attention technique is xFormers which is a transformer library developed by the Meta AI team. This technique optimizes attention operation, reduces memory usage, and enhances image generation significantly. 

Using xFormers can make a night and day difference in image generation speeds as it’s the best and faster among every other cross-attention optimization technique. 

If you’re using Automatic1111 WebUI for Stable Diffusion, you can enable xFormers by going to the Settings > Optimization page and choosing xFormers under the cross-attention optimization technique. 

Make Stable Diffusion Faster

But if you already use xFormers, you can follow the other methods in this guide to further speed up Stable Diffusion. 

Use Smaller Image Dimensions

Using smaller image dimensions can significantly improve image generation times in Stable Diffusion.

Generating high-resolution images in Stable Diffusion can take a lot of time especially if you have a weak GPU. 

By reducing the image dimensions, you can speed up this time by a lot. 

Here are a few Stable Diffuson image sizes you can try: 

  • 512×512
  • 512×768
  • 768×512
  • 640×640
  • 768×768

If you don’t have a powerful enough GPU or high vRAM, I recommend avoiding dimensions over 1024×1024 pixels. 

That’s because generating images at this size will not be fast enough and you can often run into insufficient memory errors as well. 

And if you’re wondering how will your images look high quality at such low resolutions, then you can always upscale your images in Stable Diffusion by using one of the upscalers available. 

Use Token Merging 

Another technique to speed up Stable Diffusion is to utilize Token Merging which reduces the number of tokens (words in positive & negative prompt) to be processed. 

During image generation, many tokens are often inconsequential or redundant and they can be combined through Token Merging. 

Although Token Merging can change the output image, you can still get away with it by setting a low Token Merging value in Stable Diffusion. 

Speed Up Stable Diffusion

In Automatic111 WebUI for Stable Diffusion, go to Settings > Optimization and set a value for Token Merging. You can set a value between 0.2 to 0.3 which is 20-30%. 

Setting a value higher than that can change the output image drastically so it’s a wise choice to stay between these values. 

By utilizing Token Merging, you can improve image generation times by a bit but it’s not a significant improvement. Nonetheless, it’s worth a try.  

Related: Stable Diffusion Male Models

Reduce Sampling Steps

Sampling steps are the number of iterations Stable Diffusion goes through in order to generate an image. The higher the sampling steps, the more time Stable Diffusion will take to generate your image. 

You can reduce sampling steps to improve image generation speeds in Stable Diffusion but it’s important to remember that low sampling steps could also result in bad-quality images.

So, you’ll have to use a sampling step that’s a perfect sweet spot for you in terms of image generation time and quality of the image. 

Fortunately, I’ve done this for you by conducting a test where I generated the same image at different sampling steps. 

Images Generated At Different Sampling Steps In Stable Diffusion

From the above results, you can see that the details and quality changes at different sampling steps. 

I find a sampling step between 30-35 a sweet spot as images look good and it’s fast enough as well. 

But if you struggle even at 30-35 sampling steps, try setting it at 20-25 which will be comparatively faster and the image quality won’t be affected drastically. 

Ensure Stable Diffusion Is Using Your GPU

By default, most Stable Diffusion Web UIs such as Automatic1111, ComfyUI, or Easy Diffusion are designed to use your GPU during image generation automatically. 

But in some rare cases of misconfiguration or installation errors, Stable Diffusion fails to use the GPU and instead uses the CPU for image generation. 

And since Stable Diffusion performs poorly when running on CPU instead of GPU, it might be the cause of slow image generation. 

So, the first thing you need to do is ensure that Stable Diffusion is actually using your GPU.

You can do this easily by checking your GPU usage in the Task Manager. To do this, open your Task Manager and go to the Performance tab. 

Speed Up Stable Diffusion

Here, check the GPU performance while generating an image in Stable Diffusion. 

Usually, when running Stable Diffusion, your GPU utilization should be around 50-75% or above. 

If your GPU utilization is below this, chances are that your GPU isn’t being utilized for image generation. 

However, the Task Manager often fails to show proper GPU usage when running Stable Diffusion. In such cases, you can look at the GPU temperature which can be a good indicator of whether your GPU is being used or not. 

You can also switch to the CUDA graph in the GPU dropdown to check the true GPU utilization. But if you have Hardware-Accelerated GPU Scheduling enabled, you’ll have to check the 3D graph instead. 

If Stable Diffusion is not using your CPU instead of GPU, it is most likely due to bad configuration or installation errors. 

You can get this fixed by figuring out what’s misconfigured but I recommend reinstalling Stable Diffusion entirely as it’s a much faster fix. 

Upgrade/Downgrade GPU Drivers

Your GPU drivers can have a massive effect on how fast Stable Diffusion runs. Upgrading your graphic drivers to the latest version can help make Stable Diffusion faster. 

But this hasn’t been the case for most users including me who have Nvidia GPUs. 

A while back when I updated my Nvidia drivers, I noticed a significant slowdown while running Stable Diffusion. 

On further inspection, I realized that this is a widespread issue in the Stable Diffusion community and you can find posts about it on Reddit like this, this, and another one here on GitHub. 

To summarize, Nvidia GPU drivers after version 531 have slowed down image generation times in Stable Diffusion a lot. 

In personal tests, I found that generating a 1024×1024 image with 4K upscale took me 315.63 seconds (~5 minutes) with the Nvidia 531 driver. 

But after upgrading to the latest Nvidia 537 drivers, the same image took 847.94 seconds (~14 minutes) to generate. 

And I’m not alone as evident from many users talking about it on Reddit and GitHub. 

So, if your Nvidia GPU driver is running on version 532 or later, you might be facing a slowdown as well. 

You can downgrade to the Nvidia 531 drivers from their website and test it out yourself to see if the older version makes Stable Diffusion faster. 

Switch To A Different Web UI

Many Stable Diffuson users are using the Automatic1111 Web UI which is by far the most popular and user-friendly interface for running Stable Diffusion. 

But at the same time, it’s not the best Web UI for Stable Diffusion when it comes to performance. 

There are many other Stable Diffusion Web UIs that are well-optimized and run faster than Automatic1111. 

Here are some Stable Diffusion Web UIs you can try out to run Stable Diffusion much faster: 

Personally, I’ve switched from using Automatic1111 to ComfyUI and I’m now able to generate images much faster. 

ComfyUI - Default Workflow

For reference, I run Stable Diffusion on a Lenovo Legion 5 laptop which comes with an Nvidia RTX3050 GPU with 4GB of vRAM. 

With Automatic1111, I was unable to generate images at a resolution higher than 521*768. Moreover, I wasn’t able to run ControlNet or use SDXL models

After switching to ComfyUI, I’m now able to generate images at 1024×1024 size and upscale them to 4K using SDXL models. 

Moreover, I’m also able to use ControlNet which was previously impossible in Automatic1111 due to insufficient memory errors. 

That’s because ComfyUI is optimized way better and is significantly faster. The same can be said for InvokeAI which is another great Web UI for Stable Diffusion.

So, if you’re using Automatic111 and struggle with frequent memory errors or slow image generation speeds, switching to a new Web UI can help you a lot. 

Yes, you’ll have to learn how the new Web UIs work but it might be worth the time as you’ll be able to generate much better images in faster times. 

Disable Unnecessary Apps & Services

If you’re running other apps or services on your computer while running Stable Diffusion, it could likely increase image generation time. 

While running Stable Diffusion, it’s a good idea to disable any unnecessary apps or services. To do this, simply open Task Manager and check the apps and services running. 

You can disable background apps and services such as Steam, your anti-virus software, Adobe update services, and anything else that might take up your RAM. 

While this may not have a significant impact on Stable Diffusion speed, it can still be helpful especially if your system already isn’t powerful enough. 

Reinstall Stable Diffusion 

If Stable Diffusion is still slow after doing everything mentioned above, the last resort to improve speed is to reinstall Stable Diffusion completely. 

This could be helpful in making Stable Diffusion fast as reinstalling can fix any errors or misconfigurations you had previously. 

On top of that, reinstalling Stable Diffusion will also ensure you’re running the latest version and everything else is up to date as well. 

FAQs

Here are some frequently asked questions on how to make Stable Diffusion faster. 

What are the system requirements to run Stable Diffusion?  

Stable Diffusion requires a modern Intel or AMD processor with at least 16GB of RAM, an Nvidia RTX 3060 GPU with atleast 6GB of VRAM, and atleast 10GB of storage space

Should I use CPU or GPU while using Stable Diffusion? 

You should always use your GPU while using Stable Diffusion as it’s more efficient and faster in generating images compared to the CPU. 

What’s the best GPU to run Stable Diffusion?

The best GPU to run Stable Diffusion currently is the Nvidia RTX 4090 with 24GB of vRAM as it’ll be able to generate images insanely fast. 

Conclusion 

There are many ways to make Stable Diffusion faster and we covered a lot of the methods in our guide. 

With the methods shared in this guide, you’ll be able to speed up Stable Diffusion easily as I’ve followed some of these tips myself when I wanted to make Stable Diffusion fast. 

However, if your computer doesn’t meet the system requirements for Stable Diffusion, then these tips won’t do much for you. 

In that case, your only solution would be to upgrade your hardware especially your GPU which is the main factor in running Stable DIffusion. 

If you have any questions or doubts regarding Stable Diffusion, feel free to drop your comments below.

Stable Diffusion Prompt Organizer

One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.