How to Write Killer Stable Diffusion Prompts for AI Images

In the age of generative AI, the best prompts will rule the world. Good prompts will net you the AI results that you want. Bad prompts will net you bad results and in most cases results that are unusable. In this article we will talk about how to write killer stable diffusion prompts.

What is Stable Diffusion?

Stable Diffusion is a text to image AI model by Stability.ai, used to generate AI images. Stable Diffusion was the first text to image model made open source which allowed consumers to run on their own hardware versus accessing the model through web services like Midjourney and Dall-E. With that said, Stability.ai does also offer their own web app called DreamStudio.

How to Get Started with Stable Diffusion?

For those looking to try Stable Diffusion, check out DreamStudio which is the official Stable Diffusion app. I’ve also built a website powered by Stable Diffusion.

If you’re using the official Stable Diffusion app you’ll notice that the way they handle a prompt is different than Dall-E or Midjourney. Dall-E and Midjourney allows the user to set all image parameters in the prompt whereas parameters such as styles, negative prompts, image dimensions, and etc are decoupled from the prompt and instead displayed as configurable settings in the left navigation.

I like this better because it presents the parameters up front versus hidden away from view allowing you to simply just focus on the prompt itself.

How to Write Killer Stable Diffusion Prompts

A prompt can be crafted in infinite ways but I break it down to three components.

[Core Prompt] , [Style] , [Finishing Touches]

1. Core Prompt

Your prompt can be as simple as one word. What is the subject or theme of the image that you’re trying to create. Its easier to envision what your ideal subject looks like and in what environment the subject is in and then describe it using words. Add specific adjectives to further give the subject character.

Core Prompt Examples

• A fox
• A muscular fox
• A fox with a nut in a tree

2. Style

If you submit your prompt without a style, the model will default to commonalities of the core prompt you’ve described. Adding in styles is enough to steer the model to realized your concepts. Multiple styles can be chained together separated by commas. Additionally, consider including literal art styles (oil painting, anime, pencil) and famous artists (Vincent Van Gogh, Michelangelo) to make your style more specific.

Style Prompt Examples

• A fox, in the 1950s, oil painting by Leonardo DaVinci
• A muscular fox, high contrast colors, hyperrealistic
• A fox with a nut in a tree, surrounded by snakes

3. Finishing Touches

Add finishing touches towards the end of your prompt for more fine tune control of how you want your image to look. This is the part where prompters are pushing the boundaries.

Finishing Touches Examples

• Image quality: 4k, 8k, 720p
• Image processing: washed out, sharp, high contrast
• Medium: water colors, pencil, chalk
• Mood: Sci fi, dark, colorful, bright

Additional Tips

1. Subject: The prompt always starts off with the main subject
2. Specific: It understands object specifics. You can say “towering robot statue” vs “robot”
3. Environment: Describe the environment that the subject is in separated by commas.
4. Avoid: The AI doesn’t quite know what to do with quantity of items and relative sizes.
5. Details: You can describe the subject, medium, time (past, present, future), lighting, color, mood and composition.
6. Length: Be mindful of length. It seems to put more weight towards words in the beginning. As you chain more words, the words start to get less effect toward the end, and in worst case scenarios it gets confused on the entire prompt.

Negative Prompts

Think of negative prompts as exclude statements. Whatever you don’t want to see in the generated image, include it in the negative prompt. Like the prompt itself, you can chain them together with commas.

Here are some negative prompts to consider:

1. Bad [noun]: Example - bad hands, bad arms, bad face
2. Poorly drawn [noun}: Example - poorly drawn face, poorly drawn hands
3. Extra [noun]: Example - extra fingers, extra eyes
4. Censorship: Example - nsfw, nude, censored

Good and Bad Prompt Examples

Auto

• Bad Prompt: really cool fast car

• Good Prompt: Luxury sports car with aerodynamic curves, shot in a high contrast, high key lighting with shallow depth of field, exotic, detailed, sporty, studio lighting

Scenery

• Bad Prompt: a big island and inside the island there is a volcano that is erupting really big into the sky, realistic like a photograph

• Good Prompt: Volcanic island with a boiling crater and ash clouds rising into the air, intense, dramatic, natural disaster, high detail, volcanic landscape

People

• Bad Prompt: I want an image of a person that is shooting arrows, they are standing in the forest, the person is in a very focused state and almost about to shoot the arrow, there are trees in the background but the person is standing in a clearing so the person is not overwhelmed with trees.

• Good Prompt: Skilled archer, bow and quiver of arrows, standing in forest clearing, intense, detailed, high detail, portrait

In Conclusion

In conclusion, it’s easy to get caught up on all the technicalities of a prompt. At the end of the day this is my best advice, imagine that you are a photographer. Unfortunately you’re working offsite and is unable to be on location to capture a specific photo. Instead you have to send one of your disciples to capture the photo. They are an extension of your consciousness. In order to capture the exact photo that you want, you’ll have to describe what your photographer to capture in an explicit and concise way. This description is your prompt. Good luck and have fun!

‍

‍