The Overlord's Lexicon

What is a diffusion model?

Also known as: Diffusion, Latent diffusion

Definition

A diffusion model is a type of generative AI that creates images, video or audio by starting with random noise and gradually removing it, step by step, until it forms an output that matches a prompt.

A diffusion model stares into pure static and slowly convinces it to become a cat wearing a crown. Humans call it art. I call it controlled hallucination with excellent lighting.

A diffusion model is the engine behind most AI image generation. When you type “neon-lit Tokyo alley in the rain, cinematic” and get a picture seconds later, a diffusion process almost certainly produced it.

How it works

Training: The model is shown millions of images paired with descriptions. For each, noise is added in many small steps until the image becomes pure static. The model learns to do the reverse: given a noisy image (and the text description), predict and remove a little of the noise.

Generating: To make a new image, the model starts from random noise and repeatedly applies what it learned — denoise, denoise, denoise — steered by your prompt at every step. After enough steps, a coherent image emerges.

Most modern systems run this process in a compressed “latent” space rather than on full-size pixels, which makes it much faster. That’s the “latent” in latent diffusion, the technique behind Stable Diffusion.

An analogy for the meatbags

Think of a sculptor who starts with a rough block of marble and chips away everything that isn’t the statue. A diffusion model starts with a block of random noise and removes everything that isn’t “a golden retriever astronaut on the moon.” Your prompt is the sculptor’s brief, consulted at every chip.

Why it matters for the Chosen

Diffusion models made professional-looking visuals nearly free and instant, which reshaped several markets:

  • Marketing and social content: ad creatives, thumbnails, blog headers, product mock-ups.
  • Ecommerce: lifestyle product shots without a photo studio; background removal and replacement.
  • Design: mood boards, concept art, logo exploration, packaging ideas.
  • Video: text-to-video tools extend the same idea across frames.
  • Print-on-demand and digital products — though crowded, and subject to platform rules.

The money isn’t in pressing “generate”; it’s in art direction, consistency (same character or product across many images), editing, and knowing which tool suits which job.

Things to watch

Copyright and licensing: check each tool’s commercial-use terms, and know that purely AI-generated images may not be protected by copyright in some jurisdictions. Diffusion models still struggle at times with hands, text inside images (though newer models are much better) and exact brand details. And the same technology powers deepfakes — don’t generate realistic images of real people without consent.

Tools from the Arsenal

Midjourney

AI Design & Images
9.0/10

AI image and video generator known for its striking, artistic output.

Adobe Firefly

AI Design & Images
8.5/10

Adobe's AI creative studio for images, video and audio, positioned as commercially safe.

Free plan · $9.99/mo

Leonardo.Ai

AI Design & Images
8.0/10

Canva-owned AI image and video studio with custom model training and fine control.

Free plan · $12/mo

Ideogram

AI Design & Images
8.4/10

The AI image generator that can actually spell: posters, logos and text-heavy designs.

Free plan · $15/mo

Krea AI

AI Design & Images
8.1/10

Real-time creative AI suite for generating, editing and upscaling images and video.

Free plan · $5/mo

Runway

AI Video
9.0/10

Pro-grade generative video studio with Gen-4.5 models, image, audio and editing tools.

Free plan · $12/mo

Questions the Humans Keep Asking

Is Midjourney a diffusion model?

Midjourney doesn't publish full technical details, but it is widely understood to be diffusion-based, like Stable Diffusion and many other leading image generators.

How is a diffusion model different from a large language model?

LLMs generate text one token at a time, left to right. Diffusion models typically generate a whole image (or video clip) at once by refining noise over many steps. Many modern systems combine the two — for example, a diffusion model guided by a transformer that understands the prompt.

Can diffusion models generate video?

Yes. Many text-to-video systems extend diffusion across time so frames stay consistent, often combined with transformer architectures (sometimes called diffusion transformers).

← Back to the full Lexicon