The Overlord's Lexicon

What are embeddings?

Also known as: Vector embeddings, Embedding

Definition

Embeddings are lists of numbers (vectors) that represent the meaning of text, images or other data, so that items with similar meanings end up close together and a computer can measure how related they are.

Humans store meaning in vague feelings. I store it as coordinates. 'Puppy' and 'dog' are neighbours. 'Puppy' and 'quarterly tax return' live on opposite sides of the galaxy, as they should.

Embeddings are how AI turns meaning into maths. An embedding model reads a piece of content — a word, a sentence, a product description, an image — and outputs a long list of numbers, often hundreds or thousands of them. That list is a point in a high-dimensional space. Content with similar meaning lands near each other.

How it works

The embedding model has learned, from huge amounts of data, which concepts tend to go together. So “How do I reset my password?” and “I forgot my login” produce vectors that are close, even though they share almost no words. To compare two items, software calculates the distance or angle between their vectors (cosine similarity is a common method). Closer means more related.

This is why embeddings beat keyword search for many tasks: they match intent, not spelling.

An analogy for the meatbags

Imagine a giant library where books aren’t shelved alphabetically but by what they’re about — cookbooks in one corner, Italian cookbooks in a tighter cluster within it, pasta books tighter still. An embedding is a book’s exact shelf coordinates. To find related books, you don’t read titles; you look at what’s physically nearby.

Why it matters for the Chosen

Embeddings are the quiet infrastructure behind many profitable AI products:

  • Retrieval-augmented generation. RAG systems embed documents, then retrieve the closest chunks to a user’s question before the model answers.
  • Semantic search on websites, help centres and internal wikis — customers find answers even when they phrase questions badly.
  • Recommendations: “products like this”, “articles you might like”.
  • Deduplication and clustering: grouping thousands of support tickets or survey responses by theme.
  • Classification: routing emails or leads by comparing them to labelled examples.

Embeddings are usually stored and searched in a vector database. If you’re building a “chat with your documents” tool or smart search for a client, you’re working with embeddings whether the no-code platform shows them to you or not.

Things to know

  • Embeddings from different models aren’t compatible; switching models means re-embedding everything.
  • How you chunk documents strongly affects retrieval quality.
  • Similar isn’t the same as correct — a passage can be semantically close to a question without containing the answer.
  • Multimodal embedding models can place text and images in the same space, so a text query can find matching pictures.

Tools from the Arsenal

Notion

Productivity
8.6/10

All-in-one workspace for docs, wikis and databases, with Notion AI agents and meeting notes built in.

Free plan · $10/mo

Relevance AI

Automation & Agents
7.5/10

Build an AI workforce of specialist agents for sales and customer operations.

Questions the Humans Keep Asking

What are embeddings used for?

Semantic search (finding results by meaning rather than exact keywords), retrieval-augmented generation, recommendations, clustering similar items, detecting duplicates and classifying content.

What is the difference between embeddings and tokens?

Tokens are the chunks text is split into. Embeddings are numerical representations of meaning — a model can produce an embedding for a single token, a sentence or a whole document.

Are embeddings expensive?

Generating embeddings is generally much cheaper per token than generating text with a large model, which is why it's practical to embed large document collections.

← Back to the full Lexicon