Module 02 · Lesson 2 of 12Free

Embeddings: turning words into numbers with meaning

Level

Beginner

Topic

How LLMs work: what builders need to know

Reading time

13 min

Two tickets with no words in common

Deskora is a help desk app. A shop’s support team uses it to answer customer emails from one shared inbox. Each customer problem becomes a ticket: one customer’s whole conversation with the support team.

Priya is a support agent at a kitchenware shop that uses Deskora. A new ticket arrives from Daniela: “My parcel never came.” The shop has 40,000 past tickets, and dozens of them solved this exact problem. Priya would like to see how.

Tomás, an engineer at Deskora, is building a “Similar past tickets” panel for her. His first version uses keyword search. It ranks past tickets by the words they share with the new one, ignoring small words such as “my” and “I”. It looks words up in an inverted index, so it is fast.

The ticket Priya needs says “Package not delivered.” It means the same thing and shares no words.

This lesson answers one question: how can software tell that two pieces of text mean the same thing when they share no words?

The 40,000 past tickets, and the other numbers in this lesson except Deskora’s 1,500 shops and 120,000 tickets a day, are estimates for this lesson’s scenario.

An embedding is a list of numbers

An embedding is a list of numbers that stands for a piece of text. A model produces it. It places the text so that texts with similar meaning get similar numbers. A list of numbers like this is called a vector.

Here is the start of a real one, from Anthropic’s embeddings guide. Voyage AI’s voyage-4 model turned the text “Sample text 1” into 1,024 numbers ^{1EmbeddingsAnthropic, Claude API documentation}:

{
  "object": "embedding",
  "embedding": [-0.013131560757756233, 0.019828535616397858 /* ... 1,022 more */],
  "index": 0
}

No single number has a meaning you can name. The meaning is in how the whole list compares with other lists.

To see how this works, take a hand-made version with two numbers per ticket. Pretend the first number means “about a delivery that didn’t arrive” and the second means “about a damaged item”:

The two missing-parcel tickets point almost the same way, with no shared words. The damaged-parcel ticket, keyword search’s top match, points far away.

To rank 40,000 tickets, Tomás needs “almost the same way” as a single number.

Closeness is one number: cosine similarity

Cosine similarity measures how closely two vectors point the same way. It ignores how long they are:

The score runs from 1 (same direction) through 0 (unrelated) to −1 (opposite).

The ticket Priya needs now ranks first, with 0.99 against 0.50.

Voyage AI returns vectors already scaled to length 1 ^{1EmbeddingsAnthropic, Claude API documentation}. So does OpenAI ^{2Vector embeddingsOpenAI, OpenAI API documentation}. For such vectors the plain dot product gives the same score, faster. Similarity measures compares cosine, dot product and distance in full.

Here is the same arithmetic in plain Python, with a third ticket added. Run it with any Python 3.9 or newer; it needs no libraries and no API key.

import math


def cosine(a: list[float], b: list[float]) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    length_a = math.sqrt(sum(x * x for x in a))
    length_b = math.sqrt(sum(y * y for y in b))
    return dot / (length_a * length_b)


# Hand-made vectors: (delivery didn't arrive, item damaged, account problem)
new_ticket = [0.9, 0.1, 0.0]  # "My parcel never came"
past_tickets = {
    "Package not delivered": [0.8, 0.2, 0.0],
    "My parcel came damaged": [0.4, 0.9, 0.0],
    "How do I reset my password?": [0.0, 0.1, 0.9],
}

ranked = sorted(
    past_tickets.items(),
    key=lambda item: cosine(new_ticket, item[1]),
    reverse=True,
)
for text, vector in ranked:
    print(f"{cosine(new_ticket, vector):.2f}  {text}")

It prints 0.99, 0.50 and 0.01, in that order.

Two numbers can’t hold every meaning

Look at the password ticket in the code. With only the two numbers, “delivery” and “damage”, it has nowhere to go: it is about neither. Tomás had to add a third number, “account problem”, to place it. Its score of 0.01 against Daniela’s ticket says the two are unrelated.

Real tickets differ in hundreds of ways: refunds, sizes, discount codes, tone. Each needs room in the list, so real embeddings are long. As of October 2026, voyage-4 returns 1,024 numbers by default ^{1EmbeddingsAnthropic, Claude API documentation}. OpenAI’s text-embedding-3-small returns 1,536 and text-embedding-3-large returns 3,072 ^{2Vector embeddingsOpenAI, OpenAI API documentation}. The length of the list is the embedding’s dimension.

In a real embedding nobody chooses what each number means; the model learns where to place texts during training. Cosine similarity works the same way on 1,024 numbers as on two. But where do the numbers come from?

An embedding model makes the numbers

An embedding model is a model that reads text and returns a vector. It does not write text. You call it through its own API, separate from a text-generating API such as the Claude Messages API covered in Calling a model API. The request carries a list of texts and a model name; the response carries one vector per text and a count of the tokens it read ^{1EmbeddingsAnthropic, Claude API documentation}.

Anthropic doesn’t offer its own embedding model. Its documentation points to Voyage AI as one provider ^{1EmbeddingsAnthropic, Claude API documentation}. OpenAI also offers embedding models ^{2Vector embeddingsOpenAI, OpenAI API documentation}. Embedding models covers how to choose one.

For the panel, the work splits in two:

1. Once per past ticket, in the background: send its text to the embedding model and save the vector next to the ticket.
2. When a new ticket arrives: embed it with the same model, compare its vector with the shop’s saved vectors, and show the top three.

The same vectors serve other features. Builders use embeddings for search, grouping similar items, recommendations, spotting unusual items and sorting items into categories ^{2Vector embeddingsOpenAI, OpenAI API documentation}. For Deskora, that could mean grouping this week’s tickets to find that 300 are about one courier, or suggesting help-centre articles to Priya. Finding the right documents to hand a model before it answers is RAG, built on the same search.

Three ways to find similar tickets

A language model also understands that “never came” means “not delivered”. Why not ask Claude?

The unit of cost is one completed task: one “Similar past tickets” panel. Asking Claude means one call per past ticket, 40,000 calls a panel. As of October 2026, Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens ^{4PricingAnthropic, Claude API documentation}. It thinks by default, and thinking is billed as output.

Twenty thousand calls are also far too slow for a panel Priya sees on opening the ticket. Research on an older model found the same pattern. Finding the most similar pair among 10,000 sentences by comparing every pair took about 65 hours; with embeddings it took about 5 seconds ^{3Sentence-BERT: Sentence Embeddings using Siamese BERT-NetworksNils Reimers and Iryna Gurevych, EMNLP 2019, 2019}.

Where embeddings fall short

• Exact identifiers. “Order 48213” and “order 48231” differ by two digits. Their vectors can be nearly the same, and keyword search tells them apart better.
• Small words that flip meaning. “Parcel delivered” and “parcel not delivered” share almost all their meaning, and can score close together. Priya might see a ticket about the opposite problem.
• Scale. At the default 32-bit size, 1,024 numbers take 4,096 bytes ^{1EmbeddingsAnthropic, Claude API documentation}. Deskora’s 120,000 tickets a day add about 490 MB of vectors a day. Comparing one new ticket with all of them, one by one, gets slow. Vector databases store vectors and search them without checking every one.

Where this shows up

• Design: a support assistant over company docs embeds help articles so it can find the ones that answer a customer’s question.
• Case: Intercom’s custom reranker for support RAG re-orders the passages a first search returns, a step that only helps once a search like this one exists.
• Design: an AI customer support platform finds past tickets and help articles by meaning, next to summaries and reply drafts.

Inside the language models later lessons open up, each token also starts as a list of numbers. Its numbers must then change with the words around it, so that “charge” in “charge my card” and “charge my phone” mean different things. How the model does that is the subject of Attention.

Common mistakes

Check your understanding

Summary