Module 01 · Lesson 2 of 13Free

The AI application stack: from model to product

Level

Beginner

Topic

AI engineering fundamentals

Reading time

11 min

Tomás’s prototype rests on work he didn’t do

Deskora is a made-up help desk app. A shop’s support team uses it to answer customer emails from one shared inbox. Each customer problem becomes a ticket: one customer’s whole conversation with the support team, message by message.

Priya is a support agent at a shop that uses Deskora. When she picks up a long ticket, she spends about five minutes reading it before she can reply. Tomás, an engineer at Deskora, has built a prototype that puts a short summary at the top. It is about 30 lines of Python, and the model call is five of them.

The model behind those five lines was trained by a provider, on huge amounts of text. It runs on the provider’s servers, on specialised chips called GPUs. Deskora owns none of that.

Soon 9,000 agents at 1,500 shops will use the feature, on some of Deskora’s 120,000 new tickets a day. Before launch, Tomás’s team lead asks him for a list: what Deskora has to build, and what it pays someone else for.

This lesson answers one question: which parts of an AI feature does Deskora build and own, which parts does it rent, and how does that change the way Tomás works?

Three layers, and Deskora owns only the top one

Every AI product sits on a stack of three layers. Each layer depends on the one below it.

• The infrastructure layer is the hardware and software that run a model: data centres, GPUs and serving software that answers calls from many customers at once. Running a trained model to produce output is called inference.
• The model layer is the trained model itself, such as Claude Haiku 5.5. Module 02 explains how a model is trained.
• The application layer is everything that turns a model into a feature someone uses. For Deskora, that is the summary feature. It is Tomás’s job.

Chip Huyen’s book on AI engineering uses the same three layers.^{4AI Engineering: Building Applications with Foundation ModelsChip Huyen, O’Reilly Media, 2025}

The model API is the line between what Deskora rents and what it owns. Tomás reaches the two lower layers only through it. He picks a model id, sends a prompt and reads what comes back. Choosing a model shows how to pick between the models a provider offers.

Deskora still runs its own web servers, database and background jobs. In this stack, those belong to the application layer. The infrastructure layer means only the computers that run the model.

Some teams run an open-weight model on their own GPUs instead, which brings the infrastructure layer in-house; Hosted and open-weight models compares the two. Renting both lower layers is why Deskora’s prototype took one afternoon. It doesn’t remove the work, though. It moves all of it into the top layer.

Six parts make up Deskora’s layer

Introduction to AI engineering followed one summary through five steps. Those steps are one request’s path. The application layer adds two parts outside any single request: evals check a change before it ships, and monitoring watches every call in production.

Most of this table is familiar web work: a page, a database, a background job, logs. Three things in it are new for a web team: the prompt, the eval and the bill. The provider adds a fourth.

The same tools, four new habits

Many engineers see AI engineering as full-stack work plus one API call. Half of that is right: Tomás uses the same Python, database, Git repository and code review.

One line changed the model, the price and the output at once. Ordinary web code rarely works like that, and four habits follow.

Prompts get versions, like code

The prompt decides what every summary contains. Change “list every promise” to “mention key promises”, and 40,000 summaries a day change. So the prompt is treated as code: it is reviewed, it has a version, and every saved summary records which version made it.

Here is an example of what Deskora saves with each summary:

{
  "ticket_id": 5203,
  "summary": "Problem: lamp arrived with a broken shade.\nPromised: a new shade by Thursday.\nWaiting for: the tracking number.",
  "prompt_version": "summary-v7",
  "model": "claude-haiku-5-5",
  "input_tokens": 4163,
  "output_tokens": 377,
  "thinking_tokens": 296,
  "cost_usd": 0.000605,
  "created_at": "2026-10-28T09:14:03Z"
}

When Priya flags a bad summary, Tomás can see which prompt and which model made it, and what it cost. Where the prompt itself lives is a real choice.

Evals run next to unit tests

Unit tests still check the code around the call, such as whether the output check rejects a cut-off summary. They can’t judge whether a summary kept the promise made to the customer.

An eval does that: it runs the feature on many real tickets and scores each summary. Anthropic’s guide advises evals that mirror the real mix of inputs, edge cases included.^{3Define success criteria and build evaluationsAnthropic, Claude API documentation} It also advises grading them automatically where possible.^{3Define success criteria and build evaluationsAnthropic, Claude API documentation}

Tomás’s prompt v7 keeps the key facts in 34 of 40 eval tickets. His v8 adds an instruction to quote order numbers. The unit tests pass, but the eval drops to 31 of 40: three summaries now leave out the promised delivery date. So v8 doesn’t ship.

The 40 tickets and both scores are example numbers.

An eval gives a score, not a pass or a fail. The team decides what score is good enough. Evals shows how to build one.

Every call has a price

In a web app, a code change almost never changes the monthly bill. Here, every call is billed by the token, a small piece of text. So the length of a prompt has a price.

Suppose v8 had also added 1,000 tokens of examples to the instructions. As of October 2026, Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, for prompts up to 100,000 tokens.^{2PricingAnthropic, Claude API documentation}

Estimates for this scenario: the thread sizes, the thinking tokens, and 40,000 summaries a day, 10,000 of them for long tickets.

One edit raised the bill by about 11%, so a cost check belongs in the review of every prompt change. The thinking counts are assumptions; each response reports the real number in usage.output_tokens_details.thinking_tokens. Module 10 covers ways to cut the bill, such as prompt caching and sending easy work to smaller models.

The provider retires the model you built on

A library pinned in a lockfile stays the same until you change it. A hosted model doesn’t last forever. Providers retire older models as new ones launch, and requests to a retired model fail.^{1Model deprecationsAnthropic, Claude API documentation} The model is a dependency Deskora doesn’t control.

Anthropic gives at least 60 days’ notice before it retires a publicly released model.^{1Model deprecationsAnthropic, Claude API documentation} As of October 2026, its deprecations page lists Claude Sonnet 4.5 to retire on November 30, 2026, with Claude Sonnet 5.5 as the replacement.^{1Model deprecationsAnthropic, Claude API documentation} The earliest retirement date it lists for Claude Haiku 5.5 is October 7, 2027.

So one day Tomás will move the summary to a new model. The new model writes different summaries at a different cost. He treats the move like a prompt change: run the eval, compare the cost, then switch. Anthropic recommends testing newer models well before the retirement date.^{1Model deprecationsAnthropic, Claude API documentation} Model upgrades covers doing that safely.

Where this shows up

• Design: a ticket summary feature builds every part of Deskora’s layer, with real numbers.
• Design: an AI customer support platform runs many such features, each with its own prompts, evals and costs.

The next lesson, Calling a model API, looks closely at the line between the layers: what one call sends, what comes back and what it costs.

Common mistakes

• Expecting the provider to own quality. The provider makes a general model. Deskora must measure whether it summarizes Deskora’s tickets well.
• Shipping a prompt edit on unit tests alone. The tests can’t see the summaries. Run the eval too.
• Saving outputs without their prompt version and model. When a bad summary appears, nobody can trace what made it.
• Treating the model like a pinned library. Plan the move to a new model before the retirement notice arrives.

Check your understanding

Summary