Module 01 · Lesson 1 of 13Free

Introduction to AI engineering

Level

Beginner

Topic

AI engineering fundamentals

Reading time

14 min

Priya opens a 24-message ticket

Deskora is a made-up help desk app. A shop’s support team uses it to answer customer emails in one shared inbox. Each customer problem becomes a ticket: one customer’s whole conversation with the support team about that problem, message by message.

Priya is a support agent at a kitchenware shop that uses Deskora. She starts her shift and picks up Halima’s ticket. Halima’s blender arrived cracked. The ticket has 24 messages, and two other agents have already replied. Priya needs about five minutes to read it before she can answer.

About 1,500 shops use Deskora. Between them, they have 9,000 support agents and get 120,000 new tickets a day. Many tickets are this long. So Deskora decides to build its first AI feature: a short summary at the top of each long ticket.

Tomás, an engineer at Deskora, builds a prototype in one afternoon. His code sends the ticket to a language model, a program trained on huge amounts of text that can follow instructions such as “summarize this”. A summary comes back in a few seconds, and it’s good.

The second summary used different words. It also left out the most useful fact: an earlier agent had promised Halima a replacement by Friday.

The team are good web engineers, but none of them has shipped a feature built on a language model. So the question this lesson answers is: what does it take to turn one model call into a feature 9,000 agents can trust?

You build on a model someone else trained

That work has a name: AI engineering. It means building products on top of a pretrained model, usually by calling it through an API.

A pretrained model has already been trained on a very large amount of text before you use it.^{1GlossaryAnthropic, Claude API documentation} You don’t train it. A provider runs it on its own servers. Your code sends text to the provider’s API and gets text back, and you pay for each call.

Your work is everything around that call. You decide what to send, how to check what comes back, and how the product uses it.

Two other kinds of work also deal with models:

• AI research asks how models can learn or work better. It produces new methods and findings, through experiments.
• ML engineering trains and runs a model for one task, using the company’s own data. For Deskora, that could mean training a model on two years of tickets to send each new ticket to the right team.
• AI engineering builds product features on a model that already exists. The ticket summary is AI engineering.

Chip Huyen’s book on the field draws a similar line: AI engineering adapts and evaluates existing models, while ML engineering is more about developing them.^{4AI Engineering: Building Applications with Foundation ModelsChip Huyen, O’Reilly Media, 2025} These are different jobs that need different skills, not levels on a ladder. Many engineers move between them.

The first decision in any AI feature is which of these jobs it needs.

From Tomás’s code, a model API looks like any other web API. That is where most surprises start.

A model call breaks three habits of web code

Many engineers believe a model call is one more HTTP request. The code does look the same. The behaviour doesn’t:

The same input can give a different output

Ordinary code is deterministic: the same input always gives the same output. A model’s output is non-deterministic. Anthropic’s documentation says identical inputs might produce different outputs across API calls.^{1GlossaryAnthropic, Claude API documentation}

The reason is in how a model writes. It picks each next piece of text from several likely options, so two runs can take different paths. Module 02 shows how that works.

You can’t switch this off, so you design around it. If Deskora made a new summary every time someone opened the ticket, Priya and her teammate could read two different summaries of the same ticket.

Every call costs money, and thinking is billed

Providers charge by the token. A token is a small piece of text, often part of a word. Tokens shows how text is split into them. For now, an estimate is enough.

Many models also think before they answer. They write hidden reasoning first, then the answer you see. Those thinking tokens are billed as output tokens, even when you never see them.^{3Steering thinkingAnthropic, Claude API documentation} Claude Haiku 5.5 thinks by default.^{3Steering thinkingAnthropic, Claude API documentation}

Here is the formula for one call:

As of October 2026, Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, for prompts up to 100,000 tokens.^{2PricingAnthropic, Claude API documentation} One summary is one call. This estimate leaves out retries and new summaries after new messages arrive.

Estimates for this lesson: 150 tokens a message, 40,000 summarized tickets a day out of 120,000 (one in four of them very long), and the thinking token counts.

Every price on Claude Opus 5.5 is 40 times the Haiku price, as of October 2026.^{2PricingAnthropic, Claude API documentation} With the same token counts, the same summaries would cost about $1,440 a day. So cost is a design input from the first prototype. Estimating the cost and latency of a feature builds full estimates, and Module 10 covers the ways to cut them: prompt caching, batch processing and sending easy work to smaller models.

Every call takes seconds

Here is where the time goes when Deskora makes one summary:

Example timings, not measurements.

The model call is about 99.7% of the latency. Its time grows with the size of the model and the length of the prompt and the response.^{1GlossaryAnthropic, Claude API documentation} Thinking tokens are written too, so they add time as well as cost.

So the summary is made before Priya opens the ticket

All three habits point to the same choice. If the summary were made when Priya opens a ticket, she would wait about 4 seconds every time. Each open would cost another call. Two agents could read two different summaries.

So Tomás makes the summary in the background when a new message arrives, saves it, and shows the saved copy. The design lesson at the end of this module works through it in full.

Correct means “good enough, often enough”

Tomás writes a unit test. It summarizes one saved ticket and compares the result with an expected paragraph. It fails on the next run, even though the new summary is fine. The words changed; the facts didn’t.

So the team tries something else. Priya and two other agents pick 40 real long tickets. For each one, they write down what the next agent must know: what was promised, whether a refund or replacement was agreed, and the customer’s latest question. Then they check each summary for those facts.

The prototype passes 34 of 40, or 85%. Most of the six failures drop a promise, as with Halima’s Friday replacement.

The 40 tickets and the 34 passes are example numbers.

That process is an eval: run the feature on many realistic inputs and score each output. Now the team has a number to improve, not a feeling from one demo. When Tomás changes his instructions or the model, he runs the eval again. Unit tests still check the code around the call. Evals shows how to build one.

Try it yourself

You can see all three habits with one command. You need a Claude API key in ANTHROPIC_API_KEY; if you don’t have one, skip this. Run it twice:

curl https://api.anthropic.com/v1/messages \
  -s -w "\nseconds: %{time_total}\n" \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2026-10-21" \
  -d '{"model": "claude-haiku-5-5", "max_tokens": 1024, "messages": [{"role": "user", "content": "Summarize this support ticket in one sentence for the next agent. Customer: My blender arrived cracked. Agent: Sorry! We will send a replacement by Friday. Customer: Thanks. Can I have the tracking number?"}]}'

An example of what one run might return, shortened:

{"type":"message","role":"assistant","model":"claude-haiku-5-5",
 "content":[...,{"type":"text","text":"The customer's blender arrived cracked; a replacement was promised by Friday and they now want the tracking number."}],
 "stop_reason":"end_turn",
 "usage":{"input_tokens":71,"output_tokens":164,"output_tokens_details":{"thinking_tokens":131}}}
seconds: 2.3

Compare the two runs. The text will probably be worded differently. In usage, thinking_tokens shows how much of output_tokens was thinking, and you pay for all of output_tokens. The seconds line is far longer than a typical web request.

One run is under 100 input tokens and at most 1,024 output tokens, thinking included. On Claude Haiku 5.5, as of October 2026, that is at most about $0.0005.

An AI feature is five steps around one call

Tomás’s prototype was one call. The feature Priya uses has five steps, and the model call is only one of them.

The prompt is everything you send the model in one call. Here it is Tomás’s instructions plus the ticket text, which is the context the model needs.

Step 3 has one trap worth knowing now. Each call sets a ceiling on output tokens, and thinking counts toward it.^{3Steering thinkingAnthropic, Claude API documentation} If the ceiling is too low, thinking can use it up before any summary is written. Step 4 catches that answer instead of showing it.

Here is the whole feature as a short sketch. Calling a model API explains each field of the call.

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment


def summarize_ticket(messages: list[dict[str, str]]) -> str | None:
    # 1. Product input: the ticket as plain text
    ticket = "\n".join(f"{m['author']}: {m['text']}" for m in messages)

    # 2. Prompt and context: instructions plus the ticket
    prompt = (
        "Summarize this support ticket for the agent who replies next. "
        "Use at most 5 short bullet points. Include every promise, refund "
        "or replacement already agreed, and the customer's latest question.\n\n"
        f"<ticket>\n{ticket}\n</ticket>"
    )

    # 3. Model call: max_tokens covers thinking and the summary together
    response = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )

    # 4. Check the output: no summary is better than a cut-off or bloated one
    if response.stop_reason != "end_turn":
        return None
    text = "".join(b.text for b in response.content if b.type == "text").strip()
    if not text or len(text.split()) > 120:
        return None
    return text


# 5. Show it in the product: a background job saves the result, and the
# ticket page shows it, or shows the ticket alone when this returns None.

The model call is about five lines. The rest is ordinary engineering, and it decides whether agents can trust the feature.

Where this shows up

• Design: a ticket summary feature builds all five steps for Deskora, with real numbers.
• Design: an AI customer support platform combines many features of this shape, each with its own checks and costs.

The diagram shows the model as one box at the provider. The AI application stack opens that box, from the model itself to the servers that run it and the code you write around it.

Common mistakes

• Trusting one good demo. One good summary says little about the next thousand. Score the feature on many real inputs before launch.
• Comparing exact text in tests. Good answers change wording between runs. Check facts, length and format instead.
• Leaving thinking out of the estimate. Thinking is billed as output, so a typical-case estimate without it can be a quarter too low.
• Setting a tiny output ceiling because the answer is short. Thinking shares that ceiling and can use it all. Leave room, and check for cut-off answers.
• Showing raw output with no fallback. When the check fails or the call errors, show the ticket without a summary, not a broken page.

Check your understanding

Summary