Module 01 · Lesson 13 of 13Free

Lab: a command-line chat with streaming

Level

Beginner

Topic

AI engineering fundamentals

Reading time

16 min

Tomás’s test script forgets and makes him wait

Deskora is a help desk app. A shop’s support team uses it to answer customer emails from one shared inbox. Each customer problem becomes a ticket: one customer’s whole conversation with the support team, message by message.

Tomás, an engineer at Deskora, is about to design a summary for long tickets. First he wants to try questions on a real 4,000-token ticket. His test script sends the ticket and a question. The terminal shows nothing while the model works, then the whole answer at once. He asks a follow-up, “Did anyone promise a date?”, and the model has no idea which ticket he means.

In this lab you build the tool he needs: a terminal chat that remembers the conversation, prints each reply as it arrives and shows what each turn cost.

This lab answers one question: how do you build a chat on an API that remembers nothing, and why does each turn cost more than the one before?

Deskora is made up, and the ticket sizes and token amounts below are estimates for this lab.

Every turn of the chat runs the same six steps.

Set up Python, the SDK and your key

You need Python 3.10 or newer. As of October 2026, the latest anthropic package is version 2.3.0, and it requires Python 3.10 or later ^{3anthropicAnthropic, Python Package Index, 2026}. Pinning it keeps this code working if a later SDK changes.

python3 --version              # 3.10 or newer
mkdir deskora-chat && cd deskora-chat
python3 -m venv .venv          # Windows: py -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate
pip install "anthropic==2.3.0"
export ANTHROPIC_API_KEY="paste-your-key-here"
# Windows PowerShell: $env:ANTHROPIC_API_KEY = "paste-your-key-here"

The virtual environment keeps this lab’s packages apart from the rest of your machine. The API key lives in an environment variable, a named value your shell passes to every program it starts. The SDK reads ANTHROPIC_API_KEY from there, so the key never appears in your code ^{5anthropic-sdk-pythonAnthropic, GitHub}. Anyone holding the key can spend your money, so never paste it into a file.

For the second run you also need a ticket. Save any long email thread as ticket.txt, with names and addresses removed, or write a made-up one. About 3,000 words of English is roughly 4,000 tokens; Tokens explains why the ratio varies.

What the whole lab costs

Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, for prompts up to 100,000 tokens, as of October 2026 ^{2PricingAnthropic, Claude API documentation}. Thinking tokens are billed as output ^{4Steering thinkingAnthropic, Claude API documentation}. The formula is the one from Calling a model API.

The chat prints a running total, so you always see what you have really spent.

Write the chat in three parts

Put the three parts below in one file, chat.py, in this order.

The first part sets the model, the prices and the system prompt. If you pass a ticket file, its text joins the system prompt, so every turn can see it.

import sys

import anthropic

MODEL = "claude-haiku-5-5"
MAX_TOKENS = 2000  # room for thinking plus a reply of a few paragraphs
# claude-haiku-5-5, US dollars per million tokens, prompts up to
# 100,000 tokens, as of October 2026
INPUT_PRICE = 0.10
OUTPUT_PRICE = 0.50  # thinking tokens are billed at this price too

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment


def load_system_prompt() -> str:
    system = ("You help a Deskora engineer test questions about "
              "support tickets. Answer in a few short lines.")
    if len(sys.argv) > 1:
        with open(sys.argv[1], encoding="utf-8") as f:
            system += "\n\nThe ticket:\n" + f.read()
    return system


def turn_cost(usage: anthropic.types.Usage) -> float:
    return (usage.input_tokens * INPUT_PRICE
            + usage.output_tokens * OUTPUT_PRICE) / 1_000_000

The second part sends one turn. client.messages.stream opens a streaming request. text_stream yields only the pieces of reply text, so the hidden thinking never prints. get_final_message() waits for the stream to end and returns the full message, with stop_reason and usage ^{1Streaming messagesAnthropic, Claude API documentation}. How the events travel is covered in Streaming responses.

def ask(history: list[dict], system: str) -> anthropic.types.Message:
    with client.messages.stream(
        model=MODEL,
        max_tokens=MAX_TOKENS,
        system=system,
        messages=history,
    ) as stream:
        for text in stream.text_stream:
            print(text, end="", flush=True)
        message = stream.get_final_message()
    print()
    return message

The third part is the loop. history is a plain Python list that grows by two entries per finished turn: your question and the model’s reply.

def main() -> None:
    system = load_system_prompt()
    history: list[dict] = []
    turn, total = 0, 0.0
    print(f"Chatting with {MODEL}. Ctrl+C or Ctrl+D to quit.")
    while True:
        try:
            question = input("\nyou> ").strip()
        except (KeyboardInterrupt, EOFError):
            break
        if not question:
            continue
        history.append({"role": "user", "content": question})
        try:
            print("claude> ", end="", flush=True)
            message = ask(history, system)
        except KeyboardInterrupt:
            history.pop()  # forget the unanswered question
            print("\n[stopped: this turn was not kept]")
            continue
        turn += 1
        usage = message.usage
        details = getattr(usage, "output_tokens_details", None)
        thinking = details.thinking_tokens if details else 0
        cost = turn_cost(usage)
        total += cost
        print(f"[turn {turn}: in={usage.input_tokens} out={usage.output_tokens} "
              f"(thinking={thinking}) cost=${cost:.6f} total=${total:.6f}]")
        if message.stop_reason != "end_turn":
            history.pop()  # don't build later turns on an unfinished reply
            hint = " Ask for a shorter answer." if message.stop_reason == "max_tokens" else ""
            print(f"[stop_reason={message.stop_reason}: turn not kept.{hint}]")
            continue
        history.append({"role": "assistant", "content": message.content})
    print(f"\nBye. {turn} turns, ${total:.6f} in total.")


if __name__ == "__main__":
    main()

Three details matter. The reply goes back into the history as message.content, the full list of content blocks, thinking block included. The usage line prints after the reply, because usage is final only when the stream ends; the message_delta event at the end carries cumulative counts ^{1Streaming messagesAnthropic, Claude API documentation}. If the network fails before any text arrives, the SDK retries twice by default ^{5anthropic-sdk-pythonAnthropic, GitHub}; Model API errors, rate limits and retries covers the rest.

The one real design choice: what to keep when a reply doesn’t finish.

Run it and read the usage line

Start with a plain chat, no ticket:

python chat.py

Ask three or four short questions that build on each other, like the ones below. Then quit with Ctrl+C at the you> prompt.

Here is an illustrative session, made up to show the shape; your text and numbers will differ on every run:

Chatting with claude-haiku-5-5. Ctrl+C or Ctrl+D to quit.

you> Name a common reason parcels arrive late.
claude> A common reason is a missed handover between carriers...
[turn 1: in=52 out=184 (thinking=96) cost=$0.000097 total=$0.000097]

you> How would a support agent explain that to a customer?
claude> They could say the parcel is still moving, but...
[turn 2: in=250 out=212 (thinking=88) cost=$0.000131 total=$0.000228]

What to look for:

• A pause after claude>. The model is thinking before it writes. You pay for that thinking and never see its text.
• thinking inside out. Output minus thinking is the reply you read. On an easy question, thinking can be 0, because the model may skip it.
• Turn 2’s in is close to turn 1’s in plus turn 1’s out plus your new question. Here 52 + 184 + about 14 = 250. Your counts may be off by a few tokens of message formatting.
• The second answer uses the first. The model knows what “that” means only because the list sent turn 1 again.

If the first turn fails with anthropic.AuthenticationError, the key in your shell is missing or wrong. Run the export line again in the same terminal.

Make it stop early on purpose

Set MAX_TOKENS = 100 and ask for “ten ideas for faster support replies, with a sentence on each”. Look for a reply that stops mid-list, or no text at all, then [stop_reason=max_tokens: turn not kept. Ask for a shorter answer.]. Your next turn’s in does not include the cut-off reply. Set MAX_TOKENS back to 2000.

Then ask a long question and press Ctrl+C while text is printing. The chat stays open and the turn is dropped. Tokens produced before you pressed it may still be billed, and this script can’t show them, because usage arrives only at the end of the stream.

What happened on each turn

The Messages API is a stateless service: each request stands alone, and nothing about your chat is stored on the provider’s side. The conversation exists only in history, inside your running program. Quit, and it is gone.

So every request carries the whole list. On turn 3 it holds five entries: question 1, reply 1, question 2, reply 2 and question 3. Each reply includes its thinking block. Claude Haiku 5.5 keeps earlier turns’ thinking in the conversation and bills it as input ^{4Steering thinkingAnthropic, Claude API documentation}. That is why turn 2’s input includes all of turn 1’s output, not only the text you read.

Streaming changed none of this. The SDK sent one request and the reply came back as server-sent events on the same connection ^{1Streaming messagesAnthropic, Claude API documentation}. You saw the first words sooner; the token counts are the same.

The history only grows, so the next question is how fast.

Predict the cost of turn 10

Change one thing: start the chat with your ticket.

python chat.py ticket.txt

Ask ten short follow-ups about the ticket and watch in and cost on each usage line. Here is the arithmetic to compare against.

The same per-turn growth looks very different next to different starting sizes. All rows use the same per-turn assumptions:

Input grows by a fixed amount each turn, so the cost of one turn grows in a straight line. The total for a conversation grows faster: ten turns resend turn 1 nine times. Fractions of a cent don’t matter in a test tool. In a feature thousands of agents use all day, they would.

Three levers change the curve, each taught later. Prompt caching makes the repeated start of the history cheaper to resend; see KV cache and prompt caching. Trimming or summarizing old turns keeps the list short; see Agent memory. And a long enough chat stops fitting at all, covered in Context windows.

What to take away

• A chat is a loop around a stateless call. Your list is the memory, and you choose what goes in it.
• Keep only complete turns, and print stop_reason, so a cut-off reply is visible instead of quietly shaping later turns.
• Usage arrives at the end of the stream. Log it every turn, thinking included, because it is the only true cost record.
• Input per turn grows with the history. Anything at the start of the conversation, such as a ticket, is paid for on every turn.

Tomás now has a tool for trying questions, and the same loop becomes the summary call in Design: a ticket summary feature. One thing in the lab is still unexplained: why the reply arrives a few words at a time. Next-token prediction opens the model and shows how it writes.

Common mistakes

• Creating the history list inside the loop. Every turn then sends only the newest question, and follow-ups fail without an error.
• Appending the reply before checking stop_reason. A cut-off answer then shapes every later turn and is billed again as input each time.
• Leaving the question in the list after Ctrl+C. The next request repeats the old question, and the model may answer both.
• Reading usage inside the text loop. The counts are final only after the stream ends.

Check your understanding

Summary