Module 01 · Lesson 10 of 14Free

Stateless and stateful services: impact on scaling

Level

Intermediate

Topic

System design fundamentals

Reading time

18 min

During a festival rehearsal, Maya adds 2 products to her cart on Kartly. The team restarts the app, and her cart becomes empty. Product pages still work because their data lives in PostgreSQL.

Kartly still has its Module 01 setup: one app server with 8 CPU cores and one database. For this lesson’s experiment, the cart exists only in app memory.

The team is considering another app instance, as discussed in Vertical and horizontal scaling. But a second instance would have different memory.

How can Kartly replace or add app instances without losing a customer’s progress?

The dependency is on yesterday’s request

After Maya adds a product, something must remember that choice. That remembered information is state: data from past activity that affects later behavior.

In our starting implementation, the app keeps a map:

customer 42 → cart containing product 101 and product 205

When another request arrives, the app looks up customer 42. This is simple and avoids a storage call. It also makes the customer depend on the lifetime of that particular app instance.

We call this app tier stateful because correct handling of later requests requires state held by a particular instance.

For this lesson, a stateless app tier has a narrower meaning: any ready instance can handle the next request without needing another instance’s private history. Shared storage may still hold plenty of history.

That distinction matters. “Stateless” does not mean “no database”, “no variables” or “no writes”.

HTTP itself is described as stateless in RFC 9110. Yet applications using HTTP can maintain state across requests. The protocol label does not tell you where Kartly stores its cart.^{1HTTP SemanticsRoy T. Fielding, Mark Nottingham and Julian Reschke, IETF, RFC 9110, 2022}

Classify what must survive

For our cart design, ask what must remain available after the app disappears.

A session is a record that connects later requests to an earlier login. A browser can send an identifier, and an app can retrieve the corresponding record. Django’s server-side session support uses this separation between the cookie’s identifier and stored session data.^{2How to use sessionsDjango Software Foundation, Django 5.2 documentation, 2025}

The useful test is not “does this app use memory?” It is “would losing this memory change a later request’s correct answer?”

For Maya’s cart, the answer is yes. We must change either where the cart lives or how requests find it.

Routing cannot preserve missing data

Suppose the team temporarily runs instances A and B behind a request router. This is an illustrative expansion, not Kartly’s permanent Module 01 setup.

There are two broad fixes. Keep sending Maya to A, or make her cart accessible from either instance.

Copying carts between app instances is another possible approach. Under that proposed design, an update must reach another copy before it can survive the first instance’s loss. Waiting adds coordination; not waiting leaves a loss window.

For this lesson’s requirements, that is more machinery than using the existing database. Shared storage removes the routing dependency, but now every instance needs the same rules for accessing it.

Shared records make instances replaceable

First, Kartly moves carts out of app memory while it still has one app instance. Only then does the team repeat the two-instance experiment.

This order separates two changes. If cart behavior breaks, the team can investigate the storage move before adding routing differences.

Here is the responsibility split for the experiment.

The intended guarantee is limited: replacing an app does not remove records already saved in the database. We are not claiming that this single database survives every possible failure.

One request carries a reference

For the experiment, the browser sends this request over HTTPS:

GET /v1/cart HTTP/1.1
Host: kartly.example
Cookie: session_id=<opaque-session-id>

The identifier is a reference, not the cart and not a customer ID supplied on trust. Any instance uses it to find the session’s customer.

Our illustrative storage layout is:

The app owns the interpretation of these records. PostgreSQL owns their saved contents. Authentication details belong in Sessions and cookies; here we assume a correctly implemented session lookup.

The read path

1. The router selects B.
2. B retrieves the session record and checks that it is valid.
3. B uses the record’s customer ID to retrieve the cart.
4. B returns the saved products and quantities.

In our API contract, an invalid session receives 401 Unauthorized. A successful read receives 200 OK. A database timeout receives 503 Service Unavailable, not an invented empty cart.

These are design choices for this example. They distinguish “you need to sign in” from “we cannot read your saved data”.

The write path

1. The router selects A for an add-item request.
2. A resolves the session and validates the requested product and quantity.
3. A asks the database to change the customer’s cart.
4. A sends success only after the database reports a successful commit.

A later read through B follows the same lookup path. It does not ask A what happened.

This is why moving only login sessions is insufficient. A shared session can identify Maya on B, but B still cannot display a cart left in A’s memory.

Our design has removed dependence on a particular app. It has not removed dependence on storage, nor solved concurrent changes to the same cart.

Shared state still needs update rules

Imagine Maya uses two tabs. Each reads a quantity of 1, adds one locally, and writes back 2.

If our implementation allows both read–modify–write sequences without coordination, the final quantity can be 2 rather than 3. Both requests found the same cart. The problem is how they changed it.

That failure follows from the proposed update algorithm, not from whether the app is stateless.

Kartly therefore needs the database to perform a coordinated quantity change, rather than accepting a quantity calculated from an earlier read. Transactions and ACID guarantees develops the storage mechanisms.

Retries are a separate problem. Suppose an add-item operation commits, but A disappears before returning success. The browser does not know whether the change happened.

Sending the retry to B is possible now. Whether repeating it adds another item depends on the API’s retry rules. Stateless placement alone gives no duplicate-prevention guarantee.

For our design, distinguish three promises:

• Placement: any ready app can access the cart.
• Concurrent updates: overlapping changes follow a defined rule.
• Retries: repeating an uncertain operation has a defined effect.

Only the first promise follows from moving state out of the app. The other two require explicit mechanisms.

That leaves another placement choice: could the browser carry the state instead of looking it up?

Client-held state changes the costs

A request can carry more than an identifier. For example, a signed cookie can carry session information.

A signature lets the server detect changed contents. It does not hide those contents or prove that the client returned the newest valid value. Django’s signed-cookie documentation makes both limitations explicit.^{2How to use sessionsDjango Software Foundation, Django 5.2 documentation, 2025}

Client-held state can remove a lookup for particular data. It does not make every kind of shared business state unnecessary.

Replacement still interrupts active work

Now the team restarts A after Maya’s cart has been saved. B can read that cart because the app restart does not change our database records.

But a request executing inside A can still be interrupted. Replaceable instances are not interruption-free instances.

For planned deployments, our operating procedure is to stop sending new requests to A, let active requests finish, then stop it. This is called draining. AWS documents a deregistration delay for in-flight requests and warns about errors when targets terminate connections early.^{3Edit target group attributes for your Application Load BalancerAmazon Web Services, Elastic Load Balancing documentation, 2026}

The same distinction applies to local files. If our app returns “upload saved” before moving a file off its temporary disk, replacement can lose the only copy. A cart can be shared correctly while another feature still depends on local storage.

Make failures distinguishable

For Kartly’s proposed design, the operating checks should match what customers could lose.

A crucial rule: do not interpret “storage could not answer” as “there is no saved cart”. Otherwise an outage can become a misleading empty-cart response.

Adding app instances also increases possible demand on the same database. Under our design, each cart request still needs shared storage. More app capacity does not automatically add storage capacity.

We have moved responsibility, not removed it. The next question is how much state we may safely treat as disposable.

Going deeper

Common mistakes

• Calling the entire system stateless. Name the boundary: our app tier is replaceable; its storage tier keeps the records.
• Treating sticky routing as a backup. Routing to A cannot recover memory that A has lost.
• Returning an empty cart on timeout. An unavailable record and an absent record are different outcomes.
• Moving only the obvious state. Check sessions, carts, temporary files and unfinished work before replacing an instance.
• Expecting storage placement to solve races. Shared access still needs update and retry rules.

Where this shows up

• Design: a URL shortener for millions of users: redirect instances need access to mappings created elsewhere.
• Design: a fast product catalog page: local cached data must not become the only authoritative copy.

Kartly can now replace an app instance, but the next lesson asks whether its features should remain together in one application or several services.

Check your understanding

Summary