lumo

Module 01 · Lesson 7 of 14Free

Core system properties: availability, durability, consistency

Level
Intermediate
Topic
System design fundamentals
Reading time
18 min

When you finish this page you can

  1. Distinguish an unavailable operation, a lost acknowledged write and an outdated read.
  2. Trace when an order becomes durable and identify the failures that guarantee covers.
  3. Specify what readers may observe before, during and after a stock update.
  4. Choose different guarantees for product browsing and order confirmation.

Builds on 2 lessons

Start with lesson 515 min
  1. Functional and non-functional requirements15 min
  2. Capacity estimation: users, storage and traffic14 min

Video lesson · coming soon

At , 1,000 requests arrive each second as the festival sale opens. Asha receives an order confirmation just before the database crashes. After restart, her order is missing, but the product page still loads.

Was the store working?

That depends on which promise you mean. This lesson asks one question: what must a successful operation promise about responding, surviving failure and being seen by other requests?

Check yourselfAnswer in your head first, then reveal
Every product-page request succeeds during the incident, but Asha’s confirmed order disappears. Does that prove the store had an availability problem?Show answerHide answer
Not by itself. Those page requests were available, but the missing confirmed order breaks a durability promise. You must measure order placement separately to know whether it was also unavailable.

Diagram not drawn yet: kartly-three-property-failures.svg

Three independent examples: an order request times out; a confirmed order disappears after a crash; and a product page shows old stock after a completed update. Each example breaks a different promise.

Availability asks whether the operation works now. Durability asks whether an acknowledged write survives. Consistency asks what operations may observe.

The examples below keep Kartly’s one app server and one PostgreSQL database. Any extra timings, incident counts and proposed targets are assumptions for this lesson, not measured production results.

Three promises describe different failures

asks whether an operation can provide its promised service now. For Kartly, that might mean returning a usable product page within a deadline.

asks whether an acknowledged write remains recorded after specified failures. Asha’s confirmation should not disappear because the database restarts.

describes the results that reads and writes are allowed to produce together. If a seller finishes changing stock, can the next stock check still return the previous value?

Turn the requirements from Functional and non-functional requirements into separate promises:

Operation Proposed promise
Browse a product Return a usable page within one second
Confirm an order Keep the order through a database restart
Reserve the last item Accept at most one reservation
Show stock on a page Allow an outdated display, but recheck before accepting an order

These are not one quality score. In our scenario, the database could stop answering without losing any orders. Alternatively, it could answer immediately but acknowledge orders before protecting them against a crash.

Notice the boundaries in those promises. “Survive a restart” says nothing about a destroyed disk. “Show stock” is not the same operation as “reserve stock”.

Before choosing infrastructure, you need to decide exactly which operation and failure each promise covers.

Availability counts usable outcomes

A process being alive is not enough for our proposed availability measure. Kartly’s app could answer a even when its database cannot serve orders.

Request-based availability measures the proportion of requests that succeed. Unlike whole-service uptime, it can represent an incident affecting only some traffic. Google’s SRE book also warns that failures of different operations have different consequences.1

For this lesson, define an eligible product-page request as a valid request for an existing product. Count it as good when it returns a usable page within one second.

That deadline turns the latency measurement into an operational promise. A page delivered after the customer gives up is not good under this definition.

Suppose Kartly records the following during a sale:

Count the eligible requests

100,000 eligible product-page requests

Include requests that fail or time out. Counting only completed requests would hide part of the incident.

Count the good outcomes

100,000 requests − 200 bad outcomes = 99,800 good outcomes

Here, “bad” means an error, an unusable page or a missed deadline.

Calculate the observed availability

99,800 ÷ 100,000 × 100% = 99.8%

Against a proposed target of 99.9%, this window misses the target.

These figures describe a measurement window, not a forecast of next month’s behavior.

Measure order placement separately. Otherwise, successful browsing could hide every failed checkout. A valid “sold out” response can be a good outcome: the service correctly completed its decision, even though the customer could not buy.

For the example target, the team must also specify where timing starts. Measuring inside the app excludes failures before a request reaches it. Measuring from the customer includes a different set of dependencies.

Availability tells you which customers could complete an operation. It does not tell you whether an accepted order will still exist tomorrow.

Confirmation needs a durable boundary

Imagine this proposed order path:

  1. The app sends the order to PostgreSQL.
  2. PostgreSQL accepts the change in memory.
  3. The app sends “Order confirmed”.
  4. The database makes the change persistent later.

The dangerous interval lies between confirmation and persistence. A crash there could leave Asha holding a confirmation for an order the database cannot recover.

PostgreSQL 18 normally waits for the ’s records to reach permanent storage before reporting commit success. The write-ahead log is a recovery record of database changes. With asynchronous commit, success can precede that flush, exposing recent transactions to loss after a crash.2

When should Kartly tell the customer that the order is confirmed?

Confirm before the database’s durable commit.

The response can avoid waiting for the persistent commit, but a crash can erase an order the customer believes exists.

Why this choice

For this scenario, a missing confirmed order creates support work and breaks the customer’s purchase record. Avoiding the commit wait is not worth that outcome.

Choose the second option for orders. If the data were disposable page-view events, accepting some recent loss might be reasonable. If whole-disk failure must also be covered, local persistence alone would no longer meet the requirement.

Diagram not drawn yet: kartly-order-durable-boundary.svg

The app sends an order transaction to PostgreSQL. PostgreSQL flushes recovery records to persistent storage before replying that commit succeeded. Only then does the app send the customer a confirmation. An early confirmation before the flush is marked unsafe for this promise.

The confirmation comes after the persistence boundary, not merely after the database receives the request.

For the chosen design, trace the database component’s behavior:

  1. Receive the order transaction.
  2. Produce recovery records for its changes.
  3. Wait for the required log flush before reporting commit success.
  4. After a crash, recover by replaying the flushed log.

Recovery does not require every changed table page to have been written before acknowledgement; the log provides the recovery path.2

The app’s responsibility is smaller but essential: do not send confirmation before that commit succeeds.

Suppose the database commits, then the app crashes before returning a response. The order can be durable even though the customer saw a . That outcome is uncertain to the customer, not proof that the order failed.

Now reverse the failure. Suppose the order survives on disk, but the database remains offline. Durability can hold while order lookup is unavailable.

The next question is not merely whether data survived. It is what another request is allowed to see.

Stock needs an ordering rule

In this scenario, a product has one remaining item. Asha and Ben both open its page before either buys it.

Both screens showing “1 available” is not necessarily a consistency failure. Their reads may have happened before either reservation changed the stock.

The important question is what happens when they both try to reserve it.

Start with a tempting application algorithm:

  1. Read the remaining stock.
  2. If it is positive, create a reservation.
  3. Write the reduced stock.

Walk through two overlapping requests. Both read one. Both decide they may reserve. Both write zero.

The displayed stock ends at zero, but Kartly has accepted two reservations. A final value that looks reasonable does not prove the operation was correct.

For this lesson, require one indivisible reservation operation: check availability and reduce stock together. “Indivisible” means no competing reservation can enter between that check and change.

Define its contract before selecting an implementation:

Starting stock Operation Required result
One item First accepted reservation Stock becomes zero
Zero items Another reservation Refuse without changing stock
Unknown Cannot reach authoritative stock Do not report a confirmed reservation

This is a design requirement, not a claim that ordinary reads followed by writes provide it automatically.

One useful is . Operations on one object appear to take effect in a single order that respects completed operations. A read starting after a completed write must reflect that write or a later one. Overlapping operations can take either valid order.3

Applied to our stock contract, one reservation wins. The other observes no remaining stock. It does not matter which customer’s request arrived a fraction earlier if the operations overlapped.

This model does not automatically protect an entire checkout involving stock, an order and payment. Its scope must be specified; a guarantee for one stock object is not a guarantee across every database object.3

We now have a stricter reservation operation. Does every product-page read need the same freshness?

Browsing can tolerate older information

Assume a future optimization keeps a product-page snapshot for a short interval. A snapshot is a saved version used for display rather than the authority for reservations.

Immediately after Asha reserves the last item, Ben might still see “1 available” on that saved page. If checkout rechecks authoritative stock, Kartly refuses Ben’s reservation.

Ben experiences disappointment, but Kartly does not promise him an item it cannot supply. That distinction must appear in both the implementation and the page wording.

Must browsing and reservation use the same freshness rule?

Require authoritative stock reads for every page.

Each display waits for an authoritative answer. The page avoids this snapshot delay, but browsing now depends on that answer being available.

Why this choice

Choose the second option only if the product accepts outdated browsing information. It separates a helpful display from a purchase commitment.

Choose the first when the display itself must be current. Neither option removes the need for a reservation check: another customer may buy between displaying the page and tapping Buy.

The simplest working version remains direct database reads. A snapshot is not required merely because it is possible. Add one only after a measured load or problem justifies its extra freshness rules.

If Kartly later maintains several copies, it must define how updates reach them. permits temporary disagreement but requires convergence once updates stop and propagation succeeds.

That is not a deadline. “Eventually” alone does not promise that Ben sees the new stock within a particular number of seconds. For our proposed display design, the team should separately choose a freshness limit and a fallback when it cannot meet that limit.

Most importantly, a saved display value must not become permission to accept a reservation. A relaxed read path is safe only within the product promise that allows it.

Failures reveal the guarantee’s scope

Consider this proposed recovery policy for our single-server stage:

Failure What the customer sees Detection and recovery
App stops after durable commit Timeout despite a saved order Compare request records with stored orders; restore service and let the customer retrieve the order
Database stops, storage survives Order lookup and placement fail Detect failed database calls; restart and complete recovery before confirming new orders
Display snapshot stops updating Older stock information Check snapshot age; refresh it or hide the stock claim
Database disk is destroyed Existing orders cannot be read locally Restore from a separately protected backup; determine which acknowledged orders it contains

These are proposed procedures, not guarantees supplied by the architecture alone. The app still needs a safe way to associate a timed-out attempt with its stored order before offering an automatic retry.

A local durable commit covers the crash boundary discussed earlier, assuming storage honors its persistence requests. It does not create an independent copy of the disk.

For this scenario, suppose the only surviving backup predates Asha’s purchase. Restoring it brings the store back online without restoring her order. Recovery of availability has not restored every acknowledged write.

That is why “we have backups” is incomplete. Ask which failure the backup survives, which orders it contains and whether restoration has been demonstrated. The later backup and restore lesson develops that procedure.

Do not assume more traffic automatically demands a different consistency promise. Use capacity estimation to assess load separately from the correctness requirement.

For the next design review, write a compact contract:

  • Which operation does the promise cover?
  • What counts as a good response, and by when?
  • What must be persistent before success?
  • Which failures must an acknowledged write survive?
  • What may an overlapping or later operation observe?
  • What does the customer see when the promise cannot be met?

Where this shows up

Those contracts create competing costs, which Design tradeoffs: making and explaining decisions examines next.

Common mistakes

  • Calling an open port “available”. Our availability contract requires a usable operation, not merely a running process.
  • Calling a timeout “not saved”. In the traced failure, commit succeeded before the response disappeared.
  • Calling one database “consistent”. The reservation algorithm still needs a rule for overlapping requests.
  • Calling a backup “zero loss”. In our restore example, the backup never contained the missing order.

Check your understanding

Check yourselfAnswer in your head first, then reveal
The database is offline, but all acknowledged orders remain recoverable on its disk. Which property is currently failing for order lookup?Show answerHide answer
Availability: customers cannot retrieve orders now. Not necessarily durability, because the recorded orders remain recoverable under the stated failure.
Check yourselfAnswer in your head first, then reveal
A stock update completes before a new read begins. The read returns the old value. Is that compatible with a linearizable stock object?Show answerHide answer
No, unless a later update made that old value current again. Without such an update, returning the previous value violates the ordering of completed operations.
Check yourselfAnswer in your head first, then reveal
Ben’s page says one item remains, but an authoritative reservation refuses his purchase after Asha buys it. Has Kartly necessarily violated its contract?Show answerHide answer
No. The proposed contract allows an older display but requires a correct reservation decision. It would violate a separate promise that every displayed stock value was current.
Check yourselfAnswer in your head first, then reveal
The app confirms an order before the database finishes its durable commit. No orders have been lost in testing. Does that establish durability?Show answerHide answer
No. The interval between confirmation and persistence still permits the failure described in the lesson. A test without that failure does not remove the interval.
Check yourselfAnswer in your head first, then reveal
A restored backup makes checkout work again, but it lacks the final hour of acknowledged orders. Which promises should the incident report assess separately?Show answerHide answer
Assess availability recovery and durability loss separately. Working checkout shows service has returned; it does not show that every acknowledged order survived.

Summary

  1. A successful operation needs separate promises about responding, surviving failure and being observed.
  2. Availability measures usable outcomes for a defined operation, deadline and measurement boundary.
  3. Durability begins at an acknowledgement boundary and covers specified failures, not every possible loss.
  4. Consistency constrains concurrent and later results; stock reservation needs more than a separate read and write.
  5. Browsing may allow older data while confirmation requires authoritative decisions and durable storage.

Report a mistake

Tell us what’s wrong in this lesson and we’ll fix it.