# Caching, Explained

> A cache is a copy of an expensive-to-produce answer kept somewhere fast so you don't redo the work. This guide covers what a cache really is, where caches live, and why keeping them accurate is the genuinely hard part.


---

# Caching, Explained

"We should cache that" always sounds like the easy win - the magic word that makes slow things fast. Then a user swears they updated their profile and the old name keeps showing, and suddenly caching is the reason something is *wrong*, not fast. Both moments come from the same small idea, and once you hold it clearly, caching stops being a black box.

A cache is one thing: a copy of an expensive answer, kept somewhere fast, so you don't have to produce that answer again. Everything else - CDNs, Redis, browser caches, TTLs, the famous invalidation jokes - is a variation on that single move. This guide builds the idea up cleanly so you can reason about any cache you meet, instead of memorizing rules.

## How to read this

- **Want it to finally make sense?** Read in order. Phase 1 installs the mental model, Phase 2 shows you where caches actually live, and Phase 3 covers the part everyone gets bitten by.
- **Already comfortable and here for the hard part?** Jump to [Phase 3: The Hard Part - Invalidation & Staleness](03-invalidation-and-staleness.md) - that's where stale data, TTLs, and eviction live.

## The phases

1. **[What a Cache Actually Is](01-what-a-cache-actually-is.md)** - a copy of an expensive answer kept somewhere fast; hits, misses, and the notepad mental model.
2. **[Where Caches Live](02-where-caches-live.md)** - the browser, the CDN at the edge, your application cache (in-memory / Redis), and the database's own caches; a request traveling through all of them.
3. **[The Hard Part - Invalidation & Staleness](03-invalidation-and-staleness.md)** - why the cached copy and the truth drift apart, TTLs, eviction (LRU), write-through vs. cache-aside, and when *not* to cache.

> This guide stays at the "reason about it" level. Cache stampedes, distributed cache coherence, and tuning Redis for production are deliberately left for a follow-up - you want the mental model solid before any of that helps.

**Related:** [Why Is My Query Slow?](/guides/why-is-my-query-slow) · [Designing for Scale](/guides/designing-for-scale)


---

# What a Cache Actually Is

The word "cache" gets used like it's a special database, or a setting you flip on, or some kind of speed boost the framework gives you for free. That fuzziness is why it feels mysterious. The thing itself is almost embarrassingly simple, and seeing it plainly is what lets you reason about every cache you'll ever touch.

Here's the whole idea: **a cache keeps a copy of an answer that was expensive to produce, somewhere fast to reach, so the next time you need that answer you grab the copy instead of doing the expensive work again.** That's it. Not a kind of database. Not magic. A copy, kept handy, to skip repeated work.

## The mental model: a notepad of answers you already worked out

**What it actually is.** Picture yourself doing a long column of arithmetic by hand. Halfway through you need `347 × 89` again - the same product you carefully computed two lines ago. You don't redo the multiplication; you glance at where you wrote `30883` and copy it down. That scrap of paper is a cache. The multiplication was the expensive work; the written-down answer is the cached copy; glancing at it instead of recomputing is the entire benefit.

```text
   First time you need 347 × 89:
       do the work  ──►  get 30883  ──►  jot it on the notepad
                                              │
   Next time you need 347 × 89:               ▼
       glance at the notepad  ──►  30883   (no multiplication this time)
```

Every cache in computing is this notepad. The "expensive work" might be a database query that scans a million rows, a call to a slow third-party API, rendering an image, or fetching a file from a server on another continent. The cache is wherever you stash the result so you don't pay that cost twice.

**Why people get this wrong.** The common wrong picture is that a cache is a place you *put data on purpose*, like a special fast database you write to. The core idea is the opposite: a cache holds *derived* copies of answers whose real home - the database, the API, the original file - is somewhere else. Holding that distinction, **truth lives elsewhere, the cache holds a copy**, is what makes Phase 3 make sense later.

## Hit and miss: the two things that can happen

**What it actually is.** Every time you ask a cache for something, exactly one of two things happens.

- 📝 **Cache hit** - the copy is there. You take it and skip the expensive work. Fast.
- 📝 **Cache miss** - the copy isn't there. You do the real work, return the answer, and (usually) store a copy so next time is a hit.

```mermaid
flowchart TD
  Ask["Ask the cache for 'user 42's profile'"]
  Ask --> Q{Is the copy there?}
  Q -- "yes = HIT" --> Hit["Return the copy (fast)"]
  Q -- "no = MISS" --> Miss["Do the real work (query the DB),<br/>return it, AND save a copy for next time"]
```

**What it does in real life.** The first request for something is almost always a miss - nobody's computed it yet, so the cache is empty for that item (a *cold* cache). You pay full price once; every later request for the same thing can be a hit, paying almost nothing. Caching doesn't make any single answer cheaper to *produce* - it makes *repeated* requests for the same answer nearly free.

**A real example.** Here's the pattern in plain code - ask the cache first, fall back to the real work on a miss, then save the result:

```text
   answer = cache.get("user:42:profile")

   if answer is present:          # HIT
       return answer

   answer = database.query(...)   # MISS - do the expensive work
   cache.set("user:42:profile", answer)
   return answer
```
*What just happened:* The first call for `user:42:profile` finds nothing (miss), runs the real database query, and stores the result under that key. The second call finds the stored copy (hit) and returns it without touching the database at all. The key - `user:42:profile` - is just a label so you can find the right copy again, exactly like writing `347×89=` next to your jotted answer.

**The gotcha.** A cache only helps when the *same* answer is asked for more than once. If every request is for something unique - a one-time report, a per-request random value, a query nobody repeats - there's nothing to reuse, every request is a miss, and the cache adds work (check, fail, do the job anyway) without ever paying off. Caching wins on *repetition*, and when someone suggests caching data that's different on every request, that's your cue to push back.

## Why caching is everywhere

**What it actually is.** Once you see "keep a copy of the expensive answer somewhere fast," you start seeing it at every layer of every system - the gap between *fast* and *slow* is enormous and shows up everywhere.

Producing an answer can be slow for very different reasons:

- **Distance.** The data lives on a server far away; light and network hops take real time.
- **Computation.** The answer takes real work to build - a heavy query, a rendered image, an aggregation over lots of rows.
- **A slow dependency.** You're calling something out of your control - a third-party API, a rate-limited service.

In all three cases the fix is the same shape: do the expensive thing once, keep the result somewhere closer or cheaper, and serve the copy. That's why caching appears in your browser, at the network edge, inside your application, and inside the database itself - which is exactly the tour Phase 2 takes you on.

**Why this saves you later.** Performance work is often just "find the expensive thing that's being redone, and stop redoing it." Seeing caching as *one idea applied at many layers*, rather than a pile of unrelated technologies, makes the whole landscape readable. (When the expensive thing is a database query specifically, [Why Is My Query Slow?](/guides/why-is-my-query-slow) is the companion to this guide.)

## Recap

1. **A cache is a copy of an expensive-to-produce answer, kept somewhere fast, so you skip redoing the work.** It's a notepad, not a special database.
2. **The truth lives elsewhere; the cache holds a copy.** Hold this - it's the root of every staleness problem in Phase 3.
3. **A hit means the copy is there (fast); a miss means you do the real work and save a copy for next time.**
4. **Caching only pays off when the same answer is asked for repeatedly.** No repetition, no benefit.
5. **It's everywhere because the slow/fast gap is everywhere** - distance, computation, and slow dependencies all get the same treatment.

You know what a cache is and why it helps. The next question is *where* you'd actually put one - and it turns out a single web request passes through several caches stacked one behind the other.

See why a small cache still helps - repeated keys are instant hits, and the least-recently-used entry gets evicted when it fills:

```playground-lru
```

Watch it animated: [caching](/explainers/Caching.dc.html)


---

# Where Caches Live

Knowing what a cache *is* leads straight to a practical question: where do you put one? The surprising answer is that you rarely put just one. A single web request typically passes through several caches stacked in a line, each closer to the user and faster than the next one behind it. A hit at any layer turns the request around right there - the layers behind it never even hear about the request. Understanding that line is what lets you ask the right question when something is slow, or when something is *stale*.

## The line a request travels

Here's the whole journey, from the user's screen to the source of truth, with the caches it can hit on the way:

```mermaid
flowchart LR
  User([user]) --> Browser[browser cache] --> CDN[CDN edge] --> App[app server] --> AppCache[app cache] --> DB[(database)]
```

Each box is a place a copy of an answer can live. Let's walk them from the user outward.

## 1. The browser cache: a copy on the user's own device

**What it actually is.** Your browser keeps copies of things it has already downloaded - images, CSS files, JavaScript bundles, sometimes whole pages - right on the user's disk. The next time a page needs that same logo or stylesheet, it uses the local copy instead of asking the network at all. This is why a site feels instant on your second visit but slower on the first. The server controls how long a copy lives by sending response headers (like `Cache-Control`) that say "keep this for an hour" or "never reuse this, always re-ask."

**What it's good at.** Static assets that rarely change and belong to *one* user's view - logos, fonts, compiled CSS/JS. It's the fastest cache there is, because the answer never leaves the device.

⚠️ **Gotcha.** Because the copy lives on the *user's* machine, you can't reach in and delete it. If you ship a bug in a cached file, browsers may keep serving the old one until it expires. This is why production builds put a content hash in filenames (`app.9f2a1c.js`) - a new build changes the name, so the browser sees a new thing and fetches it fresh, sidestepping the stale copy entirely. (That trick is a preview of Phase 3's whole theme.)

## 2. The CDN: a copy at the edge, near the user

**What it actually is.** A *CDN* (Content Delivery Network) is a fleet of servers spread across the world, each keeping copies of your content close to the people near it. A user in Tokyo gets your images from a server in Tokyo instead of one in Virginia.

📝 **Terminology.** "The edge" just means *the CDN servers near the user*, as opposed to "the origin" - your actual server where the content really comes from.

**What it does in real life.** The first user in a region to request something causes a miss: the CDN fetches it from your origin, hands it over, and keeps a copy. Every later user in that region gets a hit from the nearby edge server - fast, and your origin never sees those requests. This is mainly a fix for the *distance* cost from Phase 1.

**What it's good at.** Content that's the same for many users and is read far more than it changes: images, videos, static files, and increasingly whole cached HTML pages. The more users share the same answer, the more a CDN earns its keep.

## 3. The application cache: a copy your server keeps between requests

**What it actually is.** This is the cache *you* write in your own code - the `cache.get` / `cache.set` pattern from Phase 1. Your application stores the results of expensive work so it doesn't redo them on the next request. It comes in two common flavors:

- **In-memory cache** - a copy held in your application process's own RAM. Fastest possible for your code to reach (no network hop), but it lives and dies with that one process, and each server has its own separate copy.
- **A shared cache server like Redis** - a separate service, held in RAM, that all your application servers talk to over the network. Slightly slower because of the network hop, but *shared*: every server sees the same cached answers, and the cache survives a single app restart.

📝 **Terminology.** *Redis* is a fast, in-memory data store frequently used as a shared application cache. "Put it in Redis" usually means "keep this answer in our shared, fast, in-memory cache."

*In-memory - each server keeps its own copy, so they can disagree:*
```mermaid
flowchart LR
  SA[Server A] --- CA[own copy]
  SB[Server B] --- CB[own copy]
  SC[Server C] --- CC[own copy]
```
*Shared (Redis) - one network hop, but every server sees the same copy:*
```mermaid
flowchart LR
  RA[Server A] --> Redis[(shared Redis)]
  RB[Server B] --> Redis
  RC[Server C] --> Redis
```

**What it's good at.** Expensive, repeated, *computed* answers - a rendered dashboard, an aggregation, a slow third-party API response - especially things specific to your application's logic that a CDN couldn't know how to build.

⚠️ **Gotcha.** With per-process in-memory caches, each server has its *own* copy, so they can disagree. Server A may have updated its copy while Server B still serves the old one, and a user bouncing between them sees the answer flicker. Wanting everyone to agree is a big reason teams reach for a shared cache like Redis.

## 4. The database's own caches: the source of truth is faster than you think

**What it actually is.** Even your database - the source of truth - caches internally. It keeps recently-read pages of data in RAM (its *buffer cache* / *buffer pool*) so a second read of the same rows doesn't touch the disk. Many databases also cache query plans (the worked-out strategy for running a query) so they don't re-plan an identical query from scratch.

**What it does in real life.** This happens automatically - you don't write code for it - but it's why "run the same slow query twice and the second run is faster" is so common. The first run warmed the database's own cache. It makes the source of truth itself less expensive to query, without you adding any caching layer - the safety net under everything else.

💡 **Key point.** These layers stack. A request hits the browser cache first; on a miss it goes to the CDN; on a miss there, to your app and its cache; and only a miss *there* reaches the database (which then leans on its own internal caches). When something is slow, the useful question becomes "which layer is missing?" - and when something is *stale*, "which layer is holding an old copy?"

## Recap

1. **A request passes through several caches in a line**, each closer to the user and faster than the one behind it; a hit anywhere turns the request around there.
2. **Browser cache** - copies on the user's device; best for static assets; you can't clear it remotely (hence hashed filenames).
3. **CDN / edge** - copies near the user around the world; best for shared content read far more than it changes; fixes the *distance* cost.
4. **Application cache (in-memory or Redis)** - copies your code keeps between requests; best for expensive computed answers; in-memory is fastest but per-server, Redis is shared.
5. **The database's own caches** - automatic internal caching of data pages and query plans; makes the source of truth itself cheaper to read.

Every one of these layers holds a *copy* of an answer whose truth lives further down the line - which means every one can end up holding a copy that's *out of date*. That gap between the copy and the truth is the genuinely hard part of caching, and it's next.

Watch it animated: [CDN caching](/explainers/CDNCaching.dc.html)


---

# The Hard Part - Invalidation & Staleness

Everything so far has been the friendly half of caching: keep a copy, serve it fast. This phase is the half that bites. A user updates their profile and the old name keeps showing. A price changes but checkout shows the old one. You deploy a fix and half your users still see the bug. Every one of these follows from the idea you've held since Phase 1: **the cache holds a copy, but the truth lives elsewhere.** The moment the truth changes, every copy of the old answer becomes a lie waiting to be served.

⚠️ Phil Karlton's famous line - *"There are only two hard things in Computer Science: cache invalidation and naming things"* - is a joke pointing at something real. Keeping a copy is trivial. Knowing *when the copy has gone wrong and must be thrown away* is the hard part, and it's why caching causes as many bugs as it prevents.

## Staleness: when the copy and the truth disagree

**What it actually is.** *Staleness* is when a cache serves a copy that's no longer true, because the underlying data changed but the cached copy didn't. The cache isn't broken - it's doing exactly its job, faithfully serving the answer it was told to remember. It just wasn't told the answer expired.

```text
   Time 1:  DB says name = "Sam"     Cache stores "Sam"     ✓ copy matches truth
   Time 2:  user updates name to "Sammy"  ──► DB now says "Sammy"
            ...but cache still holds "Sam"  ✗ copy now STALE
   Time 3:  request comes in → cache HIT → serves "Sam"     ← the bug
```

*What just happened:* between Time 1 and Time 2 the truth changed in the database, but nothing told the cache. At Time 3 the cache does what a cache does - returns its copy on a hit - and that copy is now wrong. No error, no crash. Just an old answer served confidently. This is the entire category of "why is it showing the old value" bugs.

📝 **Terminology.** *Cache invalidation* is the act of telling a cache "this copy is no longer trustworthy - throw it away (or refresh it)." Good caching is mostly about getting invalidation right, and it's hard because the code that *changes the truth* often lives far from the code that *holds the copy*, and the two have to stay in sync.

## TTL: let copies expire on a timer

**What it actually is.** The simplest way to limit staleness is a *TTL* - "time to live." When you store a copy, you stamp it with a lifespan: "good for 60 seconds." After that, the cache treats the copy as expired - the next request is a miss, the real work runs, and a fresh copy is stored.

```text
   cache.set("user:42:profile", value, ttl = 60s)

   t=0s   store copy        (fresh)
   t=30s  request → HIT     (still within 60s, serve copy)
   t=70s  request → MISS    (past 60s, copy expired → re-fetch the truth)
```

**What it does in real life.** A TTL caps *how stale* a copy can ever be. A 60-second TTL means "I accept showing data up to a minute old, in exchange for not hammering the database every request." There's no universal right number, just the right number *for this data's tolerance for being wrong* - stock prices might tolerate seconds, a "most popular articles" list an hour, a bank balance none at all.

⚠️ **Gotcha.** A long TTL hides bugs in slow motion. You fix data in the database, refresh the page, and it's still wrong - so you assume your fix failed and start debugging code that's already correct. Always ask "is there a TTL between me and the truth?" before concluding a change didn't work.

## Eviction: caches are small, so old copies get pushed out

**What it actually is.** A cache lives in fast, limited space (RAM is smaller and pricier than disk), so when it fills up it must throw something out to make room. *Eviction* is that process - separate from expiry. A TTL removes a copy because it got *old*; eviction removes a copy because the cache is *full*.

**The common strategy: LRU.** The usual rule is *LRU* - "least recently used." When space is needed, evict the copy that hasn't been touched in the longest time, on the bet that what you haven't used lately you're least likely to need next.

```text
   Cache is full. New item needs room. LRU evicts the coldest copy:

   recently used  ←─────────────────────────►  not used in ages
   [ home ][ profile ][ search ][ ... ][ old-report ]
                                              └─ evicted to make room
```

**Why this saves you later.** LRU explains a confusing symptom: a cache that "randomly" misses on data you *did* cache. It wasn't random - that copy was evicted because the cache filled up and it was the coldest thing in there. A mysteriously low hit rate often means the cache is too small for your working set, evicting copies before they get reused - a sizing problem, not a logic bug.

## Two strategies for keeping copies accurate

When the truth changes, *something* has to keep the cache in line. Two patterns cover most real systems.

| | **Cache-aside (lazy)** | **Write-through** |
|---|---|---|
| **Reads** | App checks cache; on a miss, reads the DB and stores the copy | Same - read from cache, fall back to DB |
| **Writes** | App writes the DB, then *deletes* (invalidates) the cached copy | App writes the DB *and* updates the cache in the same step |
| **Next read after a write** | Miss → re-fetches fresh from DB → re-caches | Hit → cache already holds the new value |
| **Strength** | Simple; cache only ever holds things actually requested | Cache and DB stay in sync on every write; fewer stale windows |
| **Weakness** | A window of staleness if invalidation is missed or races a read | More write-time work; you cache things that may never be read |

**Cache-aside** is the most common pattern - the `cache.get` / `cache.set` flow from Phase 1, plus one rule: *when you change the truth, invalidate the copy.* The discipline is remembering that delete-on-write everywhere the data can change. **Write-through** keeps the cache updated as part of writing, so a read right after a write sees the new value - trading extra work on every write for fewer stale moments.

⚠️ **Gotcha.** Neither pattern removes the danger; they relocate it. With cache-aside, the bug is *forgetting to invalidate* - one code path updates the database but doesn't clear the copy, and that field goes stale forever until its TTL (if it has one) saves you. With write-through, a write that updates the DB but fails partway can leave the cache and DB disagreeing. You're choosing which failure shape you'd rather debug.

## When *not* to cache

Sometimes the right amount of caching is none. Skip it (or be very careful) when:

- **The data must always be correct, to the moment.** Account balances, inventory counts at checkout - anywhere a stale value causes real harm.
- **The data is barely reused.** Per-user, per-request, one-off answers (Phase 1's lesson): no repetition, no payoff - only overhead and staleness risk.
- **It changes far more than it's read.** You'll spend more effort invalidating copies than you ever save serving them.
- **The underlying work is already cheap and fast.** Caching something already instant adds a layer to keep accurate for a saving you won't notice. Measure first - and if the slow thing is a database query, [Why Is My Query Slow?](/guides/why-is-my-query-slow) is often the better fix than a cache papering over it.

💡 **Key point.** A cache is a deliberate trade: speed and load relief in exchange for the *risk and effort of keeping copies accurate*. When the data tolerates being a little old and gets read repeatedly, that trade is a clear win. When it must be exact, or is rarely reused, you're paying the cost of caching for none of the benefit. Decide on purpose, not by reflex.

## Recap

1. **Staleness is the core problem:** the cache holds a copy, the truth changes elsewhere, and the now-wrong copy keeps getting served on hits. No crash - just an old answer.
2. **Cache invalidation** - throwing away a copy that's no longer trustworthy - is the genuinely hard part, because the code that changes the truth is often far from the code that holds the copy.
3. **A TTL** caps how stale a copy can get by expiring it on a timer; the right TTL equals how much staleness this data tolerates. Long TTLs hide working fixes - always check for one before assuming a change failed.
4. **Eviction** removes copies because the cache is *full* (commonly **LRU** - drop the coldest); it explains "random" misses on data you thought you cached.
5. **Cache-aside vs. write-through** are the two main ways to keep copies accurate - one invalidates on write, the other updates on write - and each relocates the danger rather than removing it.
6. **Don't cache** data that must be exact, is barely reused, changes more than it's read, or is already fast. Caching is a trade; make it on purpose.

You now hold the whole idea: a cache is a copy of an expensive answer kept somewhere fast (Phase 1), those copies live stacked from the browser to the database (Phase 2), and the real work is keeping each copy true to the truth (Phase 3). That's why the old joke survives - and why, the next time something shows the wrong value, your first thought will be the right one: *which layer is holding a stale copy?*

**Related:** [Why Is My Query Slow?](/guides/why-is-my-query-slow) · [Designing for Scale](/guides/designing-for-scale)