# Processes, Memory & the CPU - Diagnosing a Slow or Stuck Machine > What '100% CPU' and 'out of memory' actually mean, and how to find the one process that's the culprit - using top and Task Manager you already have. --- # Processes, Memory & the CPU The machine is dragging. The fan is screaming, the cursor stutters, and a status bar somewhere says **100% CPU** or your editor pops up a box that says **out of memory** and dies. You don't know which program did it, or even what those words really mean - so it feels like the computer is just *unwell*, the way a person gets a fever. Here's the relief: those phrases aren't vague. They name something exact, happening right now, that you can *find* - usually one specific process, sitting in a list, misbehaving. This guide teaches you to read that list. By the end, "100% CPU" and "out of memory" stop being weather and become a row you can point at. This guide builds on [What an Operating System Is](/guides/what-an-operating-system-is). If "process," "kernel," or "RAM vs. disk" are fuzzy, read that first - we won't re-teach the basics here, we'll go deeper into the two jobs (running programs and handing out memory) that slowdowns come from. ## How to read this - **Machine on fire right now?** Jump to whichever symptom matches - CPU pinned at 100% is [Phase 2: What "100% CPU" Really Means](02-what-100-cpu-really-means.md); "out of memory" or grinding-to-a-crawl slowness is [Phase 3: What "Out of Memory" Really Means](03-what-out-of-memory-really-means.md). Each opens with how to find the culprit. - **Want it to finally make sense?** Read in order. Phase 1 gives you the vocabulary (what a process actually is and how you stop one) that the other two phases lean on. ## The phases 1. **[Processes, Up Close](01-processes-up-close.md)** - what a process really is: its PID, its parent, foreground vs. background, the states it sits in (running, sleeping, zombie), and what Ctrl-C, `kill`, and "End task" actually *do* to it. 2. **[What "100% CPU" Really Means](02-what-100-cpu-really-means.md)** - cores, the scheduler handing out turns, what "load average" tells you, why one runaway process can pin a core, and how to spot it in `top` / Task Manager. 3. **[What "Out of Memory" Really Means](03-what-out-of-memory-really-means.md)** - RAM vs. virtual memory, paging/swap and why swapping makes everything crawl, what "this app uses 4 GB" means, and the OOM killer - the OS killing a process to save itself. > This guide is about *diagnosis* - naming the culprit. Deep tuning (changing scheduler priorities with `nice`, sizing a swap file, configuring cgroup memory limits) is deferred to a follow-up guide; here we get you to "that process, right there." --- # Processes, Up Close You already know the headline from [the OS guide](/guides/what-an-operating-system-is): a **process** is a program that's actually running. That's the right mental model, but it's too smooth to debug with. When the machine is stuck, you need the texture - *which* process, who started it, what it's doing right now, and how you make it stop without nuking everything around it. A process isn't a vague cloud of "the app running." It's a thing the OS keeps careful books on: it has a number, a parent, a current state, and a small set of doorbells you can ring to tell it to quit. Learn those four facts and a "stuck" program becomes something you can actually grab. ## Every process has a number: the PID **What it actually is.** When the OS starts a process, it stamps it with a unique integer - the **PID** (process ID). That's the name the OS uses internally; the human-readable name (`firefox`, `python`) is just a label for you. The PID is how every tool refers to a process: "slow down PID 4821," "kill PID 9032." 📝 **Terminology.** *PID* = Process ID, a number the OS assigns when a process starts and reuses only after that process is long gone. It's the handle you grab a process by. **A real example.** On macOS or Linux, `ps` lists processes. Here's a focused view: ```console $ ps -o pid,ppid,stat,command PID PPID STAT COMMAND 4821 1190 S /usr/lib/firefox/firefox 9001 4821 S firefox --type=renderer (a tab) 9032 8800 R ps -o pid,ppid,stat,command ``` *What just happened:* `ps` printed four columns. `PID` is each process's own number, `PPID` is its **parent's** PID, `STAT` is its current state (more below), `COMMAND` is what's running. Notice PID `9001` (a browser tab) has `PPID 4821` - Firefox itself. The browser *started* the tab, so the tab is its child. ⚠️ **Gotcha.** PIDs get **reused**. After a process exits, the OS is free to hand its number to a brand-new, unrelated process later. So don't save a PID from this morning and assume it's the same program this afternoon - re-check the name before you act on a number. ## Parents and children: where processes come from **What it actually is.** No process appears from nowhere. Every process is *started by another process* - its **parent** - forming a family tree. Your terminal starts the commands you type; Firefox starts a process per tab. At the very root sits the first process the kernel launched at boot (PID 1), the ancestor of everything. ```mermaid flowchart TD init["PID 1 (first process, started by the kernel at boot)"] --> session[login / your desktop session] session --> terminal[terminal, PID 8800] session --> firefox[firefox, PID 4821] terminal --> ps["ps, PID 9032 (you typed this)"] firefox --> tab1[tab, PID 9001] firefox --> tab2[tab, PID 9002] ``` **Why this matters in real life.** The tree explains things that otherwise look like magic. Close Firefox and its tabs vanish too - killing a parent usually takes its children with it. It also explains *blame*: if some `python` process is eating your CPU, its parent (the `PPID`) tells you *what launched it* - a cron job? your editor? a runaway script? - which is often the real thing to fix. ## Foreground vs. background **What it actually is.** A **foreground** process is the one currently holding your terminal (or window) hostage - you typed a command and you're waiting for it, and your keystrokes (including Ctrl-C) go to it. A **background** process runs without sitting on your prompt; it's detached, doing its work while you do other things. Most of the 300-odd processes on your machine are background: services, daemons, helpers you never see. In a shell, `&` starts something in the background, and you get your prompt back immediately: ```console $ ./long-backup.sh & [1] 9105 $ ``` *What just happened:* the shell started `long-backup.sh` as a background process with PID `9105` (the `[1]` is the shell's own shorthand "job number"), then handed your prompt right back so you can keep working. ⚠️ **Gotcha.** A background process is *not* hidden from the OS - it still uses CPU and memory, and it'll still show up in `top` and Task Manager. "Background" means "not blocking your prompt," not "free." ## The states a process sits in **What it actually is.** A process is almost never running every instant - it spends most of its life *waiting*. The OS tracks which of a few states each one is in. You don't need the full list; three carry almost all the meaning: ```mermaid stateDiagram-v2 [*] --> Running Running --> Sleeping: waits for input / network / disk Sleeping --> Running: what it waited for arrived Running --> Zombie: finished Zombie --> [*]: parent collects the "I'm done" notice ``` *Running = on a CPU core (the only state that burns CPU). Sleeping = waiting, using ~no CPU; where most processes sit most of the time. Zombie = finished but not yet cleared from the table; harmless leftover bookkeeping.* In `ps`, the `STAT` column shows these: `R` = running, `S` = sleeping, `Z` = zombie (Task Manager shows similar wording under a "Status" column - "Running," "Suspended"). **Why this saves you later.** Two big misreadings die here. First: seeing "312 processes" and panicking - *almost all of them are sleeping*, costing you nothing; a busy machine and a crowded process list are different things. Second: the word **zombie** sounds alarming, but it's harmless leftover bookkeeping, not a CPU or memory hog. A pile of zombies points to a buggy *parent* not cleaning up after its children - annoying, but not what's making your fan scream. 📝 **Terminology.** A *zombie* (or "defunct") process has already exited; it lingers only as a one-line entry until its parent acknowledges it. It is not "a process gone rogue" - that's the opposite, a *running* process pinning a core (Phase 2). ## How you actually stop a process: signals This is the part nobody explains, so it feels like superstition. Ctrl-C, `kill`, "End task," "Force quit" - they all do *one underlying thing*: send the process a **signal**, a tiny predefined message the OS delivers meaning roughly "something happened - here's what." Stopping a program is choosing *which* message to send. 📝 **Terminology.** A *signal* is a short, numbered notification the OS hands to a process. A handful matter for stopping things; the names are more useful than the numbers. **The two that matter most:** ```text SIGTERM (15) "Please wrap up and exit." The POLITE ask. The process can catch it, finish writing files, close connections, then quit cleanly. This is the default - and almost always what you want. SIGKILL (9) "Stop. Now. No cleanup." The kernel removes the process immediately. It gets NO chance to save or tidy up. The nuclear option, for when a process is too stuck to even hear SIGTERM. ``` Now the everyday actions decode cleanly: - **Ctrl-C** in a terminal sends **SIGINT** ("interrupt") to the foreground process - a close cousin of "please stop." That's why Ctrl-C cancels *only* the command you're waiting on. - **`kill `** sends **SIGTERM** by default - the polite ask - despite the scary name. - **`kill -9 `** sends **SIGKILL** - the no-mercy version. - **"End task" (Task Manager) / "Force Quit" (macOS)** ask politely first and escalate to the forceful kill if the program won't go. **A real example.** ```console $ kill 9105 $ $ kill 9105 bash: kill: (9105) - No such process ``` *What just happened:* the first `kill` sent SIGTERM to PID `9105` and it exited cleanly - no output means it worked. The second `kill` failed because the process is already gone. ("No such process" here is good news, not an error to fix.) ⚠️ **Gotcha.** Reach for `kill -9` (SIGKILL) only after a plain `kill` (SIGTERM) has clearly failed. Because SIGKILL gives the process *zero* chance to clean up, you can lose unsaved work, leave a half-written file, or corrupt a database mid-write. Polite first; nuclear only when polite is ignored. 🪖 **War story.** A script "stuck forever" that gets `kill -9`'d every time can leave you confused by half-written output files an hour later. The script wasn't stuck - it was *sleeping*, waiting on a slow network call, and SIGKILL kept yanking it mid-write. A plain `kill` (SIGTERM) lets it notice, abort the call, and clean up its own mess. ## Recap 1. **PID** - every process has a unique number; that's how every tool grabs it. PIDs get reused, so re-check the name. 2. **Parent/child** - every process is started by another, forming a tree; the parent (`PPID`) tells you *what launched* a misbehaving process. 3. **Foreground vs. background** - foreground holds your prompt (and receives Ctrl-C); background runs detached but still uses real resources. 4. **States** - only **running** burns CPU; **sleeping** is the normal resting state; **zombie** is harmless dead bookkeeping. 5. **Signals** - stopping a process means sending it a signal. **SIGTERM** (plain `kill`, "End task") asks politely; **SIGKILL** (`kill -9`) is the no-cleanup nuke. Polite first. Now that you can name and grab a single process, let's use that to answer the first big "the machine is on fire" question: what does **100% CPU** actually mean, and how do you find the one process causing it? Watch it animated: [processes](/explainers/ProcessesThreads.dc.html) --- # What "100% CPU" Really Means The fan spins up to jet-engine volume, the cursor jerks, and something reports **100% CPU**. The instinct is to read that as "the computer is full, there's no room" - like a glass overflowing. That picture is wrong, and it's why the situation feels hopeless. Here's the true picture: 100% CPU doesn't mean *broken*, it means *fully booked*. The CPU is doing the maximum work it can per second, and right now demand is higher than supply. The useful question is never "why is it 100%" - it's **"who is taking it all?"** And that has an answer you can read off a list in about ten seconds. > ⏭️ This phase uses *process*, *PID*, and *running vs. sleeping* from [Phase 1](01-processes-up-close.md). If those are fuzzy, skim Phase 1's recap first. ## Find the culprit (do this first) You're probably here mid-slowdown, so the cheat-card comes first; the *why* is underneath. | You're on | Open it | Then | |---|---|---| | **Windows** | Task Manager (Ctrl + Shift + Esc) | Click the **CPU** column header to sort; the top row is your culprit | | **macOS** | Activity Monitor → **CPU** tab | Click the **% CPU** column to sort descending | | **Linux / any terminal** | type `top` | It already sorts by CPU; the top row is the culprit. Press `q` to quit | The process at the top of that sorted list, sitting at a high `%CPU`, is what's burning your machine. Now here's why that number means what it means. ## A CPU is several cores: several lanes, not one **What it actually is.** Your CPU isn't a single worker - it's several independent workers called **cores**, each able to run one process at a time. A "4-core" chip can genuinely do four things at once; an 8-core, eight. Think of cores as checkout lanes at a store: more lanes, more customers served simultaneously. 📝 **Terminology.** A *core* is one independent processing unit inside the CPU. "How many cores" = how many processes can *truly* run at the same instant. (Some chips advertise more "threads" than cores via hyper-threading - treat each thread as roughly a lane for our purposes.) **Why this matters for that percentage.** This is the trap in "100% CPU." On some tools the number is averaged across *all* cores, so 100% means *every* lane is full. But other tools - including the classic `top` - count each core as 100%, so a single process hammering one core out of four shows up as **100%** there, even though three lanes are idle. Knowing whether you're looking at per-core or overall is the difference between "one process gone wild" and "the whole machine is slammed." ## The scheduler: how dozens of processes share a few lanes **What it actually is.** You have hundreds of processes and maybe 4-8 cores. The part of the kernel that decides *who runs on which core, and for how long* is the **scheduler**. It runs a process for a few milliseconds, pauses it, runs the next, and cycles through everyone fast enough that it *looks* simultaneous - the trick from the OS guide, now with a name. ```mermaid flowchart LR ms1[ms 1: chrome] --> ms2[ms 2: music] --> ms3[ms 3: editor] --> ms4[ms 4: chrome] --> ms5[ms 5: music] --> ms6[ms 6: editor] ``` *One core, a few milliseconds: the scheduler rotates turns. Light load and everyone gets a turn instantly (feels fast); too many hungry processes and your turn comes around late (feels sluggish).* **Why this saves you later.** "Slow" usually isn't a broken CPU - it's the scheduler with more demand than it can satisfy, so each process's turn arrives later. That's why closing a couple of greedy programs makes everything *else* snap back: you freed up turns for the rest. The CPU was never sick; it was overbooked. ## Load average: the line at the door **What it actually is.** On macOS and Linux, `top` shows three **load average** numbers - roughly, the average number of processes that *wanted* a CPU core over the last 1, 5, and 15 minutes: those running plus those queued and waiting. It's the length of the line at the checkout, including people already being served. **How to read it (the key rule):** compare load to your **core count**. ```text load average: 0.42, 0.55, 0.59 ← example readout On a 4-core machine: load ~4 → lanes roughly full; healthy and busy load < 4 → cores to spare (0.42 here = very relaxed) load > 4 → more demand than cores; processes queueing, things lag load 8 on 4 cores → twice the work the cores can serve → sluggish ``` *What just happened:* these three numbers (an example, not a measurement of your machine) read newest-first: `0.42` over the last minute, `0.55` over five, `0.59` over fifteen - all well under a 4-core budget, and roughly flat, a calm machine with no buildup. If the one-minute figure were *much higher* than the fifteen-minute one, that's a spike just starting; *much lower* means a storm that's passing. ⚠️ **Gotcha.** Load average counts processes waiting on **disk and other resources**, not only the CPU. A machine can show high load while the CPU itself looks idle - usually a *disk* or *I/O* bottleneck, not a CPU one. High load + low CPU% = look at the disk, not the cores. ## Why one runaway process can pin a whole core **What it actually is.** A **runaway** process is one stuck doing useless work without pausing - most often an infinite loop, or a retry that never gives up. Remember from Phase 1: only a *running* process burns CPU. A normal program does a little work, then **sleeps** (waiting for input, network, a timer). A runaway never sleeps - it stays *running*, so the scheduler keeps handing it turns, and it devours one core completely. ```text Healthy process: work · sleep · work · sleep · work · sleep (shares nicely) Runaway process: work · work · work · work · work · work … (never lets go) └─ pins one core at 100%, fan screams ─┘ ``` **A real example.** Here's `top` with a culprit, sorted by CPU (the default): ```console $ top top - 16:48:22 up 5 days, 6:31, 2 users, load average: 1.74, 0.98, 0.71 Tasks: 318 total, 2 running, 316 sleeping %Cpu(s): 26.1 us, 1.4 sy, 72.0 id MiB Mem : 15872.0 total, 4810.2 free, 7002.1 used, 4059.7 buff/cache PID USER %CPU %MEM TIME+ COMMAND 7731 ada 99.4 0.6 2:14.83 python3 4821 ada 6.2 7.1 4:03.11 firefox 1190 ada 1.1 2.1 1:22.04 gnome-shell 9032 ada 0.4 0.3 0:00.09 top ``` *What just happened:* (an illustrative readout) `Tasks: ... 2 running` - out of 318 processes, only **two** are actually running; the other 316 are sleeping, costing nothing. The top row is the whole problem: PID `7731`, a `python3` script, at **99.4% CPU** - on this per-core tool, one core completely pinned. Its `TIME+` of `2:14` and the rising `load average` (1.74 over the last minute vs 0.71 over fifteen) both say this started recently and is climbing. Everything else - even Firefox - is barely using the CPU. The machine isn't sick; *one script* is. From here you have the two facts you need: the **name** (`python3` - what is it? a script you ran? a stuck tool?) and the **PID** (`7731` - the handle to stop it). Recall its parent with `ps` if you want to know *what launched it* before you kill it. ⚠️ **Gotcha.** Before you `kill -9` a CPU hog, check it's truly *runaway* and not just *busy*. A video export, a code compile, or a backup *should* use lots of CPU - that's the work you asked for, and it'll finish. A runaway is work that has no end: same task, climbing `TIME+`, no progress, often a process you didn't knowingly start. 🪖 **War story.** The classic 2am page - "server slow, users complaining" - is often `top` sorted by CPU showing one row at `99%`: a deploy script that hit an error and retried in a tight loop with no delay, spinning one core forever. The fix wasn't restarting the server or "adding more CPU." It was `kill 7731` (politely) and a one-line `sleep` added to the retry. ## Recap 1. **100% CPU = fully booked, not broken.** The right question is *who's taking it,* and that's a sortable list. 2. **Cores are lanes.** Several processes run truly at once; the per-tool meaning of "100%" depends on whether it counts per-core or overall. 3. **The scheduler** hands out rapid turns; "slow" is usually more demand than turns, which is why closing greedy apps revives the rest. 4. **Load average** = the line at the door; compare it to your **core count**. High load with idle CPU means a *disk/I-O* bottleneck. 5. **A runaway process** never sleeps, so it pins a core. Find it by sorting on CPU; grab it by **name + PID**; confirm it's runaway (endless, pointless) not just busy (purposeful, finite) before you kill it. CPU is one of the two things a stuck machine runs out of. The other feels different - not a screaming fan but a grinding, molasses slowness, ending in "out of memory." That's a different mechanism with a different fix, and it's next. Watch it animated: [CPU scheduling](/explainers/CPUScheduling.dc.html) --- # What "Out of Memory" Really Means This slowdown has a different flavor than the CPU one. The fan isn't screaming; instead everything turns to *molasses* - switching apps takes seconds, the disk light is solid, and eventually a box says **out of memory** and something dies. It feels like the machine is choking, slowly. The thing to understand up front: "out of memory" is rarely a clean wall the machine slams into. Long before it gives up, the OS fights to keep going by quietly shuffling memory out to the disk - and *that fight is the slowness.* The crawl isn't a symptom of running low; it's the OS's survival strategy, and it's expensive. > ⏭️ This phase leans on *RAM vs. disk* and *process* from [the OS guide](/guides/what-an-operating-system-is) and Phase 1. The one-line version: **RAM is the fast desk you work on; the disk is the slow filing cabinet.** ## Find the culprit (do this first) | You're on | Open it | Then | |---|---|---| | **Windows** | Task Manager (Ctrl + Shift + Esc) | Click the **Memory** column to sort; the top row is the hog | | **macOS** | Activity Monitor → **Memory** tab | Click **Memory** to sort; check the **Memory Pressure** graph (green = fine, red = trouble) | | **Linux / any terminal** | `top`, then press **`M`** (capital) | Sorts by memory; top row is the hog. `q` to quit | The process at the top of the memory-sorted list is using the most RAM. Now let's understand what that number is, and why a full one drags the whole machine down. ## RAM and virtual memory: the desk and the trick on top of it **What it actually is.** **RAM** is the fast working memory where processes keep what they're actively using - your desk. It's limited (say 16 GB) and far faster than disk. **Virtual memory** is a clever layer the OS puts *on top* of RAM so each process is handed its own private, tidy address space - it thinks it has a clean, continuous stretch of memory all to itself, and the OS secretly maps those addresses to wherever the real bytes happen to live (in RAM, or temporarily on disk). 📝 **Terminology.** *Virtual memory* = the illusion the OS gives each process of its own private, contiguous memory. It's what lets the OS move the *real* data around (even onto disk) without the process noticing. *Physical memory* = the actual RAM chips. **Why the OS bothers.** Two big problems solved at once. **Isolation:** because each process sees its own address space, one process literally can't read or scribble on another's memory. **Overcommitment:** the OS can promise more memory than physically exists, betting that not everything is needed at once - and when that bet gets tight, it falls back on the disk trick below. ## Paging and swap: the slow trick that saves you (and costs you) **What it actually is.** When RAM fills up, the OS doesn't crash. It finds memory that hasn't been touched in a while - a background app, an idle tab - and writes it out to a reserved area on the **disk**, freeing that RAM for whatever needs it now. Moving memory between RAM and disk like this is **paging**; the disk area it uses is called **swap** (Windows: the *page file*; macOS: *swap files* it manages automatically). ```mermaid flowchart LR RAM["RAM, fast
active tab · editor in use · idle app untouched 10 min"] -->|page out idle app| Swap["Disk / swap, slow
idle app paged out"] Swap -->|click the idle app: page it back in| RAM ``` *RAM is full and a new program needs room, so the OS pages the idle app out to disk - freeing RAM now. Later, clicking that idle app forces the OS to page it back from disk first (the beachball/spinner). That wait IS the swap cost.* **Why this is the slowness.** Disk is *dramatically* slower than RAM - orders of magnitude. As long as paging is occasional, you barely notice. But when RAM is so tight that the OS is *constantly* shuffling pages out and pulling them back - every app switch forcing another disk round-trip - the machine spends more time moving memory than doing your work. That pathological state has a name: **thrashing**. The molasses feeling, the solid disk light, the multi-second app switches - that's thrashing, the OS frantically swapping to avoid the alternative. ⚠️ **Gotcha.** A little swap *in use* is not a problem - the OS pages out genuinely-idle stuff to keep RAM free for active work; that's healthy. The alarm signal isn't "swap > 0," it's **constant** paging activity while you're actively using the machine (on macOS, the Memory Pressure graph going yellow/red; on Linux, `top`'s swap line steadily climbing). ## What "this app uses 4 GB" actually means **What it actually is.** It sounds like a simple weight, but memory has layers, which is why Task Manager and `top` can show two different numbers for the same process. The two that matter: ```text RESIDENT memory (RES / "Memory" in Task Manager) = how much actual RAM the process is using RIGHT NOW. This is the number that counts toward "are we out of RAM?" VIRTUAL memory (VIRT) = the total address space the process has CLAIMED, including parts paged out to disk and parts merely reserved-but-unused. Almost always much bigger than RES - and usually NOT worth worrying about. ``` **Why this saves you later.** When you're hunting a memory hog, look at **resident** memory (RES / "Memory"), not virtual. A process can show a frightening 30 GB of *virtual* memory while actually holding 800 MB of real RAM - that's normal, not a leak. "This app uses 4 GB" only means something scary if that 4 GB is *resident*. 📝 **Terminology.** A *memory leak* = a process that keeps allocating memory and never releases it, so its **resident** memory only ever grows. The tell isn't a big number at one moment - it's a number that *climbs and never comes back down* even when the app is idle. **A real example.** `top` after pressing `M` to sort by memory: ```console $ top top - 17:12:40 up 5 days, 6:55, 2 users, load average: 2.31, 2.10, 1.84 MiB Mem : 15872.0 total, 180.4 free, 14102.7 used, 1588.9 buff/cache MiB Swap: 4096.0 total, 402.1 free, 3693.9 used PID USER %CPU %MEM VIRT RES TIME+ COMMAND 6620 ada 4.1 61.0 18.2g 9.5g 8:40.21 java 4821 ada 2.0 14.2 6.1g 2.2g 5:11.30 firefox 1190 ada 0.8 2.0 3.0g 327540 1:40.66 gnome-shell ``` *What just happened:* (an illustrative readout) The header tells the grim story first: of ~16 GB RAM, only `180.4` MiB is **free**, and `MiB Swap` shows `3693.9` of 4096 MiB swap **used** - the OS has shoved nearly 4 GB onto disk and is still scraping for room. That's a machine deep in thrashing, which is why the `load average` is elevated too (processes stuck waiting on the slow disk, the high-load-from-I/O case from Phase 2). The hog is obvious: PID `6620`, `java`, holding **9.5 GB resident** (`RES`) - 61% of all RAM. Its `VIRT` is `18.2g`, almost double its real footprint; the number that matters is the `9.5g` RES. The diagnosis writes itself: one Java process is eating real RAM, forcing everything else out to swap, which is why the whole machine crawls. The fix is that one process - not "buy more RAM" as a reflex. ## When even swap isn't enough: the OOM killer **What it actually is.** Sometimes paging can't save the day - memory demand outruns RAM *and* swap. The OS now faces a genuinely bad choice: freeze the entire system, or sacrifice something. Linux chooses to sacrifice: a part of the kernel called the **OOM killer** ("Out Of Memory killer") wakes up, picks the process it judges most responsible (roughly: the biggest memory user, weighed against a few factors), and kills it - SIGKILL, no negotiation - to claw back enough RAM to keep the *rest* of the system alive. ```mermaid flowchart TD Full["RAM full → swap full → next allocation can't be satisfied"] --> Pick[OOM killer wakes, scores processes by memory use, picks one] Pick --> Kill["SIGKILL the chosen victim (gone with no warning, no cleanup)"] Kill --> Survive[RAM freed → system survives] ``` **Why this matters in real life.** This explains one of the most baffling experiences in computing: **a program vanishes with no error, no crash dialog, nothing.** You didn't close it; it didn't crash *itself* - the OS executed it to survive. On Linux servers the fingerprint is unmistakable in the system log: ```console $ sudo dmesg | grep -i "killed process" [284913.557] Out of memory: Killed process 6620 (java) total-vm:19089920kB, ... ``` *What just happened:* the kernel logged that *it* killed PID `6620` (`java`) because the system was out of memory. The "java just disappeared" mystery has a precise, written answer: it didn't disappear - the OOM killer ended it deliberately, and left a note. ⚠️ **Gotcha.** The OOM killer's victim **isn't always the true cause.** It targets whoever's *biggest right now*, which may be an innocent bystander that happened to be large when a *different*, leaking process pushed the system over the edge. When something gets OOM-killed, don't stop at the victim - check what was *growing* in the minutes before. The casualty and the culprit can be two different processes. 🪖 **War story.** A service that kept dying nightly "for no reason" - no crash, no error in *its* logs - got blamed by the app team for weeks. One line of `dmesg` ended it: `Out of memory: Killed process … (the- service)`. The machine was running a nightly batch job that ballooned RAM; the OOM killer reaped the biggest thing in sight, which happened to be the service, not the batch job. The real fix was capping the batch job's memory. ## Recap 1. **"Out of memory" is rarely a clean wall.** Long before it, the OS pages idle memory to disk to keep going - and that paging *is* the slowness. 2. **RAM is the fast desk; virtual memory** is the per-process illusion that lets the OS isolate processes and quietly move real data (even onto disk). 3. **Paging/swap** rescues you from crashing but is far slower than RAM; *constant* paging while you work is **thrashing** - the molasses, the solid disk light. 4. **Hunt resident memory (RES / "Memory"), not virtual.** A leak is resident memory that climbs and never falls. 5. **The OOM killer** is why a program can vanish with no error: out of RAM and swap, the OS SIGKILLs a big process to survive. Check `dmesg` - and remember the victim may not be the culprit. You now have the whole diagnostic loop. A slow or stuck machine is no longer weather: it's either CPU (a screaming fan, a *running* process pinning a core - Phase 2) or memory (molasses, a solid disk light, swap churning, maybe an OOM kill - this phase). Either way, the answer is the same calm move that started this guide: open the list, sort it, and find the one row that's the problem. > **Where next.** This guide got you to *diagnosis*. The follow-up - adjusting priorities with `nice`, > sizing swap, and setting per-process memory limits so the OOM killer reaps the *right* thing - is > *intervention*, and deserves its own guide. For broader context, the rest of the > [Operating Systems track](/guides/what-an-operating-system-is) and > [The Terminal & Shell](/guides/the-terminal-and-shell) are good neighbors. Watch it animated: [virtual memory](/explainers/VirtualMemory.dc.html)