# Linux for Servers

> The leap from desktop Linux to running a real server: a headless box you reach over SSH, services managed by systemd, logs read with journalctl, and a small discipline of users, cron, firewall, and updates that keeps it safe.


---

# Linux for Servers

You're comfortable on a Linux desktop - you live in the terminal, you know your way around files and
permissions. Now someone has handed you a server: a fresh cloud box, or the production machine the team
depends on, and the expectation is that you'll *administer* it. No wallpaper to click, no Software Center,
no "I'll figure it out in the GUI." Just an IP address, a username, and the quiet understanding that if you
break it, something real goes down.

The good news is that a server is the same Linux you already know - the kernel, the filesystem, the
permissions, all of it. What changes is the *posture*. A server has no screen, runs unattended for months,
and exists to keep a handful of long-running services alive. Once you internalize that shift, the unfamiliar
tools (`systemctl`, `journalctl`, `ufw`) stop looking like a new operating system and start looking like
exactly what you'd expect a headless, always-on machine to need.

This guide assumes you've done [Linux From Zero](/guides/linux-from-zero) and are at home in
[the terminal and shell](/guides/the-terminal-and-shell). We won't re-teach `cd`, `ls`, or `chmod`. We'll
build on them.

## How to read this
- **About to log into a box right now?** Skim [Phase 1: The Server Mindset](01-the-server-mindset.md) for
  the `ssh` connect, then keep the [server-hardening cheat-card](03-running-it-safely.md) from Phase 3 open
  in another tab.
- **Want it to finally make sense?** Read in order - the mindset makes systemd make sense, and systemd
  makes the safety practices make sense.

## The phases
1. **[The Server Mindset](01-the-server-mindset.md)** - what's actually different about a server: headless,
   reached over SSH, everything is config files and long-running services. Plus the basic `ssh user@host`
   connect, annotated.
2. **[Managing Services with systemd](02-managing-services-with-systemd.md)** - what systemd *is* (the first
   process, the service manager), `systemctl` start/stop/restart/enable/status, and reading logs with
   `journalctl`, on real services.
3. **[Running It Safely](03-running-it-safely.md)** - least-privilege users and `sudo`, scheduled jobs with
   `cron`, firewall basics with `ufw`, SSH hardening, keeping packages updated - ending in a server-hardening
   cheat-card.

> Deliberately deferred to a follow-up infrastructure track: SSH-key generation and agent forwarding in
> depth, configuration management (Ansible/cloud-init), reverse proxies and TLS, containers, and monitoring
> stacks. This guide gets you to *competently administering one box by hand* - the foundation everything
> else is automated on top of.


---

# The Server Mindset

Before a single command, let's fix the mental model - because almost everything that feels strange about a
server comes from one change, and once you see it, the strangeness evaporates.

You already know desktop Linux. A server is *that same Linux* - same kernel, same `/etc`, same permissions,
same shell. What changed is that this machine has no person sitting in front of it, and it isn't there to
be *used* - it's there to *serve*. Hold those two facts and the rest follows.

## What's actually different about a server

**What it actually is.** A server is a Linux box that runs **headless** (no monitor, keyboard, or graphical
desktop), that you reach **over the network** instead of in person, and whose whole reason for existing is
to keep a few **long-running services** alive and answering requests - a web server, a database, an SSH
daemon - for weeks or months without a human touching it.

**Why people get this wrong.** Coming from the desktop, the instinct is "where's the GUI? how do I open the
settings app?" There isn't one, and that's not a missing feature - it's the point. A GUI needs a screen, a
logged-in session, and a pile of graphics packages that would sit idle and just add attack surface. A
server strips all of that away so the machine spends its resources on the actual job.

Here's the shape of the difference, side by side:

```text
   DESKTOP LINUX                          SERVER LINUX
   ─────────────────────────────         ─────────────────────────────
   you sit in front of it          │     you reach it over SSH (a network login)
   a GUI you click through         │     a shell, config files in /etc, that's it
   apps you launch and quit        │     services that start at boot, run for months
   you log in, do a thing, log off │     it runs unattended, you visit to administer
   reboot whenever                 │     a reboot means downtime - you plan it
```

**Why this saves you later.** Every "how do I do X on a server" question answers itself once you accept the
posture. *Change a setting?* Edit a config file under `/etc` (it's text - that's why servers are scriptable
and reproducible). *Make something run all the time?* That's a service (Phase 2). *See what's happening?*
Read logs, not a status window (also Phase 2).

📝 **Terminology.** *Headless* = a machine running with no display, keyboard, or mouse attached. *Daemon* =
a long-running background service process; by convention its name often ends in `d` (`sshd`, `nginx` runs as
a daemon, `systemd` itself). *Service* and *daemon* are used almost interchangeably in practice.

## You reach it over SSH

The one skill you cannot administer a server without is **SSH** - *Secure Shell*. It's how you get a shell
on a machine that's somewhere else, with everything between you and it encrypted.

**What it actually is.** SSH gives you a terminal session on a remote machine, exactly as if you'd opened a
terminal while sitting in front of it - except the keystrokes travel over an encrypted network connection.
You type locally; the commands run *there*. There's a client on your laptop (`ssh`) talking to a daemon on
the server (`sshd`, which we'll meet again in Phase 2 as a real service).

**A real example.** The basic connect is one command:

```console
$ ssh deploy@203.0.113.10
The authenticity of host '203.0.113.10 (203.0.113.10)' can't be established.
ED25519 key fingerprint is SHA256:Z9k3...Qp1A.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '203.0.113.10' (ED25519) to the list of known hosts.
deploy@203.0.113.10's password:
Welcome to Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-41-generic x86_64)

Last login: Wed Jun 18 09:14:02 2026 from 198.51.100.7
deploy@web-prod-1:~$
```

*What just happened:* `ssh deploy@203.0.113.10` means "open a shell on the host at `203.0.113.10` as the
user `deploy`." Because this was your first connection, SSH showed you the server's **host key
fingerprint** and asked you to confirm it - proof of *which* machine you're talking to, so nobody can
impersonate it later. You said `yes`, authenticated (here with a password), and the prompt changed to
`deploy@web-prod-1:~$` - the tell that every command you type now runs *on the server*. `exit` (or Ctrl-D)
drops you back to your own machine.

⚠️ **Gotcha.** That fingerprint prompt only happens *once* per host. If you ever connect again and SSH
instead prints a loud red `WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!`, do **not** blindly type `yes`.
It means the host key is different from the one you trusted before - usually an innocent server rebuild, but
potentially someone intercepting your connection. Confirm out-of-band why the key changed before you clear
the old entry from `~/.ssh/known_hosts`.

📝 **Terminology.** *Host* = the remote machine (its IP or DNS name). *Host key* = the server's permanent
cryptographic identity; the *fingerprint* is its short, human-checkable form. *known_hosts* = the local file
where your SSH client records host keys it has trusted.

### A note on passwords vs. keys

You just logged in with a password, which works but is the weaker option. Real servers use **key-based
authentication**: you hold a private key on your laptop, the server holds your public key, and you connect
without typing a password - more convenient and dramatically harder to brute-force. We revisit setting this
up when we harden SSH in [Phase 3](03-running-it-safely.md). For now: *passwords get you in, keys are how
grown-up servers actually do it.*

🪖 **War story.** The classic first-day mistake: leave an `ssh` session open, run a long task in it, close
the laptop - and watch the task die when the connection drops. A server runs unattended, but *your shell
session does not*: it's tied to your connection. The fix is to detach long work from your session (a
`systemd` service, a scheduled job, or `tmux`), which is exactly the muscle the rest of this guide builds.

## Everything is a file you can edit

One more piece of the mindset. On a server, configuration isn't hidden behind dialog boxes - it lives in
**plain-text files**, overwhelmingly under `/etc`. The SSH daemon's behavior is in `/etc/ssh/sshd_config`. A
web server's sites are in `/etc/nginx/`. Scheduled jobs, users, mounted disks, the firewall - all text.

**Why this is the whole game.** Text config is what makes a server *operable at a distance and reproducible*
- you can read it over SSH, diff it, put it in version control, copy it to the next server, let automation
generate it. That's what lets one person administer a hundred machines. Every time you reach for a setting:
*which file under `/etc` owns this, and what does it say right now?*

## Recap

1. A server is the **same Linux** as your desktop, run **headless** - no GUI, reached over the network, built
   to keep **long-running services** alive unattended.
2. You get a shell on it with **`ssh user@host`**; the first connection asks you to verify the **host key
   fingerprint**, and the changed-prompt is your signal that commands now run *there*.
3. **Keys beat passwords** for SSH auth - we set that up properly later.
4. Configuration is **plain text under `/etc`**, which is exactly what makes a server scriptable, diffable,
   and reproducible.

Next, the tool that actually starts, stops, and supervises all those long-running services: **systemd**.


---

# Managing Services with systemd

In Phase 1 we said a server's whole job is keeping long-running services alive. This phase is about the thing
that actually does that keeping: **systemd**. If you only learn one server tool deeply, learn this one - it's
how you start, stop, supervise, and debug every service on a modern Linux box, and how you read what they're
saying.

## What systemd actually is

**What it actually is.** systemd is two things wearing one name. First, it's **PID 1** - the very first
process the kernel starts at boot, the ancestor of everything else (you'll see it at the top of `ps` as
`/sbin/init`, which is a symlink to systemd). Second, it's the **service manager**: the program that decides
what should run, starts those things in the right order, restarts them if they crash, and tracks whether
each one is healthy.

```mermaid
flowchart TD
  kernel[kernel boots] --> systemd["systemd (PID 1)<br/>first process; reads units, brings the system up"]
  systemd -->|starts and supervises| sshd["sshd.service (so you can log in)"]
  systemd --> nginx["nginx.service (your web server)"]
  systemd --> pg["postgresql.service (your database)"]
  systemd --> cron["cron.service (scheduled jobs)"]
```
*Each one is a "unit" systemd keeps watch over.*

**Why people get this wrong.** Coming from running programs by hand, the temptation is to `cd` somewhere and
launch the binary yourself - `./nginx &` or similar. That program dies when your SSH session ends (the Phase
1 war story), nothing restarts it if it crashes, and nothing starts it after a reboot. systemd exists to
take that off you: you *describe* what should run, and it babysits it forever, whether or not you're logged
in.

📝 **Terminology.** A *unit* is anything systemd manages, described by a small text file. The kind you'll
touch most is a **`.service`** unit (a daemon to run). Files live in `/lib/systemd/system/` (shipped by
packages) and `/etc/systemd/system/` (your overrides). You drive it with **`systemctl`**; you read its logs
with **`journalctl`**.

## `systemctl status` - your first move, always

Before you start or stop anything, ask systemd what it already thinks. `status` is read-only and tells you
almost everything you need in one screen:

```console
$ systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; preset: enabled)
     Active: active (running) since Wed 2026-06-18 09:15:22 UTC; 2h 31min ago
       Docs: man:nginx(8)
   Main PID: 8123 (nginx)
      Tasks: 3 (limit: 4915)
     Memory: 8.1M
        CPU: 412ms
     CGroup: /system.slice/nginx.service
             ├─8123 "nginx: master process /usr/sbin/nginx"
             ├─8124 "nginx: worker process"
             └─8125 "nginx: worker process"

Jun 18 09:15:22 web-prod-1 systemd[1]: Starting nginx.service...
Jun 18 09:15:22 web-prod-1 systemd[1]: Started nginx.service.
```

*What just happened:* the green `●` and **`Active: active (running)`** mean it's up right now, and
`since … 2h 31min ago` is its uptime. **`Loaded: … enabled`** is the other half: `enabled` means it's set to
start automatically at boot (`enabled` and `running` are different things, and confusing them bites
everyone). `Main PID: 8123` is the actual process, and the `CGroup` tree shows every process systemd tracks
as part of this service. The last lines are recent log entries - `status` hands you a peek at the journal
for free, often all you need to see what went wrong.

💡 **Key point.** Two independent questions live in that output. **Is it running *now*?** → the `Active:` line.
**Will it come back after a reboot?** → the `Loaded:` line (`enabled` vs `disabled`). A service can be running
but not enabled (works until the next reboot, then vanishes) or enabled but not running (will start next boot,
but isn't up now). Keep them separate in your head.

## start / stop / restart - controlling a running service

These three do exactly what they say, take effect *immediately*, and don't survive a reboot on their own
(that's what `enable` is for). They almost always need `sudo`, because changing what's running on the machine
is privileged.

```console
$ sudo systemctl stop nginx
$ systemctl is-active nginx
inactive
$ sudo systemctl start nginx
$ systemctl is-active nginx
active
```

*What just happened:* `stop` told systemd to shut the service down cleanly (signals the process, waits for
it to exit); `is-active` is a quick, scriptable check that prints one word instead of a whole status screen.
Then `start` brought it back. Neither command printed anything on success - on Unix, silence is success. A
failed `start` won't be silent: it'll point you at `systemctl status` and the journal.

**`restart` is stop-then-start in one step**, and it's what you reach for after editing a service's config:

```console
$ sudo systemctl restart nginx
```

*What just happened:* systemd stopped nginx and started it again, so the running process re-reads its
config. There's a gentler cousin worth knowing:

```console
$ sudo systemctl reload nginx
```

*What just happened:* `reload` asks the service to re-read its config **without** dropping its running
process - for nginx that means no dropped connections. Not every service supports it; when in doubt,
`restart` always works but causes a brief blip. There's also `reload-or-restart` for "reload if you can,
otherwise restart."

⚠️ **Gotcha.** Editing a config file changes nothing by itself. The running process loaded its config once,
at start, and won't notice your edit until you `reload` or `restart` it. "I changed the file and it didn't
take effect" is an enormous source of outages - the edit and the reload are two separate steps.

## enable / disable - surviving a reboot

This is the half that catches people. `enable` and `disable` control **boot behavior**, and *by default they
don't touch the current state at all*:

```console
$ sudo systemctl enable nginx
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /lib/systemd/system/nginx.service.
```

*What just happened:* `enable` created a symlink that wires nginx into the boot sequence - it'll start
automatically every future boot. It did *not* start nginx right now; if it was stopped, it's still stopped.
When you want both at once, say so explicitly:

```console
$ sudo systemctl enable --now nginx
```

*What just happened:* `--now` means "and also do it immediately" - enables nginx for future boots *and*
starts it now, the combination you usually want on a fresh install. (`disable --now` is the mirror image.)

💡 **Key point.** Memorize the pairing: **start/stop = right now**, **enable/disable = at boot**,
**`--now` = both**. Almost every "the service is gone after I rebooted the box" mystery is a service that was
started but never enabled.

## journalctl - reading what a service actually said

systemd captures the standard output and standard error of every service into one central, indexed log
called the **journal**. `journalctl` is how you read it, and where you go the moment something misbehaves.

The most useful invocation: "show me this one service's logs, newest stuff visible":

```console
$ journalctl -u nginx -n 20 --no-pager
Jun 18 09:15:22 web-prod-1 systemd[1]: Starting nginx.service...
Jun 18 09:15:22 web-prod-1 systemd[1]: Started nginx.service.
Jun 18 11:42:07 web-prod-1 nginx[8124]: 2026/06/18 11:42:07 [error] 8124#8124: *15 open() "/var/www/html/favicon.ico" failed (2: No such file or directory)
```

*What just happened:* `-u nginx` filters the journal down to the `nginx` **unit** (without it you'd get logs
from the *entire system* interleaved - occasionally useful, usually overwhelming). `-n 20` limits it to the
last 20 lines, and `--no-pager` dumps it straight to your terminal instead of a scrollable pager. You can
immediately read that nginx is up and that someone requested a missing favicon.

When you're actively debugging - restart the service in one window, watch the logs stream in another - the
indispensable flag is `-f`, **follow**:

```console
$ journalctl -u nginx -f
Jun 18 12:03:55 web-prod-1 nginx[8124]: 2026/06/18 12:03:55 [error] ...
^C
```

*What just happened:* `-f` tails the journal live, printing new lines as they're written, like `tail -f` on
a plain log file. Ctrl-C stops following. This is the single most useful debugging loop on a server: trigger
the thing, watch what the service says about it in real time.

A few more you'll lean on constantly:

```console
$ journalctl -u sshd --since "1 hour ago" -p err
```

*What just happened:* `--since` takes human-friendly time ("1 hour ago", "today", "2026-06-18 09:00"), and
`-p err` filters by **priority** to errors and worse - "SSH daemon errors from the last hour," a great first
query when logins are failing. (`-p` understands the syslog levels: `emerg`, `alert`, `crit`, `err`,
`warning`, `notice`, `info`, `debug`.)

⚠️ **Gotcha.** By default the journal may be stored only in memory and lost on reboot, depending on the
distro - check `Storage=` in `/etc/systemd/journald.conf` (`auto` keeps it on disk only if
`/var/log/journal/` exists; `persistent` always does). If `journalctl --list-boots` shows only the current
boot, make the journal persistent before you need it to investigate a crash that *caused* the reboot.

🪖 **War story.** A service "won't start" and the panic begins. Nine times out of ten the answer is sitting
right there: `systemctl status the-service` shows `failed`, and the last few log lines name the exact reason
- a config syntax error, a port already in use, a missing permission. The discipline: *status first, journal
second, fix third.* You almost never have to guess.

## Recap

1. **systemd** is **PID 1** (the first process) *and* the **service manager** that starts, supervises, and
   restarts your long-running services.
2. A **unit** (usually a `.service`) is a text file describing something systemd manages; you drive it with
   **`systemctl`**.
3. **`systemctl status <svc>`** is your first move - it shows running-or-not (`Active:`), start-at-boot-or-not
   (`Loaded: enabled`), the PID, and recent logs in one view.
4. **start/stop/restart act now**; **enable/disable act at boot**; **`--now` does both**. Don't confuse
   "running" with "enabled."
5. Edit a config, then **`reload`/`restart`** - the edit alone changes nothing.
6. **`journalctl -u <svc>`** reads a service's logs; **`-f`** follows them live, **`-n`** limits lines,
   **`--since`** and **`-p`** filter by time and priority.

Next, the discipline that keeps all of this from becoming a liability: running the box *safely* - users,
`sudo`, scheduled jobs, the firewall, and SSH hardening.


---

# Running It Safely

A server is exposed in a way your laptop never is: it has a public address, it's reachable around the clock,
and automated bots are knocking on its SSH port within minutes of coming online. None of that is cause for
panic - it's cause for a small, boring set of habits. This phase is those habits, each one closing a real
door. We close with a hardening cheat-card you can run down on any new box.

## The server-hardening cheat-card

> **New box, or auditing an old one? Run down this list, then read the section behind anything you're unsure
> of. Each line closes a real door.**

| Door left open | How you close it | § |
|---|---|---|
| Doing everything as `root` | Work as a normal user; reach for `sudo` per-command | §1 |
| Manual chores you'll forget | Schedule them with `cron` | §2 |
| Every port reachable from the internet | Default-deny firewall, allow only what you serve (`ufw`) | §3 |
| SSH password logins (brute-forceable) | Key-based auth; disable password & root login | §4 |
| Known-vulnerable packages | Update regularly; enable unattended security updates | §5 |

---

## 1. Users and `sudo` - least privilege as a habit

**What it actually is.** `root` is the all-powerful account: it can read, change, or destroy anything, and
the kernel does *not* second-guess it. **Least privilege** means not living there - you do day-to-day work
as an ordinary user, and borrow root's power one command at a time with **`sudo`** only when a task
genuinely needs it.

**Why people get this wrong.** It's tempting to `sudo -i` into a root shell and stay there to stop the
password prompts. But a root shell means *every* command - including the typo'd `rm -rf` and the
copy-pasted script you didn't fully read - runs with the power to wreck the machine. `sudo` per-command is
the guardrail: it forces a half-second of intent before each privileged action, and leaves an audit trail.

**A real example.** Reading a protected file fails as a normal user; `sudo` grants exactly that one action:

```console
$ cat /etc/ssh/sshd_config
cat: /etc/ssh/sshd_config: Permission denied
$ sudo cat /etc/ssh/sshd_config
[sudo] password for deploy:
# This is the sshd server system-wide configuration file.
Port 22
...
```

*What just happened:* as `deploy` you weren't allowed to read the SSH config - good, that's the protection
working. Prefixing `sudo` ran *that single command* as root after you proved who you are with **your own**
password (not root's - root may not even have a usable password). The grant lasts only for that command;
the next one is back to being plain `deploy`.

📝 **Terminology.** Membership in the **`sudo`** group (`wheel` on Red Hat–family distros) is what lets a
user run `sudo` at all. Add a user with `sudo usermod -aG sudo alice`, which appends them without disturbing
their other groups.

⚠️ **Gotcha.** That `-aG` is load-bearing. `usermod -G sudo alice` (no `a`) *replaces* all of alice's
supplementary groups with just `sudo`, silently dropping her from every other group. Always `-aG` -
*append*. Group membership also only takes effect on a fresh login, so alice must log out and back in
first.

🪖 **War story.** The cleanest disasters come from a root shell left open in a forgotten terminal tab - hours
later you fat-finger a command into the wrong window, and because it's root, nothing stops it. Living as a
normal user and reaching for `sudo` deliberately is the cheap insurance that turns that into a harmless
"Permission denied."

## 2. Scheduled jobs with `cron`

**What it actually is.** `cron` is the service that runs commands on a schedule - every night at 3am, every
15 minutes, the first of every month. You hand it a line describing *when* and *what*, and it runs that
command unattended, forever, whether or not you're logged in. (Recall the Phase 1 war story about work dying
with your SSH session - this is the right place to put recurring work instead.)

You edit your personal schedule - your **crontab** ("cron table") - with `crontab -e`. Each line is five
time fields followed by the command:

```text
   ┌───────────── minute        (0–59)
   │ ┌─────────── hour          (0–23)
   │ │ ┌───────── day of month  (1–31)
   │ │ │ ┌─────── month         (1–12)
   │ │ │ │ ┌───── day of week    (0–6, Sunday = 0)
   │ │ │ │ │
   * * * * *   command to run
```

A `*` means "every value of this field." So a real backup line reads:

```console
$ crontab -e
# add this line, save, and exit the editor:
0 3 * * *  /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1
```

*What just happened:* `0 3 * * *` is "minute 0 of hour 3, every day of month, every month, every day of week"
- **3:00am daily**. The command runs `backup.sh`, and `>> /var/log/backup.log 2>&1` appends both its normal
output and errors to a log file so you can see afterward whether it worked.

A couple more patterns so the syntax clicks:

```text
   */15 * * * *   →  every 15 minutes
   0 0 * * 0      →  midnight every Sunday
   30 2 1 * *     →  02:30 on the 1st of every month
```

⚠️ **Gotcha.** Cron runs your job with a **minimal environment** - a bare `PATH`, often no `cd` to your home
directory, none of the shell setup from your `.bashrc`. A script that works fine by hand can fail under cron
because it relied on a `PATH` entry or `cd` that isn't there. Defend against it: use **absolute paths** for
every command and file, and *always* redirect output to a log (`>> file 2>&1`) - a cron job that fails
silently at 3am is invisible until the day you needed the backup it never made.

💡 **Key point.** `crontab -e` edits *your* schedule; `crontab -l` lists it; `sudo crontab -e` edits *root's*
separate schedule. System-wide jobs also live in `/etc/cron.d/` and `/etc/cron.daily/`. For jobs that must
survive the machine being off at the scheduled time, systemd **timers** are the more robust modern
alternative - but cron is universal and worth knowing first.

## 3. The firewall with `ufw`

**What it actually is.** A firewall decides which network connections the machine accepts and which it
drops. The only safe default for a server is **deny everything inbound, then allow back exactly the ports
you intend to serve.** `ufw` ("Uncomplicated Firewall") is the friendly front-end that makes this a few
readable commands instead of raw `iptables` rules.

The canonical setup, in order:

```console
$ sudo ufw default deny incoming
Default incoming policy changed to 'deny'
(be sure to update your rules accordingly)
$ sudo ufw default allow outgoing
Default outgoing policy changed to 'allow'
$ sudo ufw allow OpenSSH
Rules updated
Rules updated (v6)
$ sudo ufw allow 'Nginx Full'
Rules updated
Rules updated (v6)
$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
```

*What just happened:* you set the policy to refuse all incoming connections and permit all outgoing (so the
server can still reach out for updates). Then you punched back exactly two holes by **named profile**:
`OpenSSH` (so you don't lock yourself out - see the gotcha below) and `Nginx Full` (ports 80/443). `ufw
enable` turned it on and wired it to start at boot. From here, anything you didn't allow is quietly dropped.

```console
$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW IN    Anywhere
Nginx Full                 ALLOW IN    Anywhere
OpenSSH (v6)               ALLOW IN    Anywhere (v6)
Nginx Full (v6)            ALLOW IN    Anywhere (v6)
```

*What just happened:* `status verbose` confirms the firewall is active, restates the default policies, and
lists every allow rule for IPv4 and IPv6. Check this screen after every change.

⚠️ **Gotcha - the one that locks you out.** If you `sudo ufw enable` over SSH *before* allowing SSH, the
default-deny policy cuts your own connection with no way back in (short of a cloud console). Always
`sudo ufw allow OpenSSH` (or `allow 22`) **first**, confirm it's in `ufw status`, *then* enable. This has
stranded countless people on day one.

## 4. Hardening SSH

SSH is your front door, and the most-probed service on the box. Three changes, made in
`/etc/ssh/sshd_config`, dramatically shrink the attack surface - and the order matters, because two of them
can lock you out if done carelessly.

**1) Use key-based authentication.** As flagged in Phase 1, a private key on your laptop paired with its
public half on the server is both more convenient and far stronger than any password a human will choose.
Get keys working and confirm you can log in with them **before** you touch anything else.

**2) Then turn off password authentication.** Once keys work, passwords are pure liability - they're what
the brute-force bots hammer. Disable them:

```console
$ sudo nano /etc/ssh/sshd_config
# set (uncommenting if needed):
PasswordAuthentication no
PermitRootLogin no
```

*What just happened:* `PasswordAuthentication no` tells `sshd` to refuse password logins entirely, so
password-guessing attacks become pointless. `PermitRootLogin no` blocks logging in *directly* as root, so an
attacker must compromise a known username *and* its key *and* then escalate.

**3) Apply and verify - without dropping yourself.** Changing the config does nothing until you restart the
service (Phase 2's lesson). Restart with care:

```console
$ sudo sshd -t
$ sudo systemctl restart ssh
```

*What just happened:* `sshd -t` does a **config syntax test** - if you typo'd a directive, it tells you now,
before the restart, instead of leaving you with a daemon that won't start and a door you can't open. Only
after it passes silently do you `restart`.

⚠️ **Gotcha - keep a lifeline open.** After restarting SSH, do **not** close your current session. Open a
*second*, brand-new SSH connection and confirm you can still log in with your key. If something is
misconfigured, your existing session is your only way in to fix it. Only once the new connection succeeds
is the change done.

📝 **Terminology.** On some distros the SSH service unit is `ssh`, on others `sshd`; `systemctl status ssh`
(or `sshd`) tells you which name your box uses.

## 5. Keeping packages updated

**What it actually is.** The single highest-leverage security habit is also the most boring: **install
security updates promptly.** Most real-world server compromises exploit a *known* vulnerability with a patch
already available - the attacker is betting you didn't apply it.

On a Debian/Ubuntu box, the routine is two commands:

```console
$ sudo apt update
Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
...
12 packages can be upgraded. Run 'apt list --upgradable' to see them.
$ sudo apt upgrade
Reading package lists... Done
...
The following packages will be upgraded:
  libssl3 openssl ...
After this operation, 1,024 kB of additional disk space will be used.
Do you want to continue? [Y/n]
```

*What just happened:* `apt update` refreshes the local catalog of *what versions are available* - it
installs nothing, just learns what's out there (the "12 packages can be upgraded" line is the news). Then
`apt upgrade` downloads and installs the newer versions, after showing you the list and asking for
confirmation. The two-step exists so you always see what's about to change before it changes.

💡 **Key point - automate the security-critical part.** Applying updates by hand is fine until the week you
forget. On Debian/Ubuntu, `unattended-upgrades` installs *security* updates automatically in the
background:

```console
$ sudo apt install unattended-upgrades
$ sudo dpkg-reconfigure --priority=low unattended-upgrades
```

*What just happened:* you installed the tool and ran its config dialog to switch on automatic security
upgrades. From now on the box patches known security holes on its own, closing the gap between "a fix
exists" and "the fix is applied" without depending on you remembering.

⚠️ **Gotcha.** Some updates (a new kernel, a core library) only take full effect after a **reboot** or
service restart - the machine will tell you, often via a `*** System restart required ***` message at login
or a file at `/var/run/reboot-required`. Automatic *installation* doesn't automatically *reboot*; plan a
window for the ones that need it.

## Recap

1. **Least privilege:** work as a normal user, borrow root per-command with `sudo`; add sudo access with
   `usermod -aG sudo` (always `-aG`).
2. **`cron`** runs commands on a schedule - five time fields plus a command; use **absolute paths** and
   **redirect output to a log**, because cron's environment is bare and silent failures are invisible.
3. **`ufw`:** default-deny incoming, allow only what you serve - and **allow SSH before you `enable`**, or you
   lock yourself out.
4. **Harden SSH** in `/etc/ssh/sshd_config`: keys first, then `PasswordAuthentication no` and
   `PermitRootLogin no`; `sshd -t` before restart, and keep a second session open to verify.
5. **Update regularly** (`apt update` then `apt upgrade`) and let **`unattended-upgrades`** handle security
   patches automatically; reboot when a kernel/library update asks for it.

That's the full arc: you understand the server posture, you can manage and debug its services with systemd,
and you can run it without leaving doors open. From here, the infrastructure track is where this same
knowledge gets automated across many machines - but it all rests on being able to do it by hand, calmly, on
one box. Which you now can.

## Try it yourself

Inspect a fake server filesystem - `ls -l`, `cat /etc/hostname`, `cat readme.txt | grep shell`, `tail -n 2 projects/todo.txt`:

```playground-terminal
```