# Prometheus & Grafana, Explained
> Prometheus scrapes and stores your metrics; Grafana queries and draws them. What each one actually does, how to read a PromQL query, and how to build dashboards and alerts that don't lie to you.
---
# Prometheus & Grafana, Explained
You've seen the dashboards - the wall of green graphs someone set up, the one with a red spike at 3am that everyone points at during the incident review. Maybe you've even been handed a Grafana URL and a vague "the metrics are in there somewhere." But nobody ever sat you down and explained which tool does what, why the graphs show `rate(...)` instead of a plain number, or why one bad label can take the whole thing down.
Here's the short version, and the thing that makes the rest make sense: **these are two tools with two jobs.** Prometheus collects and stores the numbers. Grafana draws pictures of them. Once that division of labor clicks, everything else - the query language, the panels, the alerts - falls into place.
This guide walks you through both, calmly, with the mental model first.
> ๐ **Metrics** are the numbers a system reports about itself - requests served, memory used, errors thrown. If "metrics vs logs vs traces" is fuzzy, start with [Observability: Logs, Metrics & Traces](/guides/observability-logs-metrics-traces) and come back. This guide assumes you know roughly what a metric is.
## How to read this
- **Just need the lay of the land?** Read [Phase 1: What Each One Does](01-what-each-one-does.md) - it's the whole division of labor in one sitting.
- **Want it to finally make sense?** Read in order. Each phase builds on the last: what the tools are, how to read what they store, then how to display and alert on it.
## The phases
1. **[What Each One Does](01-what-each-one-does.md)** - Prometheus is a time-series database that *scrapes* metrics from your services and stores them; Grafana is the dashboard layer that *queries and visualizes* them. The collect-and-store vs display split, drawn out.
2. **[Metrics & a Taste of PromQL](02-metrics-and-promql.md)** - the metric types (counter, gauge, histogram), what labels are for, and how to read a real query - including why you graph a *rate* over a counter instead of the raw counter.
3. **[Dashboards & Alerting](03-dashboards-and-alerting.md)** - building a panel that answers a real question, the RED/USE method for deciding *what* to chart, and getting an alert to fire on a condition - plus the three ways this all goes wrong: dashboards nobody reads, alert fatigue, and cardinality blowups.
> Deep PromQL (joins, subqueries, recording rules) and running Prometheus at scale (federation, remote-write, long-term storage) are deliberately left out - this guide is about understanding the pair well enough to use them, not operating them at scale. If you want the all-in-one commercial alternative, see [Reading Dynatrace](/guides/reading-dynatrace) for how a full APM packages collection, storage, and display into one product.
---
# What Each One Does
The single most common confusion about this pair is thinking they're one thing - "the monitoring system" - or that they're interchangeable. They're not. They're a database and a drawing tool that happen to be used together so often that people say their names in one breath: "promandgrafana."
Untangling them is the whole game. So let's name the jobs.
## The division of labor
**What it actually is.** You have two separate tools, each doing one job:
- **Prometheus** is a **time-series database** with a collector built in. Its job is to go out, *pull* numbers from your services on a schedule, and *store* them with timestamps. It also answers queries about that stored data.
- **Grafana** is a **dashboard tool**. It stores no metrics of its own. Its job is to *ask* a data source (Prometheus, here) for numbers and *draw* them - graphs, gauges, tables, the wall of green.
> ๐ **Time-series database (TSDB):** a database built for data that is "a number, at a time, with some labels," recorded over and over. `http_requests_total{path="/login"} = 4072 at 14:03:10`. It's optimized for "show me this number's shape over the last hour," which a normal SQL table is bad at.
**Why people get this wrong.** Because you only ever *see* Grafana, it's natural to assume Grafana is "the monitoring system" and Prometheus is some plumbing detail. It's the reverse: Prometheus holds the truth (the actual stored numbers), and Grafana is a window onto it. Delete Grafana and your data is fine. Delete Prometheus and Grafana has nothing to draw.
**What it does in real life.** Here's the flow, end to end:
```mermaid
flowchart LR
app["app:8080 /metrics"]
db["db:9100 /metrics"]
prom["Prometheus
scrape + store (TSDB) + query"]
graf["Grafana
dashboards & panels"]
prom -->|scrape every 15s| app
prom -->|scrape every 15s| db
graf -->|PromQL query over HTTP| prom
```
*What just happened:* Prometheus reaches out every scrape interval (15s is the common default) and reads a plain-text page each service publishes at `/metrics`. It stores every number it finds, stamped with the time. Grafana, when you open a dashboard, sends PromQL queries to Prometheus and renders whatever comes back. The arrows point *from* Prometheus *to* your services on purpose - Prometheus pulls; your services don't push.
## Prometheus pulls - it doesn't wait to be told
**What it actually is.** Each service you want to monitor exposes an HTTP endpoint, conventionally `/metrics`, that prints its current numbers as text. Prometheus is configured with a list of these endpoints (its **scrape targets**) and visits each one on a timer.
**What a `/metrics` page looks like.** It's just text. If you `curl` one yourself, you'll see something like:
```console
$ curl localhost:8080/metrics
# HELP http_requests_total Total HTTP requests handled.
# TYPE http_requests_total counter
http_requests_total{method="GET",status="200"} 18342
http_requests_total{method="GET",status="500"} 12
# HELP process_resident_memory_bytes Resident memory in use.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 5.81824e+07
```
*What just happened:* The service printed its live counters and gauges as a snapshot - "right now, I've handled 18,342 successful GETs and 12 failed ones, and I'm using ~58 MB of memory." Prometheus reads exactly this page every 15 seconds and files each line away with a timestamp, building a history out of repeated snapshots. (The `# HELP` and `# TYPE` lines are the metric describing itself - we'll use those types in Phase 2.)
**The gotcha.** Because Prometheus *pulls* on a schedule, it can only see what a service is exposing *at scrape time*. A request that started and finished entirely between two scrapes still gets counted (the counter went up), but a brief spike in a gauge that rises and falls within 15 seconds can be invisible. Pull-based monitoring trades that blind spot for a big simplicity win: Prometheus always knows whether a target is up (the scrape either succeeds or it doesn't), and your services don't need to know where Prometheus is.
๐ก **Key point.** This is a design decision, not an accident. The alternative - services *pushing* metrics out - means every service needs to know the monitoring address and you lose the free "is it up?" signal. Prometheus chose pull. (Push exists too, via a component called the Pushgateway, for short-lived jobs that die before a scrape can reach them - but pull is the default and the norm.)
## Grafana draws - it stores nothing
**What it actually is.** Grafana connects to one or more **data sources** (Prometheus is one of many it supports) and turns query results into panels. A **panel** is one visualization - a graph, a single stat, a table. A **dashboard** is a page full of panels.
**What it does in real life.** When you load a dashboard, Grafana fires off the PromQL queries behind each panel, Prometheus answers with time-stamped numbers, and Grafana plots them. Refresh the page and it queries again. There's no metric data living inside Grafana - it's a live view, re-fetched each time.
**Why this matters.** It explains a lot of "weird" behavior you'll hit:
- A panel showing "No data" usually means the *query* found nothing in Prometheus, or the data source is misconfigured - not that Grafana lost anything.
- You can point Grafana at a different Prometheus and the same dashboard "just works" against the new data, because the dashboard is only queries and layout.
- If Prometheus only keeps 15 days of history (a common retention setting), Grafana can't show you last month - there's nothing to ask for. Retention is Prometheus's concern, not Grafana's.
**The gotcha.** Grafana can connect to many data sources - Prometheus, Loki for logs, a SQL database, cloud-vendor metrics. So "it's in Grafana" tells you almost nothing about *where the data actually lives*. When a panel misbehaves, your first question is always: *which data source is this panel querying, and is the data really there?* Open the panel's query, run it against Prometheus directly, and you'll know in seconds whether the problem is the data or the drawing.
## Why this saves you later
The next time a dashboard shows a flatline or "No data" during an incident, you won't flail. You'll know the data lives in Prometheus and Grafana is just the window - so you check the right thing first: *is Prometheus actually scraping that target?* (Prometheus has its own page listing every target and whether the last scrape succeeded.) Half of "the monitoring is broken" turns out to be one dead scrape target, and you can only see that if you know who's responsible for what.
## Recap
1. **Two tools, two jobs.** Prometheus collects and stores; Grafana queries and displays.
2. **Prometheus pulls.** It scrapes a text `/metrics` page from each target on a timer and stores every number with a timestamp.
3. **Grafana stores nothing.** It's a live window - it asks a data source for numbers and draws them on demand.
4. **The truth lives in Prometheus.** When a graph looks wrong, check whether the data is really there before blaming the picture.
Now that you know *where* the numbers live, let's learn how to read them - the metric types, labels, and your first PromQL query.
---
# Metrics & a Taste of PromQL
PromQL queries look intimidating the first time - `rate(http_requests_total{status="500"}[5m])` is a lot of punctuation to meet at once. But almost every query you'll read is built from a few small ideas stacked together. Learn the ideas and the punctuation stops being scary.
The single most important one, the thing that confuses *everybody* at first: **why you almost never graph a counter directly, and reach for `rate()` instead.**
## The three metric types
Prometheus metrics come in a few types, and the type tells you how to *read* the number. Three cover the vast majority of what you'll see.
**Counter - a number that only ever goes up.** Total requests served, total errors, total bytes sent. A counter starts at zero when the process starts and climbs. It resets to zero only when the process restarts.
- The wrong picture: "the counter is the current value, like a speedometer." It isn't. `http_requests_total = 18342` doesn't mean "18,342 requests per second" - it means "18,342 requests *ever*, since this process started." On its own that number is nearly useless. What you care about is *how fast it's climbing*, which is the whole reason `rate()` exists (next section).
**Gauge - a number that goes up and down.** Current memory in use, current temperature, number of items in a queue, active connections. A gauge is a snapshot of "right now," and reading it directly *does* make sense - `process_resident_memory_bytes = 58000000` means "using 58 MB right now."
**Histogram - counts bucketed by size.** A histogram answers "how were these values distributed?" - most often for request durations. Instead of one number, it records counts in buckets: "how many requests finished under 0.1s, under 0.5s, under 1s, โฆ". This is what lets you ask for a **percentile** later - "95% of requests finished faster than X" - which is how you actually talk about latency. (A single average latency hides the slow tail; percentiles don't.)
> ๐ There's a fourth type, **summary**, a close cousin of histogram that computes percentiles service-side. Histograms are the more common, more flexible choice - treat summary as histogram's relative and don't lose sleep over it yet.
โ ๏ธ **Gotcha - reading a counter like a gauge.** This is the classic beginner mistake. You put `http_requests_total` on a graph, see a line marching steadily up and to the right, and panic that traffic is exploding. It isn't - a counter *always* goes up and to the right, by definition. A straight diagonal line means *steady* traffic. To see the actual traffic, you need its rate of change.
## Labels - the same metric, sliced many ways
**What they actually are.** Labels are key-value tags attached to a metric that let one metric name cover many related streams. Look back at the `/metrics` page from Phase 1:
```text
http_requests_total{method="GET",status="200"} 18342
http_requests_total{method="GET",status="500"} 12
http_requests_total{method="POST",status="200"} 4071
```
*What just happened:* That's *one* metric name, `http_requests_total`, split into three separate time series by its labels. Each unique combination of label values is its own counter. So you can ask "all requests," or narrow to "just the 500s," or "just POSTs" - all from the same metric, by filtering on labels.
**Why this is the heart of PromQL.** Filtering and grouping by labels is how you turn a firehose of numbers into an answer to a specific question. "How many errors?" โ filter to `status="500"`. "Errors per endpoint?" โ group by the `path` label. We'll see both below.
โ ๏ธ **Gotcha (the one that bites hardest, covered fully in Phase 3).** Every distinct combination of label values creates a *new* time series Prometheus must store. Put something with unbounded values in a label - a user ID, a full URL with query strings, a request ID - and you generate millions of series. That's a **cardinality** explosion, and it's the most common way people accidentally take Prometheus down. Labels are for things with a *small, bounded* set of values (method, status code, endpoint name), never for unique identifiers.
## Reading your first real query
Let's take that scary-looking query apart, piece by piece:
```promql
rate(http_requests_total{status="500"}[5m])
```
Reading it from the inside out:
- `http_requests_total` - the metric: total HTTP requests, a **counter**.
- `{status="500"}` - a **label filter**: narrow to just the series where the status code is 500 (server errors).
- `[5m]` - a **range**: "give me the last 5 minutes of samples for each matching series," not just the latest point. (This is called a *range vector* - a window of values, which `rate()` needs to do its math.)
- `rate(...)` - the function: take that 5-minute window and compute the **per-second average rate of increase** of the counter.
*What it's showing you:* "Over the last 5 minutes, how many server errors per second, on average?" The output isn't a total - it's a *speed*: errors/sec, calculated freshly at each point on the graph. A flat line at `0` means no errors; a line that jumps to `3` means you're suddenly taking three 500s every second.
## Why you graph rates, not raw counters
This is the payoff. Here's the contrast, drawn out:
```text
RAW COUNTER: http_requests_total rate(http_requests_total[5m])
("everything ever") ("requests per second, now")
count req/s
โ ____/ โ โฑโฒ ___
โ ____/ โ ___โฑ โฒ____โฑ โฒ__
โ ____/ โ โฑ
โ___/ โโฑ
โโโโโโโโโโโโโโโโโโโโโโโโบ time โโโโโโโโโโโโโโโโโโโโโโโโบ time
always climbs; slope = the the actual traffic shape:
real signal, but hard to read spikes and dips you can see
```
*What just happened:* The raw counter only ever rises, so the *information* you want - how busy the service is right now - is hidden in the line's slope, which your eyes are bad at reading. `rate()` does the differencing for you: it turns "total ever" into "per second now," so a traffic spike becomes a visible bump and a quiet period becomes a dip. That's why almost every counter you ever graph is wrapped in `rate()`.
`rate()` is also smart about counter **resets**: when a process restarts and the counter drops back to zero, `rate()` recognizes that as a restart rather than a giant negative blip, and accounts for it. That's another reason to use it instead of subtracting values by hand.
๐ก **Key point - the rule of thumb.** *Counters get wrapped in `rate()`. Gauges you read directly.* If you find yourself plotting a bare counter and wondering why every line goes up and to the right, that's the rule reminding you. (A close relative, `increase(...)[1h]`, answers "how many total in the last hour?" - same idea, expressed as a count rather than a per-second rate.)
> The numbers and shapes above are illustrative - drawn to show the *behavior*, not measured from a real system.
## Why this saves you later
When you're staring at a dashboard during an incident and someone asks "is the error rate climbing?", you'll know the panel's query is a per-second rate over a window, not a raw total - and trust the line for the right reasons. In the next phase you'll reach for `rate()` on a counter without thinking, instead of plotting a useless diagonal.
## Recap
1. **Type tells you how to read it.** Counters only go up (read their *rate*); gauges go up and down (read them *directly*); histograms bucket values so you can ask for percentiles.
2. **Labels slice one metric into many series** - filter and group by them to ask specific questions.
3. **High-cardinality labels are dangerous** - never put unique IDs in a label (more in Phase 3).
4. **`rate(counter[window])`** turns "total ever" into "per second now," and handles counter resets for you.
5. **The rule:** wrap counters in `rate()`; read gauges raw.
Now you can read what's stored. Let's put it on a screen and make it page someone when it matters.
---
# Dashboards & Alerting
It's tempting to think the goal here is "make a nice dashboard." It isn't. The goal is to *answer questions* and *get woken up only when something is genuinely wrong*. A beautiful dashboard nobody looks at is wasted work; an alert that cries wolf every night gets muted, and then it misses the real fire.
This is less about clicking buttons than judgment: what to chart, what to alert on, and the three ways this setup quietly rots.
## The "what goes wrong" cheat-card
> **Recognize the failure mode, then read the section.**
| Symptom | What's actually happening | Where to look |
|---|---|---|
| 40-panel dashboard nobody opens | Built to *display data*, not *answer a question* | ยง1, ยง2 |
| "I don't know what to put on this dashboard" | No method - charting at random | ยง2 (RED / USE) |
| Team mutes the alerts channel | Alert fatigue - too many, too noisy, not actionable | ยง4 |
| Prometheus slow / OOM / disk full | Cardinality blowup - too many label values | ยง5 |
| "Should we just buy an APM instead?" | Reasonable question - trade-offs are real | ยง6 |
---
## 1. A panel answers one question
**What it actually is.** A good panel starts with a *question*, not a metric. "Are we serving errors to users right now?" is a question. "Let me put `http_requests_total` on a graph" is not - it's just data with no point.
**What it does in real life.** Say the question is "what's our error rate?" You already know how to express that from Phase 2. The panel's query might be:
```promql
sum(rate(http_requests_total{status=~"5.."}[5m]))
/
sum(rate(http_requests_total[5m]))
```
*What it's showing you:* the fraction of requests that are server errors, as a live ratio. The top line is "5xx errors per second" (`status=~"5.."` matches any 5xx code with a regex), the bottom is "all requests per second," and dividing gives you "what share of traffic is failing." A panel like this answers the question directly: `0` is healthy, `0.05` means 5% of requests are failing right now.
๐ก **Key point.** Before you add any panel, finish this sentence: *"This panel exists so I can tell whether ______."* If you can't fill the blank, don't add the panel.
## 2. RED and USE - how to decide *what* to chart
The blank-page problem ("what do I even put here?") has two well-known answers, depending on whether you're looking at a *service* or a *resource*.
**RED - for request-driven services** (an API, a web app). For each service, chart three things:
```text
R โ Rate how many requests per second? rate(requests[5m])
E โ Errors how many of them are failing? rate(requests{status="error"}[5m])
D โ Duration how long are they taking? a latency percentile from a histogram
```
*What this gives you:* with just those three, you can answer "is the service busy, is it healthy, is it fast?" - which is almost everything you want to know about a service from the outside. (RED is widely attributed to Tom Wilkie; USE below to Brendan Gregg. Both are well-documented industry conventions.)
**USE - for resources** (a CPU, a disk, a memory pool, a connection pool). For each resource, chart:
```text
U โ Utilization how busy is it? (% CPU, % memory used)
S โ Saturation how much is it queued / waiting? (run queue, swap)
E โ Errors is it throwing errors? (disk errors, dropped packets)
```
*What this gives you:* a way to find the *constrained* resource when a service is slow - the disk that's 100% utilized, the connection pool that's saturated.
**The gotcha.** Don't apply both to everything. RED is for things that *serve requests*; USE is for things that *get consumed*. Picking the right lens is most of the skill - a service's dashboard is mostly RED, an infrastructure dashboard is mostly USE.
## 3. โ ๏ธ Dashboards nobody reads
**What's actually happening.** The natural failure mode of dashboards is *accretion*. Someone adds a panel during an incident "to see," never removes it, and a year later the dashboard has 40 panels, three of which anyone actually uses. A 40-panel wall isn't more informative than a 5-panel one - it's *less*, because the signal is buried and nobody can hold it in their head.
**The calm fix.** Treat panels like code: if it doesn't earn its place, delete it. A dashboard that answers five real questions clearly beats one that displays fifty metrics nobody reads. When in doubt, ask whose question each panel answers - if nobody's, cut it.
## 4. Alerting - and โ ๏ธ alert fatigue
**What it actually is.** An **alert rule** is a PromQL expression plus a condition and a duration: "if *this* is true for *this long*, fire." Prometheus evaluates the rule on a timer; when it fires, it hands the alert to **Alertmanager**, a separate component whose job is routing - deciding who gets paged, grouping related alerts, and silencing them during maintenance.
```mermaid
flowchart LR
prom["Prometheus
evaluates rule: error rate over 5% for 10m
(is it true, and has it stayed true?)"]
am["Alertmanager
group, dedupe, route, silence
(who should hear, and how?)"]
you["You
page / Slack / email"]
prom -->|fires| am
am --> you
```
*What just happened:* Prometheus decides *whether* something is wrong; Alertmanager decides *what to do about it*. Splitting those two jobs is deliberate - it means one Alertmanager can handle alerts from many Prometheus servers, and you tune *routing* (who, when, how loud) without touching the *rules* (what counts as wrong).
**A real alert rule.** Rules live in Prometheus's config as YAML:
```yaml
groups:
- name: api-health
rules:
- alert: HighErrorRate
expr: |
sum(rate(http_requests_total{status=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m])) > 0.05
for: 10m
labels:
severity: page
annotations:
summary: "5xx error rate above 5% for 10 minutes"
```
*What just happened:* This says "if more than 5% of requests are 5xx errors, *sustained for 10 minutes*, fire a `page`-severity alert." That `for: 10m` is the most important line - it's what stops a one-second blip from paging someone at 3am.
โ ๏ธ **Gotcha - alert fatigue is how monitoring dies.** If alerts fire on things that aren't actionable, or fire so often they're background noise, people mute the channel - and then miss the *real* incident. The fixes are discipline, not technology:
- **Alert on symptoms, not causes.** Page on "users are getting errors" (a symptom) rather than "CPU is at 80%" (a cause that may be totally fine). High CPU that isn't hurting anyone is not an emergency.
- **Every page must be actionable.** If the on-call person can't *do* anything about it at 3am, it shouldn't page - make it a ticket or a dashboard line instead.
- **Use `for:` generously.** Most things that are bad for two seconds and then fine don't need a human.
- **Match severity to urgency.** Reserve paging for "wake someone up" problems; route everything else to a chat channel.
## 5. โ ๏ธ Cardinality blowups - the silent killer
**What's actually happening.** Remember from Phase 2: every unique combination of label values is a *separate* time series Prometheus stores in memory and on disk. **Cardinality** is the count of those distinct series. It's fine when labels have small, bounded value sets (a handful of HTTP methods, a dozen status codes). It becomes catastrophic when a label can take unlimited values.
The classic mistake:
```text
SAFE label (bounded): DANGEROUS label (unbounded):
status="200" | "404" | "500" user_id="a1f3" | "b7c2" | ...millions
โ a few series โ one series per user โ explosion
path="/login" | "/cart" path="/cart?id=8832&ref=email&t=..."
โ tens of series โ one series per unique URL โ explosion
```
*What just happened:* Putting `user_id`, a raw URL with query parameters, a request ID, or an email address in a label means Prometheus creates a brand-new time series for *every distinct value it ever sees*. Memory balloons, queries crawl, and eventually Prometheus runs out of memory and falls over - taking your monitoring down at exactly the wrong moment.
**The calm fix.** Labels are for dimensions with a *small, known, bounded* set of values. Before adding a label, ask: *"How many distinct values can this ever have?"* If the answer is "unbounded" or "one per user/request/URL," it does not belong in a label. Normalize it first - use the *route template* `/cart/:id` instead of the literal URL, drop the query string, bucket the user into a plan tier. When Prometheus gets slow or hungry, cardinality is the first thing to suspect.
## 6. The contrast - Prometheus + Grafana vs an all-in-one APM
You've now seen the cost: you assemble and operate the pieces yourself - Prometheus, Grafana, Alertmanager, exporters, dashboards, retention. The upside is that it's open-source, vendor-neutral, and bends to whatever you need. The trade-off is that *you* are the integrator.
The other option is an all-in-one **APM** (Application Performance Monitoring) product - Dynatrace, Datadog, New Relic - that bundles collection, storage, dashboards, alerting, tracing, and often auto-instrumentation into one paid product. You write far less plumbing; you pay for it (usually per host or per volume) and you're tied to that vendor.
| | Prometheus + Grafana | All-in-one APM |
|---|---|---|
| Cost | Open-source; you pay for the infra and your time | Paid, often per-host or per-data-volume |
| Setup | You wire collection, storage, dashboards, alerting | Mostly turnkey; auto-instrumentation common |
| Control / flexibility | High - vendor-neutral, customize anything | Lower - you live inside the product's model |
| Who operates it | You do | The vendor does the heavy lifting |
| Best when | You want control, no vendor lock-in, and have the time | You want answers fast and will pay to skip the plumbing |
*Neither is "better."* Plenty of teams run both. To see what the all-in-one experience feels like from the inside - how a packaged APM presents the same metrics, traces, and alerts - see [Reading Dynatrace](/guides/reading-dynatrace).
## Recap
1. **A panel answers one question** - start from the question, not the metric.
2. **RED for services** (Rate, Errors, Duration), **USE for resources** (Utilization, Saturation, Errors) - pick the right lens.
3. **Dashboards rot by accretion** - delete panels that no longer earn their place.
4. **Prometheus decides *whether* something is wrong; Alertmanager decides *what to do*** - and `for:` plus "alert on symptoms" are how you avoid alert fatigue.
5. **Cardinality kills** - never put unbounded values (user IDs, raw URLs, request IDs) in a label.
6. **The pair vs an APM** is a control-vs-convenience trade - both are valid.
That's the pair, end to end: Prometheus collects and stores, Grafana displays, PromQL is how you ask, and good dashboards and alerts are a matter of judgment, not just configuration. You can now walk up to someone else's Grafana and actually understand what you're looking at - and build your own that someone will thank you for.