# Webhooks & Message Queues - Events and Async Integration Beyond Request/Response

> How services talk to each other when one of them isn't waiting for an answer: webhooks let another system call your URL when something happens, and message queues let your own services hand off work to be done later.


---

# Webhooks & Message Queues

You already know how to call an API: you send a request, you wait, you get a response back. That model
is everywhere, and it works - until the thing you care about hasn't happened *yet*. The payment will
clear sometime in the next minute. The video will finish encoding eventually. Ten thousand orders just
landed in the same second and you can't process them all right now. The request/response model has no
good answer for "later," and that's the gap this guide fills.

There are two tools for "later," and people constantly confuse them. A **webhook** is how *another
company's* system tells *yours* that something happened, by calling a URL you gave them. A **message
queue** is how *your own* services hand work to each other without waiting around. Both are about events
and asynchronous work, but they solve different problems, and reaching for the wrong one makes a mess. By
the end you'll know which is which, how each actually works under the hood, and the handful of gotchas
(signatures, duplicates, retries) that bite everyone the first time.

## How to read this
- **Need to wire up a webhook *right now*?** Jump to [Phase 1: Push vs Pull](01-push-vs-pull-webhooks.md) - it's a complete walkthrough of registering a URL and verifying the events are real.
- **Want it to finally make sense?** Read in order. Phase 1 and 2 build the two mental models, and Phase 3 puts them side by side and names the traps.

## The phases
1. **[Push vs Pull: Webhooks](01-push-vs-pull-webhooks.md)** - polling wastes effort; a webhook flips it so the other service calls *your* URL when something happens. How to register, what a delivery looks like, and how to verify it's genuine.
2. **[Message Queues](02-message-queues.md)** - a to-do list between your own services. A producer drops a message, a consumer picks it up when ready. Decoupling, absorbing spikes, and surviving outages.
3. **[When to Use Which (and the Gotchas)](03-when-to-use-which.md)** - webhooks for cross-system notifications, queues for internal async work. Delivery guarantees, why duplicates happen, idempotency, retries, ordering, and dead-letter queues - at a gentle level.

> This guide stops at the mental models and the everyday traps. The deep operational material - 
> partitioning for throughput, exactly-once semantics, choosing a specific broker, and event-sourcing
> as an architecture - is deferred to a follow-up guide so this one stays a thing you can read in a
> sitting.

Related guides: [REST APIs, Explained](/guides/rest-apis-explained) · [Designing APIs That Last](/guides/designing-apis-that-last)


---

# Push vs Pull: Webhooks

You integrated with a payment provider. A customer pays, but the money doesn't land instantly - it
clears a few seconds (sometimes minutes) later. You need to know the moment it clears so you can ship
the order. So you do the obvious thing: every few seconds, you ask. "Did it clear yet?" "No." "Did it
clear yet?" "No." "Did it clear yet?" "Yes."

That's **polling**, and it works, but it's the integration equivalent of a kid in the back seat asking
"are we there yet?" You make a thousand calls to hear "no" nine hundred and ninety-nine times. A
webhook flips the whole thing around: instead of you asking, *they* tell you. This phase is about that
flip - what it really is, how to wire it up, and the one security step that everyone skips and later
regrets.

## Polling: the model you already have

Polling is request/response on a loop. You repeatedly call an API endpoint to check whether some state
has changed yet. The "push" is really still a "pull" - you're pulling status, over and over, hoping this
is the time the answer is different.

Most of your calls return nothing new. You're paying for the network round-trips, the other side is
paying to answer "nothing changed," and there's always lag: if you poll every 30 seconds, you can be up
to 30 seconds late noticing the thing you cared about.

```console
$ curl https://api.payments.example/v1/charges/ch_8Hk2
{"id":"ch_8Hk2","status":"pending"}

$ curl https://api.payments.example/v1/charges/ch_8Hk2
{"id":"ch_8Hk2","status":"pending"}

$ curl https://api.payments.example/v1/charges/ch_8Hk2
{"id":"ch_8Hk2","status":"succeeded"}
```
You asked the same question three times. The first two calls were pure waste - the charge hadn't changed.
Only the third told you anything new, and you still don't know how long it had already been `succeeded`
before you happened to ask.

⚠️ **The polling tax.** Polling tightens into a bad corner: poll *often* and you drown both sides in
useless traffic and may hit rate limits; poll *rarely* and you're slow to react. There's no setting
that's both cheap and fast. That tension is exactly what webhooks remove.

## The webhook: they call you

A webhook is a reversed API call. Instead of *you* calling *them*, you give them a URL, and *they* send
an HTTP POST to that URL whenever a specific event happens. Your server stops asking and starts
listening.

📝 **Terminology.** A *webhook* is sometimes called a "reverse API," an "HTTP callback," or a "web
callback." It's the same idea each time: an event happens on their side, so they make an HTTP request
to an endpoint on your side. The *event* is the thing that happened (payment succeeded, PR opened,
email bounced); the *delivery* is the individual HTTP POST that tells you about it.

**The flip, in one picture.**
```mermaid
sequenceDiagram
  participant You
  participant Them
  Note over You,Them: POLLING - you pull, over and over
  loop until the answer finally changes
    You->>Them: "any news?"
    Them-->>You: "no"
  end
  You->>Them: "any news?"
  Them-->>You: "yes!"
  Note over You,Them: WEBHOOK - you register a URL once, then they push
  Them->>You: "it happened!" (one POST, only when there's something to say)
```

**How it works, step by step.**
1. You register a URL with the provider - usually in their dashboard or via an API call - and pick
   which events you care about (`charge.succeeded`, `pull_request.opened`, and so on).
2. The event happens on their side.
3. Their server sends an HTTP POST to your URL, with the event details in the request body (almost
   always JSON).
4. Your endpoint does its work and replies with a `2xx` status code to say "got it."

Here's what a real webhook POST looks like arriving at your server - the raw HTTP request *they* send
*you*:
```console
POST /webhooks/payments HTTP/1.1
Host: yourapp.example
Content-Type: application/json
Webhook-Signature: t=1718800000,v1=5257a869e7...

{
  "id": "evt_9aB",
  "type": "charge.succeeded",
  "data": { "id": "ch_8Hk2", "amount": 4200, "currency": "usd" }
}
```
The payment provider made an HTTP request *to you* the instant the charge cleared. The body tells you
exactly what happened and to which charge. You didn't ask - you got told, with no lag and no wasted "are
we there yet" calls. Your job now is to read `type`, do the right thing (ship the order), and return a
`2xx`.

**Replying correctly.** The status code you send back is a signal, not a formality. A `2xx` means
"received, you're done." Anything else (a `500`, a timeout) tells the sender you *didn't* get it - and
most providers will then **retry** the delivery later. That retry behavior is a feature, but it has
sharp edges we'll cover in [Phase 3](03-when-to-use-which.md).

⚠️ **Return `2xx` fast, then do the heavy work.** Don't run a 30-second job before replying. If your
handler is slow, the sender may give up waiting, decide the delivery failed, and retry - and now you're
processing the same event twice. The common pattern: validate the request, drop the work onto an
internal queue (that's Phase 2), return `200` immediately, and let a worker do the slow part.

## Anyone can POST to your URL

Here's the part people skip, and it's the most important paragraph in this phase. Your webhook URL is
just a public HTTP endpoint sitting on the internet. The provider can POST to it - but so can anyone
else who learns the address. Nothing about receiving a POST proves it came from who you think.

🪖 **War story.** A team once wired up an "order paid" webhook and shipped goods the moment it fired.
The URL leaked. Someone POSTed a hand-crafted `charge.succeeded` body to it, and the system happily
shipped a real product for a payment that never happened. The fix was already sitting in the docs they
hadn't read: verify the signature.

**Signature verification.** Reputable providers sign every delivery. They take the raw request body,
combine it with a **shared secret** that only you and they know, run it through a hashing function
(typically HMAC-SHA256), and put the result in a header. You repeat the exact same computation on your
side and check that your result matches theirs. If it matches, the request genuinely came from someone
holding the secret - the provider - and the body wasn't tampered with in transit.

📝 **Terminology.** *HMAC* (Hash-based Message Authentication Code) is a way to produce a fingerprint of
some data *using a secret key*. Without the key you can't produce the right fingerprint, and you can't
forge the data without changing the fingerprint. The provider gives you the secret once (you store it
like a password); it never travels in the request itself.

Verifying in Node.js:
```javascript
const crypto = require("crypto");

// rawBody is the EXACT bytes of the request body, before any JSON parsing.
function isGenuine(rawBody, signatureHeader, secret) {
  const expected = crypto
    .createHmac("sha256", secret)
    .update(rawBody)
    .digest("hex");

  // Constant-time compare so attackers can't guess the signature byte by byte.
  return crypto.timingSafeEqual(
    Buffer.from(expected),
    Buffer.from(signatureHeader)
  );
}
```
You recomputed the fingerprint of the body using the secret only you and the provider share. If your
`expected` value equals the signature they sent, the delivery is authentic. If it doesn't, you reject it
 - return a `400` and do nothing. The forged `charge.succeeded` from the war story fails this check,
because the attacker didn't have the secret.

⚠️ **Verify against the raw body, not the parsed object.** The signature is computed over the exact
bytes that were sent. If your web framework parses the JSON and re-serializes it before you check the
signature, even a difference in spacing or key order will make the fingerprint mismatch and reject a
legitimate request. Capture the raw body *first*, verify, *then* parse. This trips up almost everyone
once.

💡 **Key point.** A webhook endpoint without signature verification is an open door. Treat verification
as part of "wiring up the webhook," not an optional hardening step you'll get to later.

## Recap

1. **Polling** is request/response on a loop - you keep asking "any news?" and mostly hear "no." It's
   either expensive or slow, never both cheap and fast.
2. **A webhook** flips it: you register a URL once, and the other service POSTs to it the moment an
   event happens. No asking, no lag.
3. The delivery is **an HTTP POST with the event in the body**; you reply `2xx` to confirm receipt, and
   you reply *fast* - offload slow work so the sender doesn't time out and retry.
4. **Anyone can POST to a public URL**, so you must **verify the signature**: recompute the HMAC of the
   raw body with the shared secret and check it matches. No match, no action.

That's how another company tells *you* something happened. Next, the mirror image: how your *own*
services hand work to each other without anyone waiting around.

Watch it animated: [webhooks](/explainers/Webhooks.dc.html)


---

# Message Queues

A user signs up on your site. Behind that one click, you want to: save the account, send a welcome
email, generate a thumbnail for their avatar, and notify your analytics. If you do all of that *before*
showing them "Welcome!", they're staring at a spinner while four things happen in a row - and if the
email service is having a bad day, the whole signup *fails* over a welcome email that nobody needed
immediately.

The fix is to stop doing everything in line. Save the account, then drop little notes that say "send
this person a welcome email" and "make a thumbnail" onto a list, and tell the user "Welcome!" right
away. Something else picks up those notes and does the work in its own time. That list is a **message
queue**, and once you have the mental model, a huge category of architecture problems gets simple.

## The mental model: a to-do list between services

A message queue is a buffer that sits *between* two pieces of your system. One side writes short
messages into it; the other side reads them out and acts on them. Think of the shared ticket spike in a
diner kitchen: the server clips an order onto the rail and walks away; the cook pulls tickets off and
cooks them at the cook's pace. The server never waits for the food, and the cook never deals with the
customer.

📝 **Terminology.** The thing that *writes* messages is the **producer** (or publisher). The thing that
*reads and acts on* them is the **consumer** (or worker / subscriber). The **message** is a small,
self-contained description of work to do or something that happened - usually a little JSON blob, not the
actual heavy lifting. The software that runs the queue (RabbitMQ, Amazon SQS, and others) is the
**broker**.

**The picture.**
```mermaid
flowchart LR
  P1[signup service] -->|drop| Q
  P2[upload service] -->|drop| Q
  Q["the queue (FIFO: newest in, oldest out)"] -->|pull| W1[worker A]
  Q -->|pull| W2[worker B]
  Q -->|pull| W3[worker C]
```
*Reading the diagram:* producers drop messages in on the left and immediately move on. Messages wait in
line. Consumers pull them off the front when they have capacity. Nobody on the left waits for anybody on
the right.

The common wrong picture is "a queue is just a fancy way to call another service." It isn't. A direct
call couples the two services in time: the caller waits, and if the callee is down, the call fails *now*.
A queue deliberately *breaks* that coupling - the producer's job is done the moment the message is safely
in the queue, no matter what the consumer is doing. That decoupling is the entire point, and it buys you
three specific things.

## What it buys you #1: decoupling

The producer and consumer don't have to know about each other, don't have to be written in the same
language, and don't have to be running at the same time. The producer only knows "the queue." You can
rewrite, redeploy, or scale the consumer without touching the producer at all.

The producer's side:
```console
$ # The signup service just saved the account. Now it drops a job and returns.
$ enqueue --queue emails '{"task":"welcome_email","user_id":4821}'
enqueued message id=msg_5f3 to "emails" (queue depth: 1)
```
The signup service handed off "send a welcome email" in a fraction of a second and is now free to return
"Welcome!" to the user. It did not open SMTP connections, did not wait for the mail provider, and does
not care whether the email worker is even running right now. Its responsibility ended at "message is in
the queue."

When the email service breaks, signups keep working - the messages quietly wait. When you want to move
email sending to a different team's service, the signup code doesn't change. The queue is a stable
contract in the middle, and stable contracts are what let big systems evolve without everything breaking
at once.

## What it buys you #2: load-leveling (absorbing spikes)

Traffic is bursty. You might get ten thousand uploads in one minute during a launch and almost nothing an
hour later. If each upload directly triggered heavy processing, that spike would overwhelm your workers
(or your database) all at once. A queue acts as a shock absorber: the spike fills the queue quickly, and
the consumers drain it at a steady, survivable rate.

📝 **Terminology.** *Queue depth* (or backlog) is how many messages are waiting. It rising during a
spike is normal and healthy - it means the queue is doing its job, holding work so your workers aren't
crushed. It rising *and never coming back down* is the warning sign (more on that in
[Phase 3](03-when-to-use-which.md)).

**The picture.**
```text
   bursty arrivals                steady processing

   ▇▇▇▇▇▇▇▇▇▇  ──►  [ queue absorbs the burst ]  ──►  ▇ ▇ ▇ ▇ ▇ ▇
   10k in a minute       depth rises, then falls       workers chew through
                                                        at a safe, even pace
```

"We went viral and the site fell over" is often really "a spike hit a component that could only handle a
trickle." Putting a queue in front of the slow component turns a crash into a temporary backlog that
clears itself. You trade a little latency under load (work gets done a bit later) for not falling over
(work gets done *at all*). For most background work, that's a trade you'll happily take.

## What it buys you #3: resilience

Because messages sit in the queue until a consumer successfully handles them, work survives a consumer
being down. Deploy a new version of the worker, let the old one crash, scale to zero overnight - when a
consumer comes back, the waiting messages are still there, and it picks up where things left off.

The consumer side, including a crash:
```console
$ worker start --queue emails
[worker] received msg_5f3 (welcome_email, user 4821)
[worker] sending email... DONE
[worker] acknowledged msg_5f3  ← removed from queue only after success

$ worker start --queue emails
[worker] received msg_5f3 (welcome_email, user 4821)
[worker] sending email...
[worker] CRASHED (no acknowledgment sent)
$ # msg_5f3 was NOT acknowledged, so the broker puts it back for another worker.
```
In the first run the worker finished and **acknowledged** the message, so the broker dropped it from the
queue. In the second run the worker died mid-task and never acknowledged. Because the broker only removes
a message once it's been acknowledged, it considers the work unfinished and makes the message available
again. The job isn't lost - another worker (or the same one after restart) will get it.

📝 **Terminology.** *Acknowledging* (often "ack") a message is the consumer telling the broker "I
finished this, you can delete it." Until that ack, the broker assumes the work might not have happened
and keeps the message safe. This is the mechanism that makes "work survives a crash" actually true.

Without a queue, a worker crashing mid-job usually means that job is gone and someone files a "I never got
my email" ticket. With a queue, a crash is a non-event - the message quietly comes back and gets done.
Your deploys get less scary, because in-flight work isn't tied to the process you're about to restart.

⚠️ **The flip side of "comes back again."** That same redelivery - "if it wasn't acknowledged, do it
again" - means a message can be delivered *more than once*: a worker might finish the real work and then
crash *right before* sending the ack. The broker, seeing no ack, hands the message out again, and now
the work runs twice. This is the single most important gotcha with queues, and it's exactly where the
next phase begins.

## Recap

1. **A message queue is a to-do list between services** - a producer drops a message and moves on; a
   consumer pulls it off and does the work when it's ready.
2. **Decoupling:** the producer's job ends when the message is in the queue. The two sides don't share a
   language, a runtime, or even uptime.
3. **Load-leveling:** the queue absorbs bursts so a steady stream of workers can drain them without
   being crushed - a backlog instead of a crash.
4. **Resilience:** messages stay until **acknowledged**, so an unfinished job comes back rather than
   vanishing when a consumer dies.
5. That redelivery is a gift *and* a trap: the same message can arrive more than once. Hold that thought.

Watch it animated: [message queues](/explainers/MessageQueues.dc.html)


---

# When to Use Which (and the Gotchas)

You now have both tools. The danger is using them interchangeably, because they overlap enough to be
confusing and differ enough to bite you. This phase draws the line, then walks through the four traps
that catch everyone - duplicates, retries, ordering, and the message that just won't process. If you're
here mid-incident, start with the cheat-card.

## Cheat-card: symptom → calm fix

| Symptom | What's likely happening | Calm fix |
|---|---|---|
| "We charged/emailed the customer twice" | The same event was delivered more than once (at-least-once) | Make the handler **idempotent** - see below |
| "The webhook sender keeps re-sending the same event" | Your endpoint isn't returning `2xx` fast enough, or at all | Return `2xx` immediately; do slow work in a queue |
| "Messages process out of order" | Queues don't guarantee strict global order by default | Don't rely on order; use a key/version in the message |
| "One bad message is stuck and blocks the rest" | A poison message keeps failing and retrying | Route it to a **dead-letter queue** after N tries |
| "The backlog keeps growing and never drains" | Consumers are too slow or too few for the arrival rate | Add consumers, or speed up the work; check for a crash loop |

Now the explanations underneath.

## Which one? Webhooks vs queues

**The one-line rule:** use a **webhook** when *another system* needs to tell *yours* that something
happened; use a **queue** when *your own* services need to hand work to each other.

A webhook crosses an organizational boundary - it's how Stripe, GitHub, or your email provider reaches
into your system from the outside. A queue lives *inside* your system - it's how your signup service
hands work to your email worker. They even pair up: a very common, very healthy pattern is to *receive a
webhook and immediately drop its payload onto an internal queue*, so your endpoint returns `2xx` in
milliseconds and your own workers do the real processing on their own schedule.

```mermaid
flowchart LR
  subgraph Outside["outside world (they tell you)"]
    S[Stripe / GitHub / email provider]
  end
  subgraph Yours["your system (you pass work around internally)"]
    EP["/webhooks (verify, ack fast)"] -->|drop| Q[queue]
    Q --> W[workers: slow work, retries, etc.]
  end
  S -->|webhook POST| EP
```

**A clear comparison.** Neither is "better"; they answer different questions.

| | Webhook | Message queue |
|---|---|---|
| Direction | An outside system → you | Your service → your service |
| You control both ends? | No (they own the sender) | Yes (you own producer and consumer) |
| Main job | Notify you of an external event | Decouple, absorb spikes, survive outages |
| If the receiver is down | Sender retries on *its* schedule (or gives up) | Messages wait safely until a consumer returns |
| Delivery guarantee | Typically at-least-once | Typically at-least-once |

That last row is the same for both, and it's the source of the biggest gotcha - so let's take it head on.

## Gotcha 1: at-least-once means duplicates

Both webhooks and queues almost always promise **at-least-once delivery**: they guarantee a message will
arrive *at least* one time, and accept that it might arrive *more* than once. They choose this on
purpose. The alternative - *at-most-once* - risks losing messages entirely, which is usually worse. So
the systems lean toward "send it again if we're not sure it got through," and the price is the occasional
duplicate.

📝 **Terminology.** *At-least-once* = never lost, possibly repeated. *At-most-once* = never repeated,
possibly lost. *Exactly-once* = the holy grail (never lost, never repeated) and genuinely hard to achieve
end-to-end - which is why most real systems give you at-least-once and ask you to handle duplicates
yourself.

**Why it happens.** You saw the mechanism in Phase 2: a worker finishes the real work, then crashes
*before* sending its acknowledgment. The broker, having heard no ack, hands the message out again.
Webhooks have the twin of this: your endpoint does the work but is slow to reply, the sender times out
and assumes failure, and re-delivers. In both cases the work was actually done - the *confirmation* was
what got lost.

⚠️ **This is not rare.** It's tempting to treat duplicates as a freak event you can ignore. Don't. Over
enough volume, duplicate deliveries *will* happen, and "we billed them twice" or "we sent three welcome
emails" is a real, embarrassing bug. Design for it from the start.

## The fix: idempotency

An operation is **idempotent** if doing it twice has the same effect as doing it once. "Set the order's
status to `paid`" is idempotent - run it five times, the status is still `paid`. "Add $42 to the balance"
is *not* - run it five times and you've added $210. The whole game is making your handlers idempotent so
a duplicate delivery is harmless.

📝 **Terminology.** *Idempotent* comes from math: applying the operation again doesn't change the result
beyond the first application. In practice it means "safe to retry."

The standard move: every message/event carries a unique ID (the `evt_9aB` you saw in the Phase 1 webhook
body, or a message ID from the queue). Before doing the work, record that ID; if you've already seen it,
skip the work.

```javascript
async function handleEvent(event) {
  // Try to record this event's id. The DB column has a UNIQUE constraint.
  const firstTime = await db.tryInsertProcessedId(event.id);

  if (!firstTime) {
    // We've handled this exact event before - a duplicate delivery. Do nothing.
    return ack();
  }

  await doTheRealWork(event);   // ship the order, send the email, etc.
  return ack();
}
```
The first time `evt_9aB` arrives, the insert succeeds, you do the work, and you ack. When the duplicate
of `evt_9aB` arrives, the insert fails the uniqueness check, so you recognize it as already-handled, skip
the work, and ack anyway. The customer is charged once no matter how many times the event is delivered.

💡 **Key point.** You cannot prevent duplicates at the delivery layer; you neutralize them at the
*handling* layer. "Make the consumer idempotent" is the single most valuable habit in async systems.

## Gotcha 2: retries (the helpful thing that can hurt)

When a delivery fails - your endpoint returned a `500`, or a worker didn't ack - the sender retries. This
is what makes at-least-once work, and it's genuinely good: a brief blip doesn't lose the message.

**The sharp edge.** Retries are usually **exponential backoff**: try again after a few seconds, then
longer, then longer still, often with some randomness so a thousand retries don't all land at the same
instant. Two things to internalize:

- Retries *amplify* the duplicate problem - every retry is another chance for the work to run twice. Your
  idempotency from above is what makes retries safe.
- A handler that fails *permanently* (bad data it will never be able to process) will be retried over and
  over, wasting effort and potentially clogging the queue, until you stop it. Which leads to the last
  two gotchas.

## Gotcha 3: ordering is not guaranteed

By default, you should *not* assume messages are processed in the order they were sent. With multiple
consumers working in parallel, message #2 can finish before message #1. Even a single consumer can
re-process a redelivered message "late." Strict global ordering is expensive, so most queues don't
promise it out of the box.

People write a consumer that assumes "the `created` event always arrives before the `updated` event."
Then one day it doesn't, and `updated` fails because the record doesn't exist yet - or worse, an old
`updated` overwrites a newer one.

Don't depend on arrival order. Instead, make each message carry enough context to stand alone, and use a
version or timestamp to ignore stale updates. "Apply this update only if its version is newer than what I
have" is order-independent and survives duplicates and reordering both. If you *truly* need ordering,
queues offer features for it (like ordered/FIFO modes or partitioning by a key) - but reach for those
deliberately, knowing they cost throughput, rather than assuming order for free.

## Gotcha 4: the poison message → dead-letter queues

Sometimes a message cannot be processed at all - it's malformed, references a deleted record, or trips a
bug. It fails, gets retried, fails again, retried again, forever. Worse, while it sits at the front being
retried, it can hold up everything behind it. That's a **poison message**.

A **dead-letter queue (DLQ)** is a separate queue where messages go after they've failed too many times
(you set the limit, e.g. 5 attempts). Instead of retrying a hopeless message forever, the broker moves it
aside into the DLQ, and the main queue keeps flowing.

```mermaid
flowchart LR
  MQ[main queue] --> C[consumer]
  C -->|fails| R[retry, up to N times]
  R -->|fails again| C
  R -->|after N failures| DLQ["dead-letter queue (parked for a human to inspect later)"]
```

📝 **Terminology.** *Dead-letter queue* (DLQ) - the holding pen for messages that have repeatedly failed.
Nothing reads from it automatically; it's where you go to investigate "what's broken?" without those
messages poisoning live processing.

A DLQ turns "one bad message silently stalled our entire pipeline at 3am" into "the pipeline kept
running, and there are four messages in the DLQ to look at on Monday." You get to debug the failures on
your own time instead of having them take down the healthy traffic with them.

## Recap

1. **Webhook** = an *outside* system tells *yours* something happened. **Queue** = your *own* services
   pass work to each other. They pair beautifully: receive the webhook, ack fast, enqueue the work.
2. Both are **at-least-once**: never lost, possibly duplicated. Plan for duplicates from day one.
3. **Idempotency** is the cure - record each event's unique ID and skip work you've already done, so a
   second delivery is harmless.
4. **Retries** make at-least-once work but amplify duplicates; idempotency is what makes them safe.
5. **Ordering** isn't guaranteed by default - don't depend on it; carry versions/timestamps and ignore
   stale updates.
6. A **dead-letter queue** parks messages that fail too many times, so one poison message can't stall
   everything behind it.

You came in knowing request/response. You leave knowing how systems handle "later" - both when the news
comes from outside (webhooks) and when the work flows inside (queues) - and the handful of traps that
turn a nice async design into a 2am page. That's the whole shape of event-driven integration, named.

Related guides: [REST APIs, Explained](/guides/rest-apis-explained) · [Designing APIs That Last](/guides/designing-apis-that-last)