Kafka, From Zero
The distributed log everyone runs in production: topics, partitions, offsets, and consumer groups — append-only streams you can replay, not a traditional queue.
- It's a Log, Not a Queue Kafka is a durable, append-only log split into partitions; consumers track their own position with offsets and reading never deletes anything, which is why one stream can feed many readers and be replayed.
- Producing and Consuming for Real The everyday loop: producers pick a partition by key, consumer groups split partitions among members for parallelism, and committing offsets is how you remember what you've processed.
- Production Reality Delivery guarantees and why duplicates happen, building idempotent consumers, surviving rebalances, how retention quietly deletes old data, and when Kafka is the wrong tool.