The Missing Manual
⌘K
All topics / PyTorch From Zero

PyTorch From Zero

Learn the deep-learning framework that runs modern AI: tensors and the GPU, autograd (automatic differentiation), building models with nn.Module, loss functions and optimizers, the training loop, Dataset/DataLoader, training a real classifier, saving and inference, performance pitfalls, and where to go. The three ideas under all of it — tensors, autograd, the loop — made plain.

  1. What PyTorch Is & Tensors The mental model that makes PyTorch click: a tensor is a NumPy array with two superpowers — it runs on a GPU and it tracks gradients. Meet tensors, how to make them, and shape, dtype, device.
  2. Tensor Operations & the GPU Do real math on tensors: elementwise ops, matrix multiply (the heart of neural nets), broadcasting, reshaping and indexing, then move the whole thing onto a GPU with device-agnostic code.
  3. Autograd: Automatic Differentiation The conceptual heart of PyTorch: how requires_grad records your math into a computation graph and .backward() walks it in reverse to compute every gradient automatically — the engine that makes training possible.
  4. Building Models with nn.Module A model is a Python class: subclass nn.Module, define layers in __init__ and the forward pass in forward(). Covers nn.Linear, activations, nn.Sequential, and how parameters are tracked.
  5. Loss Functions & Optimizers How a model knows it's wrong and how it fixes itself: loss functions turn predictions vs. truth into one number, and optimizers (SGD, Adam) use the gradients to nudge the weights.
  6. The Training Loop The five-line ritual that trains every PyTorch model: forward, loss, zero_grad, backward, step, repeated over epochs and batches — why each line is there and what breaks without it.
  7. Data: Dataset & DataLoader How PyTorch feeds data to the training loop: Dataset knows how to fetch one sample, DataLoader batches, shuffles, and parallel-loads them. The plumbing that turns raw data into the batches your loop consumes.
  8. Training a Real Classifier The payoff: assemble tensors, an nn.Module, a loss, an optimizer, the training loop, and DataLoaders into a working MNIST digit classifier — train it, evaluate it honestly, and read the results.
  9. Saving, Loading & Inference Save a trained model's state_dict, recreate the architecture and load it back, then run predictions correctly with eval() and no_grad() — turning raw logits into a labeled answer with confidence.
  10. GPUs, Performance & Common Pitfalls A field guide to the PyTorch bugs that waste days — device mismatches, CUDA out-of-memory, silent training failures — plus how to keep the GPU fed and debug like a pro.
  11. Where to Go Next You can train and save a real classifier. Now skip training from scratch — fine-tune pretrained models, meet the ecosystem (Hugging Face, Lightning), see how LLMs are just PyTorch, and pick what to build.