Running Models Locally

What it really means to run an LLM on your own machine - the clear-eyed trade-off against a hosted API, a real Ollama session from download to local API call, and how model size, RAM/VRAM, and quantization decide whether it runs at all.

1. Why (and Why Not) Run Locally The clear-eyed trade-off between running an LLM on your own machine and calling a hosted API: privacy, zero per-token cost, offline use, and control - weighed against weaker models, your hardware's limits, and setup effort.
2. Getting One Running (Ollama) The mental model - download an open-weights model and run it locally - then a real Ollama session: pull a model, chat with it in the terminal, and hit its local API endpoint from your own code.
3. Hardware, Quantization & Reality What actually decides whether a model runs: its size in parameters versus your RAM/VRAM, and quantization - shrinking the weights to fit, trading a little quality for a lot of memory. CPU versus GPU speed, and how to match a model to your machine.