Using an LLM API in Your App

Calling a hosted language model is a normal HTTP request: you POST a list of messages, you get back generated text - and this guide builds the mental model, the cost picture, and the reliability habits you need to ship it without a foot-gun.

1. It's Just an API Call An LLM API is a normal HTTP request: you POST a list of messages with roles (system, user), and you get back a generated assistant reply - here's the exact request and response shape, annotated.
2. Tokens, Context & Cost A token is a chunk of text; the context window is the model's limited short-term memory that input and output must both fit inside; and you pay per token - so long histories cost more and can overflow. Plus streaming for responsiveness.
3. Building Reliably The model is non-deterministic, can be confidently wrong, can be slow, and can fail - so build for it: control randomness with temperature, verify outputs instead of trusting them, handle timeouts and retries, and ask for structured output like JSON.