# CircleCI, From Zero

> Cloud-native CI/CD with config.yml: jobs, workflows, executors, and orbs - fast parallel pipelines without running your own server.


---

# CircleCI, From Zero

You pushed a branch, opened a pull request, and now a little status check is spinning in the corner. Someone set that up months ago, it lives in a file called `.circleci/config.yml`, and when it goes red nobody quite knows why. This guide turns that file from a black box into something you can read, edit, and trust.

CircleCI runs your tests and builds in the cloud every time you push. You describe what should happen in one YAML file; CircleCI rents fresh machines, runs your commands, and reports back. No build server to patch, no Jenkins to babysit at 2am. The cost is that you learn one specific way of describing work - and that's exactly what we'll do here.

## How to read this

Read the phases in order; each one builds the mental model the next assumes. If you've never touched a CI system at all, skim [What CI/CD does](/guides/what-cicd-does) first so the *why* is already in place. If you know GitHub Actions, you already understand the shape of the problem - CircleCI names the pieces differently, and we'll call out the mapping as we go.

## The phases

1. [The four nouns: jobs, executors, steps, workflows](01-the-four-nouns.md) - the mental model of how a CircleCI pipeline is actually built.
2. [Writing a real config: caching, orbs, and fan-out](02-writing-a-real-config.md) - how you use it day to day to get fast, readable pipelines.
3. [When it breaks: flaky tests, slow builds, and managed-CI tradeoffs](03-when-it-breaks.md) - the gotchas, parallelism, and the limits of someone else's infrastructure.


---

# The four nouns: jobs, executors, steps, workflows

Open someone's `.circleci/config.yml` cold and it looks like a wall of nested keys. The trick is that almost all of it is built from four nouns. Once you can name them, the wall turns into sentences. Let's meet them in the order they nest.

## A step is a single command

The smallest unit is a **step**. A step is one thing you do: check out the code, run a shell command, restore a cache. Most steps are shell commands.

```yaml
steps:
  - checkout
  - run: npm ci
  - run: npm test
```

*What just happened:* `checkout` is a built-in step that clones your repo into the working directory. The two `run` steps execute shell commands in order - install dependencies, then run tests. If `npm ci` fails (non-zero exit), CircleCI stops and never reaches `npm test`. Steps run top to bottom and the first failure ends the show.

A `run` step can be a one-liner like above, or a named multi-line block when you want it to read clearly in the UI:

```yaml
steps:
  - run:
      name: Run the test suite
      command: npm test
```

*What just happened:* same command, but now the CircleCI dashboard labels this step "Run the test suite" instead of showing the raw command. The `name` is purely for humans reading the build output - worth it for anything non-obvious.

## A job is a list of steps on one machine

A **job** is a named bundle of steps that run together on a single fresh machine. When the job ends, that machine is thrown away. This is the unit CircleCI schedules, reports, and shows you as a green or red dot.

```yaml
jobs:
  build-and-test:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - run: npm ci
      - run: npm test
```

*What just happened:* we defined one job named `build-and-test`. It runs every step on a clean Docker container based on `cimg/node:20.11` (CircleCI's pre-built Node image). The job is the boundary of a workspace: those three steps share the same filesystem and the same machine, but nothing carries over to any other job unless you explicitly pass it.

If you come from GitHub Actions, a CircleCI **job** maps to an Actions **job**, and a CircleCI **step** maps to an Actions **step**. The shapes line up. See [Your first pipeline with GitHub Actions](/guides/your-first-pipeline-github-actions) if that's your reference point.

## An executor is the machine the job runs on

That `docker:` key under the job is the **executor** - your choice of *what kind of machine* the steps run on. This is where CircleCI gives you a real decision, so it's worth understanding the main options.

```yaml
jobs:
  fast-job:
    docker:
      - image: cimg/python:3.12      # runs your steps inside this container
  needs-real-vm:
    machine:
      image: ubuntu-2204:current     # a full Linux VM, not a container
  mac-build:
    macos:
      xcode: "15.3.0"                # a macOS machine for iOS/Mac builds
```

*What just happened:* three jobs, three executor types. The `docker` executor is the default workhorse - fast to start, your commands run inside the named container. The `machine` executor gives you a full virtual machine, which you need when your job itself runs Docker (building images, docker-compose) or needs kernel-level access a container can't give. The `macos` executor is a real Mac, required for anything Apple.

The mental rule: **reach for `docker` first** because it boots fastest. Move up to `machine` only when a container genuinely can't do the job - almost always because you need to build or run Docker images yourself.

> The `cimg/` images (short for "CircleCI image") come pre-loaded with the language plus common build tools, so they start faster than a raw `node` or `python` image where CircleCI has to install extras. Prefer them when one exists for your language.

## A workflow orchestrates the jobs

One job is rarely the whole story. You might want to lint, test, and build - and only deploy if all three pass. A **workflow** is the conductor: it says which jobs run, in what order, and what depends on what.

```yaml
workflows:
  test-then-deploy:
    jobs:
      - lint
      - test
      - build
      - deploy:
          requires:
            - lint
            - test
            - build
```

*What just happened:* `lint`, `test`, and `build` have no `requires`, so they start at the same time - they **fan out** and run in parallel. `deploy` lists all three under `requires`, so CircleCI holds it back until every one of them succeeds. If `test` goes red, `deploy` never runs at all. This dependency graph is the entire point of workflows: cheap parallelism where work is independent, hard gates where order matters.

```mermaid
graph LR
  lint --> deploy
  test --> deploy
  build --> deploy
```

*What just happened:* the three independent jobs fan out, then converge on `deploy`. CircleCI runs anything with no unmet dependency as early as it can, which is why putting independent work in separate jobs makes your pipeline finish sooner.

## How the four nouns stack

Put together, the hierarchy reads cleanly from the bottom up: **steps** live inside a **job**, the job runs on an **executor**, and a **workflow** decides how the jobs relate. Every CircleCI config you'll ever read is some arrangement of these four. The top of the file always declares the config version so CircleCI knows which feature set you're using:

```yaml
version: 2.1
jobs:
  # ... your jobs here
workflows:
  # ... your workflows here
```

*What just happened:* `version: 2.1` is the modern config format - it unlocks orbs, reusable commands, and parameters, which we lean on in the next phase. Always start a new config with it.

In the wild: a healthy repo's config is mostly workflow plumbing plus two or three focused jobs. When you see a single giant job doing everything in sequence, that's usually a config that grew without anyone splitting it - and a slow pipeline as a result, because nothing can run in parallel.

```quiz
[
  {
    "q": "In a CircleCI config, what does the executor (docker/machine/macos) decide?",
    "choices": [
      "The order jobs run in",
      "What kind of machine the job's steps run on",
      "Which branch triggers the build",
      "How many times a step retries on failure"
    ],
    "answer": 1,
    "explain": "The executor is the machine type for a job. Order between jobs is the workflow's responsibility, not the executor's."
  },
  {
    "q": "Three jobs in a workflow have no 'requires' key. What happens?",
    "choices": [
      "They run one after another in file order",
      "Only the first one runs",
      "They all start in parallel (fan-out)",
      "The config is invalid"
    ],
    "answer": 2,
    "explain": "Jobs with no unmet dependency start as early as possible, so jobs with no 'requires' fan out and run at the same time."
  },
  {
    "q": "When should you prefer the 'machine' executor over 'docker'?",
    "choices": [
      "Whenever you want faster startup",
      "When the job itself needs to build or run Docker images",
      "For every Node.js project",
      "Only for macOS builds"
    ],
    "answer": 1,
    "explain": "docker boots fastest and is the default choice. machine gives a full VM, which you need mainly when the job runs Docker itself; macOS builds use the macos executor."
  }
]
```


---

# Writing a real config: caching, orbs, and fan-out

The four-noun config from phase 1 works, but it's slow and verbose. Every run reinstalls every dependency from scratch, and you're hand-writing steps that thousands of teams have already written. This phase is the everyday craft: making the pipeline fast with caching, making it short with orbs, and gating a deploy behind a human.

## Caching: stop reinstalling the world every run

Each job gets a clean machine, which means `node_modules` is empty every single time. On a real project that's minutes of `npm ci` you pay on every push. Caching saves a directory at the end of one run and restores it at the start of the next.

```yaml
jobs:
  test:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - restore_cache:
          keys:
            - deps-v1-{{ checksum "package-lock.json" }}
            - deps-v1-
      - run: npm ci
      - save_cache:
          key: deps-v1-{{ checksum "package-lock.json" }}
          paths:
            - node_modules
      - run: npm test
```

*What just happened:* `restore_cache` looks for a saved cache whose key starts with one of the listed prefixes, newest match wins. The key embeds `checksum "package-lock.json"`, so when your lockfile is unchanged the exact cache is restored and `npm ci` finishes in seconds. When the lockfile changes the checksum changes, the exact key misses, the fallback `deps-v1-` restores the most recent old cache (a warm start), and `save_cache` writes a fresh one under the new key.

Two things bite people here. **Cache keys are immutable** - once a key is written, CircleCI never overwrites it. That's why the checksum is in the key: a new lockfile means a new key. And the `deps-v1-` prefix is a manual version: bump it to `deps-v2-` when you want to throw the whole cache away.

> Caching is an optimization, never a source of truth. Your build must still pass with an empty cache, because that's exactly what happens on a fresh branch or after a key bump. If a green build depends on a warm cache, you have a bug, not a cache.

## Orbs: reusable config packages

An **orb** is a shareable bundle of jobs, commands, and executors that someone already wrote and tested. Instead of hand-coding the Node setup-and-cache dance above, you pull in the official Node orb and call its commands.

```yaml
version: 2.1
orbs:
  node: circleci/node@5.2.0
jobs:
  test:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - node/install-packages       # handles install + caching for you
      - run: npm test
```

*What just happened:* the `orbs` block imports `circleci/node` at version `5.2.0`. That gives us `node/install-packages`, a single command that does the restore-cache, install, and save-cache sequence we wrote by hand above. The config got shorter and harder to get wrong, because the caching logic is maintained by the orb's authors instead of by you.

Orbs are named `namespace/name@version`. Pin the version (don't float to "latest") so a pipeline that's green today doesn't break tomorrow when the orb publishes a change. There are orbs for AWS, Slack notifications, Docker, browser tools, and most things you'd otherwise script - but each one is third-party code running in your pipeline, so prefer the `circleci/` official orbs and read what an unfamiliar orb does before trusting it with credentials.

## Approval gates: putting a human in the loop

Continuous deployment is great until "every merge ships to production" gives someone a heart attack. CircleCI lets you pause a workflow and wait for a person to click a button.

```yaml
workflows:
  build-test-deploy:
    jobs:
      - test
      - build:
          requires:
            - test
      - hold-for-approval:
          type: approval
          requires:
            - build
      - deploy:
          requires:
            - hold-for-approval
```

*What just happened:* `hold-for-approval` has `type: approval`, which is a special job that runs no commands - it pauses the workflow and shows a button in the CircleCI UI. Nothing downstream of it runs until someone with access clicks approve. Because `deploy` requires `hold-for-approval`, your code is built and tested automatically, then waits for a human green light before it ships. That's the standard pattern for "automate everything up to production, then ask."

## Filters: run jobs only on the right branches

You rarely want to deploy from every branch. **Filters** restrict when a job runs based on the branch or tag.

```yaml
workflows:
  build-test-deploy:
    jobs:
      - test
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: main
```

*What just happened:* `test` runs on every branch and every pull request. `deploy` carries a filter saying `branches: only: main`, so it's skipped entirely on feature branches and only fires when the commit is on `main`. This is how one config serves both "check my PR" and "ship the merge" without two separate pipelines.

## Putting it together

A real, readable config for a Node service ends up looking like this - orb for the boring parts, a clear workflow, a filtered deploy:

```yaml
version: 2.1
orbs:
  node: circleci/node@5.2.0
jobs:
  test:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - node/install-packages
      - run: npm test
  deploy:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - node/install-packages
      - run: npm run deploy
workflows:
  ci:
    jobs:
      - test
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: main
```

*What just happened:* every push runs `test`. Only a push to `main` that passes `test` runs `deploy`. The caching is handled by the orb, the dependency graph is one `requires`, and the whole thing fits on a screen. That's the target shape - boring, short, and obvious.

For builders: keep the config as flat as you can. When you feel the urge to add a fifth job or a clever conditional, ask whether an orb already solves it. The best CircleCI configs are the ones a teammate can read in thirty seconds.

```quiz
[
  {
    "q": "Why is checksum \"package-lock.json\" put inside the cache key?",
    "choices": [
      "To make the key shorter",
      "So a changed lockfile produces a new key and avoids restoring a stale cache",
      "To encrypt the cache contents",
      "It is required syntax with no effect"
    ],
    "answer": 1,
    "explain": "Cache keys are immutable. Embedding the lockfile checksum means any dependency change yields a new key, so you never restore node_modules that no longer matches the lockfile."
  },
  {
    "q": "What does a job with 'type: approval' do?",
    "choices": [
      "Runs your deploy commands automatically",
      "Retries the previous job until it passes",
      "Pauses the workflow until a person clicks approve in the UI",
      "Sends an email and continues"
    ],
    "answer": 2,
    "explain": "An approval job runs no commands; it halts the workflow and waits for a human to approve before downstream jobs run."
  },
  {
    "q": "What is the safest way to reference an orb in your config?",
    "choices": [
      "Use @latest so you always get fixes",
      "Pin a specific version like circleci/node@5.2.0",
      "Copy the orb's source into your config",
      "Reference it without a version"
    ],
    "answer": 1,
    "explain": "Pin the version so a green pipeline stays green. Floating to latest lets an upstream change break your build without any commit of yours."
  }
]
```


---

# When it breaks: flaky tests, slow builds, and managed-CI tradeoffs

A pipeline that works on day one will eventually go red for reasons that have nothing to do with your code. Tests that pass locally fail in CI. The build that took two minutes now takes twelve. And one morning your whole team is blocked because someone else's infrastructure is having a bad day. This phase is about those moments - speeding pipelines up, debugging the weird ones, and being clear about what you give up by not running your own CI.

## Parallelism and test splitting: the real speed lever

Caching shaves the install. The test run itself is usually the long pole, and the fix is to run it across several machines at once. Set `parallelism` and CircleCI spins up that many copies of the job, then you tell it to split the test files between them.

```yaml
jobs:
  test:
    docker:
      - image: cimg/node:20.11
    parallelism: 4
    steps:
      - checkout
      - run: npm ci
      - run: |
          TESTS=$(circleci tests glob "test/**/*.test.js" | circleci tests split --split-by=timings)
          npx jest $TESTS
```

*What just happened:* `parallelism: 4` launches four identical containers for this one job. `circleci tests glob` lists all the test files; `circleci tests split` hands each container a different slice. With `--split-by=timings`, CircleCI uses timing data from past runs to balance the slices so each machine finishes at roughly the same moment, instead of one container getting all the slow tests. Four machines, roughly a quarter of the wall-clock time.

The catch: `--split-by=timings` needs timing data to balance well, and it gets that from `store_test_results`. If you never store results, the first runs split blindly by filename.

```yaml
      - run: npx jest --reporters=jest-junit
      - store_test_results:
          path: ./test-results
```

*What just happened:* `store_test_results` uploads JUnit-format XML so CircleCI learns how long each test took. Next run, the timings-based split is accurate, and as a bonus the UI shows you which specific tests failed instead of a wall of console output.

## Debugging the "passes locally, fails in CI" classic

This is the most common CircleCI frustration, and it's almost never CircleCI's fault. The CI machine is a clean, fresh environment - which means it's exposing something your laptop was hiding.

The usual culprits, in order of how often they're the answer:

- **An uncommitted file.** CI only has what's in git. If a config file, fixture, or env file is gitignored and your tests need it, CI won't have it.
- **A dependency you installed globally on your laptop** months ago and forgot. CI installs only what's in your lockfile.
- **Test-order dependence.** Tests that share state pass locally in one order and fail when parallelism splits them across machines in a different order.
- **An environment variable** set in your shell but not in CircleCI's project settings.

When reading is not enough, **SSH into the build**. CircleCI lets you re-run a failed job with SSH access and poke at the actual machine where it failed.

```text
# In the CircleCI UI: "Rerun job with SSH", then:
ssh -p <port> <user>@<ip>
cd project
npm test          # reproduce it on the exact machine that failed
ls -la            # is that file actually here?
env | grep API    # is the variable actually set?
```

*What just happened:* you're now on the real CI container, in the real working directory, with the real environment. This turns "it fails in CI and I can't see why" into ordinary local debugging. Reproducing the failure by hand here beats a dozen speculative commit-and-push rounds every time.

> Secrets like API keys never belong in `config.yml` - that file is in your repo for the world to read. Set them as environment variables in the CircleCI project settings (or a context for sharing across projects). They're injected at runtime and masked in logs.

## Contexts: secrets shared across projects

A single project's env vars live in its settings. When several projects need the same secret - a shared deploy key, a registry token - a **context** holds it once and grants jobs access.

```yaml
workflows:
  deploy:
    jobs:
      - deploy:
          context: production-secrets
          filters:
            branches:
              only: main
```

*What just happened:* the `deploy` job gains every environment variable stored in the `production-secrets` context. You can restrict which teams or branches may use a context in the org settings, which is how you keep production credentials out of reach of every random branch build.

## The managed-CI tradeoff: what you're really buying

CircleCI is somebody else's infrastructure. That's the whole pitch and the whole cost, so be clear-eyed about both sides.

What you get: no build servers to patch, fresh clean machines on every run, instant scale-out across dozens of parallel containers, and pre-built executors for languages and macOS you'd struggle to maintain yourself. For most teams this is an obvious win - running a reliable Jenkins fleet is a real job nobody enjoys.

What you give up:

- **Cost scales with minutes.** Heavy parallelism is fast but burns credits. The lever that speeds you up is the same lever that grows the bill, so tune `parallelism` to a real need, not a vibe.
- **You don't control the outage.** When CircleCI has an incident, your whole team's merges stop and you can only wait. Self-hosted means you own that risk instead of outsourcing it.
- **Data leaves your network.** Your source and secrets run on their machines. For some regulated environments that alone forces self-hosted runners.
- **The config is theirs.** You write CircleCI YAML, not a portable standard. Moving to another CI later is a rewrite, not a copy.

CircleCI's answer to the network and control concerns is **self-hosted runners** - your own machines doing the execution while CircleCI still orchestrates. That's the hybrid middle: you keep the nice workflow UI and orbs, but sensitive jobs run on hardware you control. It's more to operate than pure cloud, less than a full Jenkins.

```mermaid
graph LR
  A[Pure cloud CircleCI] -->|more control, more ops| B[Self-hosted runners]
  B -->|full control, full ops| C[Run your own Jenkins]
```

*What just happened:* the same axis runs through every CI choice - convenience on one end, control on the other. CircleCI's cloud sits at the convenient end, self-hosted runners in the middle, rolling your own at the far end. Pick by what your situation actually demands, not by what sounds impressive.

In the wild: most teams over-provision parallelism and under-invest in test speed, then wonder why the CI bill climbed. Faster tests beat more machines almost every time - four parallel containers running a slow, flaky suite is paying four times to be frustrated. Fix the suite first, then split it.

For the bigger picture of where CircleCI fits among other tools, [What CI/CD does](/guides/what-cicd-does) is the map; this guide was the territory for one specific city on it.

```quiz
[
  {
    "q": "What makes '--split-by=timings' balance test slices well across parallel machines?",
    "choices": [
      "It reads the file sizes",
      "It uses timing data from store_test_results in past runs",
      "It runs every test twice to measure",
      "It splits alphabetically"
    ],
    "answer": 1,
    "explain": "Timings-based splitting relies on JUnit results uploaded via store_test_results. Without that data the split falls back to balancing by filename."
  },
  {
    "q": "Where should an API key used by a deploy job live?",
    "choices": [
      "Plain text in config.yml",
      "In a comment in the repo README",
      "As an env var in CircleCI project settings or a context",
      "Hardcoded into the test files"
    ],
    "answer": 2,
    "explain": "config.yml is in your repo for anyone to read. Secrets go in project settings or a context, injected at runtime and masked in logs."
  },
  {
    "q": "Which is a genuine tradeoff of managed cloud CI versus self-hosting?",
    "choices": [
      "You must patch the build servers yourself",
      "When the provider has an outage, your merges stop and you can only wait",
      "You cannot run tests in parallel",
      "You have to maintain your own macOS hardware"
    ],
    "answer": 1,
    "explain": "With managed CI you outsource the infrastructure but also the outage risk: an incident on their side blocks your team. Self-hosting trades that for ops you own."
  }
]
```