Deploying with Kamal — Guidebook

How to ship Heatwave to the containerized stack. The entry point is bin/deploy; read README.md first for the architecture. For day-2 operations after a deploy, see MANAGING.md.

Production is on Kamal. bin/deploy production targets the Kamal prod config (config/deploy.yml) and performs a normal rolling deploy — prod cut over from Capistrano on 2026-06-07. The same bin/deploy flow drives both staging and production (pick the env); everything below applies to both.

TL;DR

# Deploy the current branch to staging (auto-migrates, full edge purge):
bin/deploy staging

# Deploy + run migrations across both DBs (gated on prod):
bin/deploy production --migrate

# Throwaway test of uncommitted work on staging:
bin/deploy staging --allow-dirty

# Ship clean origin/master from a throwaway worktree (don't disturb your tree):
bin/deploy production --in-worktree

bin/deploy --help prints the full flag list.

Before you can deploy (one-time per machine)

1Password — be signed in to the warmlyyours.1password.com account with the IT vault, or drop a service-account token at .kamal/.op-service-account-token (gitignored — the headless-reliable path; see MANAGING.md → Secrets).
config/master.key present locally (it backs RAILS_MASTER_KEY and the staging/production env-keys). Worktrees get it symlinked by bin/setup-worktree.
Toolchain — mise pins Ruby/Node; bin/deploy runs Kamal via mise exec -- bundle exec kamal. gum gives the nice prompts (optional).
Validate secret resolution before your first deploy:
Terminal window
```
mise exec -- bundle exec kamal secrets print -d staging
```

What branch ships

Kamal builds the working tree (builder.context: '.'), and for staging it builds remotely on the box (builder.remote: ssh://deploy@100.123.47.52, local: false) — no Mac emulation. There is no “pick a branch” prompt like the old Capistrano bin/deploy had. Instead:

Default: whatever is checked out must be clean and in sync with its upstream — bin/deploy hard-gates on this (require_clean_tree) so the image always equals a pushed commit. To deploy a different branch, check it out and push it.
--allow-dirty: skip the gate and ship the working tree as-is (loud warning). Use only for a throwaway staging test — the image will match no git commit.
--in-worktree: deploy clean origin/master from a throwaway worktree (~/.heatwave-deploy by default) without disturbing your current checkout.

Staging is fine to deploy from whatever worktree/branch you’re on, as long as the tree is clean and pushed. The clean-tree gate is the safety net that replaces “which branch?” — what ships is always reproducible from origin.

The deploy lifecycle

sequenceDiagram
    autonumber
    participant Dev as bin/deploy
    participant Git as git
    participant OP as 1Password
    participant K as kamal
    participant Box as Box (build+host)
    participant Reg as GHCR
    participant Proxy as kamal-proxy

    Dev->>Git: require_clean_tree (clean + in sync w/ upstream)
    Dev->>OP: op_session (unlock once / SA token)
    Dev->>K: kamal build deliver [-d staging]
    K->>Box: build image (remote, builder.context = working tree; GIT_REVISION build arg → ENV APP_REVISION)
    Box->>Reg: push image (+ pull onto the hosts)
    Dev->>K: MIGRATE before the swap — app exec --primary --roles=sidekiq db:migrate (both DBs)
    Note over Dev: a failed migration aborts here — live app untouched
    Dev->>K: kamal deploy --skip-push [-d staging]
    K->>Box: pre-deploy hook → quiet Sidekiq (TSTP)
    K->>Proxy: boot new container, wait for /up (≤ deploy_timeout 90s)
    Proxy-->>K: healthy → route traffic to new, stop old
    K->>Box: post-deploy hook → clear quiet marker + reap stale containers
    K-->>Dev: deploy ok
    Dev->>K: R2 frontend-asset sync (prod: failure aborts)
    Dev->>K: detach AppSignal sourcemap upload in the web container (DELETE_MAPS)
    Dev->>K: edge-cache purge (staging: full zone · prod: none by default)
    Note over Dev: optional — edge worker (-e), bulk redirects (-r)

Deployment concurrency

The Kamal deploy lock is narrower than bin/deploy. Image delivery, the wrapper’s pre-swap migration, and its post-deploy R2/AppSignal/cache work happen outside the locked app-swap section. Two operators can therefore build at the same time; if their swap windows do not overlap, both releases can succeed one after the other instead of the second command being rejected. kamal lock status reports an active Kamal mutation, not another wrapper still building.

Until bin/deploy has its own end-to-end production lock:

coordinate one production deploy at a time from command start through its final post-deploy step;
do not reboot or replace Postgres, PgBouncer, HAProxy, or Valkey while a deploy is in progress;
if two deploys were started, stop the one that has not entered its mutating phase and verify the live revision before continuing;
after any suspected overlap, verify the revision in all four app roles, public /up, Sidekiq fetch state, PgBouncer waiters, and application errors.

This is why a second production deploy begun a few minutes after another may not appear blocked: the lock serializes the swap, not the full release pipeline. The implementation follow-up is tracked in Sidekiq operations and memory hardening.

Step by step

Clean-tree gate (require_clean_tree) — refuses a dirty or unpushed tree. Bypass with --allow-dirty (throwaway) or --in-worktree (clean master).
1Password unlock (op_session) — one approval up front, before the ~minute-long build, so a secret failure surfaces early. Reused by every op Kamal spawns. A service-account token skips the desktop app entirely.
Build + deliver (kamal build deliver) — build on the remote builder, push to GHCR, pull onto the hosts. No container swap yet.
- The deploy revision arrives as the GIT_REVISION build arg → ENV APP_REVISION (→ config.x.revision, served to the front-end via #page-config). Nothing revision-specific is baked into the webpack bundles, so the cached asset layers survive Ruby-only deploys.
Migrate BEFORE the swap — always, on every deploy (no flag, no confirm; --skip-migrate is the rare escape hatch). A single runner (kamal app exec --primary --roles=sidekiq --version <HEAD> 'bin/rails db:migrate') migrates both heatwave and heatwave_versions on the just-delivered image. A failed migration aborts the deploy here — the live app keeps running old code on the old schema. Migrations must stay backward-compatible with the still-running old code (expand/contract). On a first deploy to a host (no role env-file yet) the migrate is deferred to right after the boot.
Boot/swap (kamal deploy --skip-push) — rolling boot behind kamal-proxy of the already-delivered image.
- The new container must answer /up with 200 within deploy_timeout: 90s (Puma preload is ~20s). kamal-proxy keeps the old container serving until then.
- pre-deploy quiets Sidekiq (TSTP) so in-flight jobs drain during the boot window. Sidekiq Pro super_fetch recovers anything still running regardless.
- post-deploy (success only) clears the quiet marker and reaps stale app containers.
R2 frontend-asset sync — pushes this deploy’s content-hashed bundles to the per-env heatwave-frontend-assets-* bucket (additive; never deletes; skips *.map + the manifest). A failed sync on production aborts — un-synced bundles would orphan on the next deploy (the stale-chunk 404). On staging a failed sync only warns and the deploy continues (origin still serves that deploy’s bundles).
Sourcemap upload (post-deploy, detached in the container) — the .map files ride in the image; after the R2 sync, bin/deploy daemonizes the AppSignal upload inside the live web container (--primary, so one upload even with multiple web hosts; survives the local runner exiting — CI/agent safe), using the account-wide push key (never echoed), then deletes the maps from the asset volume (DELETE_MAPS=true). Its output goes to the container’s stdout — i.e. kamal app logs --roles=web — and nowhere else (there is no local log file); the deploy doesn’t wait for it.
Edge-cache purge — staging purges the whole zone; production purges NOTHING by default (bundles are content-hashed + immutable on R2, and HTML is intentionally never flushed on a prod deploy; --purge-full forces a full-zone purge). A tmp/cloudflare_purge_urls.txt queue file, if present, is also purged.
Optional — -e deploys the Cloudflare edge worker; -r re-uploads bulk redirects. With gum, these (plus purge toggles) are a checkbox menu.

Flags

Flag	Effect
`staging` / `production`	Destination (else prompted). Staging adds `-d staging`.
`--migrate`	Accepted but a no-op (back-compat) — migrations always run now.
`--skip-migrate`, `--no-migrate`	Skip the pre-swap migration (rare escape hatch, e.g. re-boot of unchanged code).
`--allow-dirty`	Skip the clean/in-sync gate — ship the working tree as-is.
`--in-worktree[=PATH]`	Deploy clean `origin/master` from a throwaway worktree.
`--purge-full`	Force a full-zone edge purge (prod default purges nothing).
`--skip-cache-purge`	Deploy without any edge-cache purge.
`-e`, `--deploy-edge-worker`	Also deploy the Cloudflare `www-edge` worker.
`-r`, `--upload-bulk-redirects`	Re-upload `data/cloudflare_rules/*.csv` to Cloudflare.
`-P`, `--skip-push`	Boot an already-pushed image (skip the build; still migrates).
`-y`, `--yes`, `--non-interactive`	No gum menus — options from flags (auto when no TTY).

Migrations

Heatwave spans two databases (heatwave + heatwave_versions), so a migrate runs against both. bin/deploy uses kamal app exec --primary --roles=sidekiq so it executes on exactly one host (and keeps a heavy migration off web).
Never auto-run on boot — the image entrypoint does not migrate. This is a project hard rule (schema/data risk).
Both envs migrate BEFORE the swap, on every deploy — no flag, no confirm. The migration runs on the freshly-delivered image pinned to `–version `; a failure aborts before any traffic shifts. `--skip-migrate` is the only opt-out. Migrations must be expand/contract-safe (old code keeps serving until the swap).

Run them standalone any time with the alias:

mise exec -- bundle exec kamal migrate            # = app exec --reuse 'bin/rails db:migrate'
mise exec -- bundle exec kamal app exec --primary -d staging 'bin/rails db:migrate'

Dev db:migrate is allowed freely (it no longer rewrites db/structure.sql — the schema files are regenerated on master by bin/deploy production via bin/regen-structure); prod migrations and any db:rollback/db:migrate:redo still require explicit human go-ahead (see CLAUDE.md hard-block table).

Rollback

kamal-proxy keeps prior image versions, so rollback is a re-point, not a rebuild:

mise exec -- bundle exec kamal app versions -d staging     # list deployed versions
mise exec -- bundle exec kamal rollback <VERSION> -d staging

A rollback re-fires the pre-deploy/post-deploy hooks (Sidekiq is quieted then swapped). If the schema moved forward with a deploy you’re rolling back, roll the code back first, then decide on the data — db:rollback is gated and reverts data, so think before running it.

If a deploy fails mid-flight, bin/deploy un-quiets Sidekiq automatically (it TERMs PID 1; Kamal’s --restart unless-stopped revives a fresh fetching process). Bare-kamal users resume with:

mise exec -- bundle exec kamal app boot --roles=sidekiq,sidekiq_heavy,sidekiq_edi -d staging

Deploying without `bin/deploy`

bin/deploy is a convenience wrapper; the underlying Kamal commands work directly (you lose the clean-tree gate, the gated-migration UX, the sourcemap upload, and the edge purge — do those by hand):

mise exec -- bundle exec kamal deploy -d staging
mise exec -- bundle exec kamal app logs -d staging -f

See MANAGING.md for the full command surface (console, shell, dbc, accessory boot, secrets).