Daily Focus Re-run Runbook

Audience: engineer on call for the CRM / Sunny stack.
When: the morning Daily Focus briefings failed, generated garbage, or are
stuck in "Processing", and you need to regenerate them for today.
Why it needs a runbook: a naive re-run of the orchestrator silently does
nothing in three separate ways. Each one looks like "the job ran and no
briefing appeared."

Design and feature background (roles, coverage policy, support-rep variant) is
in doc/tasks/202607161719_DAILY_FOCUS_SUPPORT_REPS.md.
That is a design doc, not a procedure — this is the procedure.

What normally runs

Piece	Schedule (America/Chicago)	Does
`DailyFocusAnalysisWorker`	`0 8 * * 1-5`	Fans out one `DailyFocusAnalysisRepWorker` per `RunPolicy` target (sales + support), staggered `index * 30s`, then `DailyFocusManagerReportWorker` after 20 min
`DailyFocusAnalysisRepWorker`	on demand	Creates/reclaims the rep's briefing and generates it through `DailyFocus::ChatRunner`
`DailyFocusOrphanReaperWorker`	`/10 5-18 * 1-5`	Re-enqueues briefings stranded in `processing` past `ORPHAN_AFTER`

A briefing is an AssistantConversation with metadata
conversation_type='daily_briefing', daily_focus_date, daily_focus_status
(processing / ready / failed), daily_focus_target_employee_id,
daily_focus_covered_employee_ids, daily_focus_audience. Manager approval
(DailyFocusController#approve) strips daily_focus_target_employee_id and
daily_focus_status and sets user_id to the rep.

Before you touch anything: triage

Run in a production Rails runner (see "Running commands in production" below):

convs = AssistantConversation.for_daily_focus_date(Date.current)
convs.map { |c|
  { id: c.id,
    rep: c.daily_focus_target_employee_id,
    user: c.user_id,
    status: c.daily_focus_status,      # nil => APPROVED
    audience: c.daily_focus_audience,
    updated: c.updated_at,
    live: DailyFocus.briefing_live?(c) }
}

Read the result against this table before choosing a step.

`daily_focus_status`	`target_employee_id`	State	Will a plain re-run regenerate it?
`ready`	set	Pending manager review	No — `briefing_complete?`
`nil`	`nil`	Approved (already emailed)	No — `briefing_complete?`
`processing`	set, `updated_at` < 15 min ago	Live or just-fired	No — `briefing_live?`
`processing`	set, `updated_at` > 15 min ago	Orphaned	Yes (reaper does it too)
`failed`	set	Failed	Yes
—	—	No row at all	Yes

"The briefing said something wrong" — read the prompt, not the output

Most reports of a bad briefing are a rendering fault, not a data fault, and
the two are told apart in one query. A conversation holds the prompt as its
first user message and the briefing as its last assistant message, so the
replica answers "did we hand it the wrong facts, or did it mangle the right
ones?" without touching production:

FOCUS_DATE=2026-08-10        # the day the rep is complaining about

# 1. find the conversation ids for that day
psql "$POSTGRES_REPLICA_URI" -v ON_ERROR_STOP=1 -v focus_date="'$FOCUS_DATE'" <<'SQL'
SET default_transaction_read_only = on;
SELECT ac.id, p.full_name, ac.metadata->>'daily_focus_status' AS status
FROM assistant_conversations ac
LEFT JOIN parties p ON p.id = (ac.metadata->>'daily_focus_target_employee_id')::bigint
WHERE ac.metadata->>'daily_focus_date' = :focus_date;
SQL

CONV=4647                    # one id from the list above

# 2. what we handed it — Section 1 arrives precomputed, so this is ground truth
psql "$POSTGRES_REPLICA_URI" -v ON_ERROR_STOP=1 -v conv="$CONV" <<'SQL'
SET default_transaction_read_only = on;
SELECT substring(content FROM position('SECTION 1 — ROLLING QUEUE COVERAGE' IN content) FOR 2500)
FROM assistant_messages WHERE assistant_conversation_id = :conv AND role = 'user' ORDER BY id ASC LIMIT 1;
SQL

# 3. what it produced
psql "$POSTGRES_REPLICA_URI" -v ON_ERROR_STOP=1 -v conv="$CONV" <<'SQL'
SET default_transaction_read_only = on;
SELECT content FROM assistant_messages
WHERE assistant_conversation_id = :conv AND role = 'assistant' ORDER BY id DESC LIMIT 1;
SQL

Step 2's marker tracks DailyFocus::CoverageSection.prompt_section — if it
returns nothing, read that method for the current heading rather than assuming
the section is missing.

Compare two reps' briefings for the same day before concluding anything: the
coverage table is a team view, so any disagreement between them is a rendering
bug by definition. That comparison is what found the 2026-08-10 batch — the
prompt had Chris's lunch, both briefings lost it differently (Basecamp
10137434670).

Procedure

1. Delete anything that counts as "complete"

DailyFocusAnalysisRepWorker#briefing_complete? is
daily_focus_status == 'ready' || daily_focus_target_employee_id.nil?. An
approved briefing has a nil target, so it counts as complete and is skipped
— even when its content is a failure message that a manager approved anyway.
There is no force flag. You must delete first.

This mirrors DailyFocusController#reject:

ids = [...]   # from triage; be explicit, never a blanket destroy_all
AssistantConversation.where(id: ids).find_each do |c|
  c.release_processing_lock! if c.processing_by_id.present?
  c.destroy!
end

destroy! cascades the conversation's messages and shares. Deleting also
clears gotcha 3 below, which is why it's the deterministic option.

2. Re-fire

Whole day, all reps — same path as the 8 AM cron:

DailyFocusAnalysisWorker.perform_async

It applies the 30-second stagger itself.

Specific reps — stagger them yourself, 60s apart:

[rep_id_1, rep_id_2, rep_id_3].each_with_index do |rep_id, i|
  DailyFocusAnalysisRepWorker.perform_in(i * 60, rep_id, nil, [])
end

Pass nil for conversation_id so the worker self-creates under its per-rep
advisory lock (daily_focus:<employee_id>:<date>) and reclaims orphans. The
third argument is covered_employee_ids — carry over the ids from the deleted
briefing if the rep was covering for someone.

⚠️ Passing a real conversation_id bypasses ensure_conversation_for
entirely — no complete check, no live check, no advisory lock. That's the CRM
"Generate" button's path (DailyFocusController#generate), which is fine
because it just created the conversation. Don't reuse it for a re-run.

3. Watch

AssistantConversation.for_daily_focus_date(Date.current)
                     .map { |c| [c.id, c.daily_focus_target_employee_id, c.daily_focus_status] }

A healthy generation takes up to ~7 minutes and keeps bumping updated_at as
PlanOrchestrator rewrites the execution_plan metadata. Fresh briefings land
ready — pending review, not sent. A manager re-approves each one from the
CRM Daily Focus review page (/daily_focus/review), which re-sends
InternalReportsMailer.daily_focus_approved. Reps who were already emailed from
the first (bad) approval will get a second email.

4. If some reps stall

Wait for DailyFocusOrphanReaperWorker (every 10 min, 05:00–18:00 weekdays) —
it re-enqueues anything in processing with updated_at older than 15 minutes,
preserving covered reps. If you don't want to wait, go back to step 1 for those
conversations and re-fire them individually with the 60-second stagger.

Why it silently did nothing

Symptom	Cause	Fix
Job ran, logs show "Enqueuing daily focus for N targets", but a rep's briefing is unchanged	`briefing_complete?` — the briefing is `ready`, or approved (target id stripped by `#approve`, so nil ⇒ complete)	Delete it first (step 1). No force flag exists.
Re-fired a stuck briefing, worker logged nothing and exited	`DailyFocus.briefing_live?` — `processing?` or `updated_at` within `DailyFocus::ORPHAN_AFTER` (15 min). The worker hits `next nil` and skips without reclaiming.	Wait >15 min, or destroy the conversation explicitly and re-fire
Fired everyone at once; several briefings frozen in `processing` with `processing_by_id` nil and a stale `execution_plan`	Sidekiq box OOM'd under ~8 concurrent LLM generations and hard-killed workers. `Sidekiq::Shutdown` is an `Interrupt`, not a `StandardError`, so the worker's `rescue` never marked them `failed`.	Stagger re-fires ~60s apart. Recover the stranded ones via the reaper or step 1.
Everything failed at once with "Step failed"	Upstream model/registry problem, not the briefing pipeline (`DailyFocus::CHAT_MODEL_KEY` is `gemini-flash`; a missing entry raises `ModelNotFoundError` that surfaces as a failed step)	Fix the model config first, then run this procedure

Running commands in production

kamal app exec --reuse --roles=web "bin/rails runner \"eval(Base64.decode64('<b64>'))\""

Pin --roles=web. Without it kamal app exec runs on every role, so a
perform_async enqueues once per role.
Base64-encode the runner script to dodge nested shell quoting — multi-line
Ruby through kamal app exec mangles otherwise.
bin/deploy and rails console are hard-blocked for agents; ask before
running anything here.

Reference

Thing	Where
`ORPHAN_AFTER`, `briefing_live?`, `CHAT_MODEL_KEY`, `rep_scope`	`app/services/daily_focus.rb`
`briefing_complete?`, `ensure_conversation_for`, reclaim	`app/workers/daily_focus_analysis_rep_worker.rb`
Fan-out + 30s stagger	`app/workers/daily_focus_analysis_worker.rb`
Orphan sweep	`app/workers/daily_focus_orphan_reaper_worker.rb`
`approve` / `reject` / `generate`	`app/controllers/daily_focus_controller.rb`
Metadata scopes	`app/models/assistant_conversation.rb`
Cron entries	`config/sidekiq_production_schedule.yml` (`daily_focus_analysis_worker`, `daily_focus_orphan_reaper`)