Class: AiCostReconciliationWorker

Inherits:
Object
  • Object
show all
Includes:
Sidekiq::Job
Defined in:
app/workers/ai_cost_reconciliation_worker.rb

Overview

Monthly check that /admin/ai_usage still matches the Anthropic invoice.

+AiUsage::CostReconciler+ and +rake ai_usage:reconcile+ have existed since the
manual audit that found Sunny's Opus usage was only ~43% captured, but nothing
ran them, so drift could only be found by someone thinking to look. This runs
the same reconciler on a schedule and reports when coverage slips.

The gap it watches for is structural: +ai_usage_logs+ infers token counts from
what RubyLLM's +complete+ returns, and streaming / tool-halt turns can return
nothing to infer from (see RubyLLM::Instrumentation.usage_from). Those calls
are billed but land unpriced. Anthropic's Cost API bills from the wire and
cannot miss them, which makes it the only ground truth we have.

Examples:

Run for the last 7 days instead of 30

AiCostReconciliationWorker.perform_async(7)

Constant Summary collapse

COVERAGE_THRESHOLD =

Report when tracked cost falls below this share of billed cost.

Not 100: the Cost API is org-wide (every Anthropic key, including the team's
Claude Code usage) while +ai_usage_logs+ is app-only, so some permanent
shortfall is expected and normal. 90 flags a real regression without firing
on that baseline.

90.0

Instance Method Summary collapse

Instance Method Details

#perform(since_days = 30) ⇒ void

This method returns an undefined value.

Parameters:

  • since_days (Integer) (defaults to: 30) —

    window size to reconcile



34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
 
# File 'app/workers/ai_cost_reconciliation_worker.rb', line 34

def perform(since_days = 30)
  report = AiUsage::CostReconciler.new.call(since_days: since_days)

  Rails.logger.info(
    "[AiCostReconciliation] #{since_days}d — billed $#{format('%.2f', report.real_total_usd)}, " \
    "tracked $#{format('%.2f', report.tracked_total_usd)} (#{report.coverage_pct}% coverage)"
  )

  return if report.coverage_pct >= COVERAGE_THRESHOLD

  report_shortfall(report, since_days)
rescue AiUsage::AnthropicAdminClient::MissingKeyError => e
  # No admin key in this environment. Not worth a retry storm — the key is a
  # deploy-time concern, not a transient failure.
  Rails.logger.warn "[AiCostReconciliation] skipped: #{e.message}"
end