Class: AiUsage::SpendGuard

Inherits:
Object
  • Object
show all
Defined in:
app/services/ai_usage/spend_guard.rb

Overview

Circuit breaker on LLM spend, checked before every chat completion.

This is a runaway guard, not a budget. Caps sit far above normal usage;
tripping one means something is looping, not that the month was busy. The
thing it exists to stop is the AI analogue of the 315k-job enqueue in
AGENTS.md — an agent that keeps calling until someone notices the invoice.

== Where it runs

+config/initializers/ruby_llm_instrumentation.rb+ calls SpendGuard.check! from the
+RubyLLMCompleteDepthTracking+ module already prepended to
+RubyLLM::Chat#complete+. That is the single chokepoint every completion
passes through, including RubyLLM's recursive tool-loop rounds and calls
from the 17 sites that never wrapped themselves in
+RubyLLM::Instrumentation.with+ (those log as feature "unknown" and are
capped under DEFAULT_FEATURE_CAP).

Checking at every recursion depth — not just the outermost call — is
deliberate: the runaway we most want to catch is an agent stuck in its own
tool loop inside a single +ask+, which a depth-1-only check would miss
entirely.

== Failure policy

Fails open. A guard that takes AI down when Postgres hiccups is worse
than the runaway it prevents, so every internal error is logged and
swallowed. Only CapExceeded escapes.

== Disabling

Set +AI_SPEND_CAP_DISABLED=1+ to switch the guard off. It is declared in
+config/deploy.yml+'s +env.clear+ specifically so flipping it is a
+kamal env push+ plus a restart rather than a full deploy — a breaker you
cannot turn off quickly is its own outage.

Defined Under Namespace

Classes: CapExceeded

Constant Summary collapse

WINDOW =

Rolling lookback. Deliberately not a calendar day — a midnight reset
would hand a runaway a fresh budget a few hours into the incident.

24.hours
FEATURE_CAPS =

Per-feature ceilings in USD over WINDOW.

Sized off 60 days of production +ai_usage_logs+ (observed daily max in
brackets), targeting roughly 2.5x the worst legitimate day so a busy
Monday never trips the breaker:

assistant_chat avg $15.46 p95 $50.05 [max $154.58]
call_record_analysis avg $ 2.05 p95 $ 3.46 [max $ 7.03]
video_seo avg $ 0.43 p95 $ 1.77 [max $ 5.62]
blog_schema avg $ 0.40 p95 $ 1.15 [max $ 4.81]

Everything not listed gets DEFAULT_FEATURE_CAP, which already clears
every non-Sunny feature's worst day by ~7x.

{
  'assistant_chat' => 400.0
}.freeze
DEFAULT_FEATURE_CAP =

Ceiling for any feature without an entry in FEATURE_CAPS.

50.0
GLOBAL_CAP =

Ceiling across all features combined, in USD over WINDOW. Catches a
runaway that spreads itself thinly enough to stay under every per-feature
cap. Observed production max is $157.32/day.

600.0

Class Method Summary collapse

Class Method Details

.cap_for(feature) ⇒ Float

Returns the USD ceiling that applies to this feature.

Parameters:

  • feature (String)

Returns:

  • (Float) —

    the USD ceiling that applies to this feature



101
102
103
 
# File 'app/services/ai_usage/spend_guard.rb', line 101

def cap_for(feature)
  FEATURE_CAPS.fetch(feature, DEFAULT_FEATURE_CAP)
end

.check!(feature) ⇒ void

This method returns an undefined value.

Raise if the feature (or the app as a whole) has already spent its
WINDOW allowance.

Parameters:

  • feature (String, Symbol, nil) —

    the RubyLLM::Instrumentation
    feature tag; nil/blank is treated as "unknown", same as AiUsageLog

Raises:



83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
 
# File 'app/services/ai_usage/spend_guard.rb', line 83

def check!(feature)
  return if disabled?

  name = feature.presence&.to_s || 'unknown'
  feature_usd, global_usd = spend_in_window(name)

  enforce!(name, feature_usd, cap_for(name), scope: "feature #{name}")
  enforce!(name, global_usd, GLOBAL_CAP, scope: 'all features')
rescue CapExceeded
  raise
rescue StandardError => e
  # Fail open — see the class docs. A broken breaker must not break AI.
  Rails.logger.error "[AiUsage::SpendGuard] check failed open: #{e.class}: #{e.message}"
  nil
end

.disabled? ⇒ Boolean

Returns true when the kill switch is set.

Returns:

  • (Boolean) —

    true when the kill switch is set



106
107
108
 
# File 'app/services/ai_usage/spend_guard.rb', line 106

def disabled?
  ENV['AI_SPEND_CAP_DISABLED'].to_b
end