Module: Assistant::AdvisorPilot

Defined in:
app/services/assistant/advisor_pilot.rb

Overview

Anthropic advisor-tool pilot (beta +advisor-tool-2026-03-01+).

A cheaper EXECUTOR model (Sunny's Sonnet content-authoring tier) consults a
higher-tier ADVISOR model (Opus 5) mid-generation for strategic guidance —
the advisor reads the full transcript server-side and returns a plan; the
executor continues informed by it. This is the surgical alternative to the
ComplexityEscalator's whole-conversation jump to Opus, which busts the
(model-scoped) prompt cache and pays Opus rates for every subsequent token.

Two coordinated halves, both keyed off AdvisorPilot.active_for?:

  • ChatService#configure_conversation adds the beta HEADER for eligible
    executor models.
  • config/initializers/ruby_llm_advisor_tool.rb injects the TOOL entry into
    the payload — only when the header rides the same request, so a tool
    without its beta header (a 400) is impossible by construction.

Known v1 limitations (accepted, documented in
doc/tasks/202608041259_SUNNY_TOOL_USE_FEATURE_EVAL.md):

  • RubyLLM drops the advisor/server_tool_use blocks from parsed messages,
    so advice text is not re-sent on later rounds/turns. Dropping BOTH
    blocks keeps requests API-valid; the advice still shapes the response
    it was fetched for (executor reads it within the same API call).
  • ponytail: no pause_turn continuation — max_uses 2 keeps the server-side
    loop far from the ~10-iteration pause threshold; revisit if server
    tools (web_search) are ever added.
  • Advisor spend is observed via the 'advisor_usage.sunny' notification
    (+ log line), not yet folded into CostCalculator / MonthlyBudget.

Kill switch: the pilot is OFF unless +SUNNY_ADVISOR+ is truthy.

Constant Summary collapse

BETA =

Beta header value gating the advisor tool API surface.

'advisor-tool-2026-03-01'
MAX_USES_PER_REQUEST =

Advisor calls allowed per request (per tool-loop round, not per
conversation). Bounds Opus sub-inference spend and keeps the server-side
loop short enough that pause_turn (unhandled by RubyLLM) stays out of reach.

2
MAX_CALLS_PER_TURN =

Hard ceiling on advisor calls across ALL rounds of one turn. max_uses is
per-request only, and a tool-heavy turn re-sends the tool array every
round — without this cap an ~18-round turn could consult the advisor
2×N times. Once reached, later rounds simply omit the advisor entry
(safe in this stack: RubyLLM 1.16 drops advisor blocks from re-sent
history, so removal never orphans an advisor_tool_result).

4
ADVISOR_MAX_TOKENS =

Cap on the advisor's output (thinking + text) per call. API minimum 1024.

4096

Class Method Summary collapse

Class Method Details

.active_for?(model_id) ⇒ Boolean

Whether requests for +model_id+ should carry the advisor tool + header.

Parameters:

  • model_id (String, nil)

Returns:

  • (Boolean) 


79
80
81
 
# File 'app/services/assistant/advisor_pilot.rb', line 79

def active_for?(model_id)
  enabled? && executor_model_ids.include?(model_id.to_s)
end

.advisor_model_id ⇒ String

The advisor model — Opus 5 (valid advisor for a Sonnet 5 executor).

Returns:

  • (String) 


72
73
74
 
# File 'app/services/assistant/advisor_pilot.rb', line 72

def advisor_model_id
  AiModelConstants.id(:anthropic_opus)
end

.begin_request! ⇒ void

This method returns an undefined value.

Reset the per-request observation high-water mark (provider complete
prepend, once per API request). Streaming message_delta events carry the
CUMULATIVE usage.iterations array — the mark lets observe_iterations
count and report only the increment instead of re-counting every delta.



97
98
99
 
# File 'app/services/assistant/advisor_pilot.rb', line 97

def begin_request!
  Thread.current[:sunny_advisor_request_seen] = 0
end

.begin_turn! ⇒ void

This method returns an undefined value.

Reset the counter at turn start (ChatService, when it grants the beta
header for a NEW user turn — not on complete_only retries).



88
89
90
 
# File 'app/services/assistant/advisor_pilot.rb', line 88

def begin_turn!
  Thread.current[:sunny_advisor_turn_calls] = 0
end

.count_turn_calls(count) ⇒ void

This method returns an undefined value.

Accumulate observed advisor calls against the turn budget.

Parameters:

  • count (Integer) 


116
117
118
 
# File 'app/services/assistant/advisor_pilot.rb', line 116

def count_turn_calls(count)
  Thread.current[:sunny_advisor_turn_calls] = Thread.current[:sunny_advisor_turn_calls].to_i + count
end

.enabled? ⇒ Boolean

Whether the pilot is switched on (deploy-time opt-in).

Returns:

  • (Boolean) 


57
58
59
 
# File 'app/services/assistant/advisor_pilot.rb', line 57

def enabled?
  ENV['SUNNY_ADVISOR'].to_b
end

.executor_model_ids ⇒ Array<String>

Executor models that get the advisor. Sonnet only: the pairing table
requires the advisor to be at least as capable as the executor, and the
pilot's whole point is lifting the mid-tier — Opus executors already are
the advisor tier.

Returns:

  • (Array<String>) 


66
67
68
 
# File 'app/services/assistant/advisor_pilot.rb', line 66

def executor_model_ids
  [AiModelConstants.id(:anthropic_sonnet)]
end

.observe_iterations(iterations) ⇒ void

This method returns an undefined value.

Surface advisor sub-inference spend. Advisor tokens bill at the ADVISOR
model's rates and are deliberately absent from the top-level usage RubyLLM
parses — without this they are invisible client-side. Emits one
'advisor_usage.sunny' notification (and a log line) per usage payload
observed; consumers aggregate. Streaming may deliver the iterations array
more than once — treat events as observability, not billing.

Parameters:

  • iterations (Array<Hash>, nil) —

    +usage.iterations+ from a response



148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
 
# File 'app/services/assistant/advisor_pilot.rb', line 148

def observe_iterations(iterations)
  advisor = Array(iterations).select { |entry| entry.is_a?(Hash) && entry['type'] == 'advisor_message' }
  return if advisor.empty?

  # Streaming deltas repeat the cumulative array — count/report only the
  # entries beyond this request's high-water mark (append-only array).
  seen = Thread.current[:sunny_advisor_request_seen].to_i
  fresh = advisor.drop(seen)
  return if fresh.empty?

  Thread.current[:sunny_advisor_request_seen] = advisor.size
  count_turn_calls(fresh.size)
  input  = fresh.sum { |entry| entry['input_tokens'].to_i }
  output = fresh.sum { |entry| entry['output_tokens'].to_i }
  ActiveSupport::Notifications.instrument(
    'advisor_usage.sunny',
    calls: fresh.size, input_tokens: input, output_tokens: output,
    models: fresh.pluck('model').uniq.compact
  )
  Rails.logger.info(
    "[AdvisorPilot] #{fresh.size} advisor call(s): #{input} in / #{output} out tokens " \
    "(billed at advisor-model rates)"
  )
end

.remaining_turn_calls ⇒ Integer

Advisor calls this turn may still make.

Returns:

  • (Integer) 


103
104
105
 
# File 'app/services/assistant/advisor_pilot.rb', line 103

def remaining_turn_calls
  [MAX_CALLS_PER_TURN - Thread.current[:sunny_advisor_turn_calls].to_i, 0].max
end

.tool_entry(max_uses: MAX_USES_PER_REQUEST) ⇒ Hash

The raw Anthropic tool entry appended to +payload[:tools]+. +caching+
turns on advisor-side prompt caching (the advisor re-reads the whole
transcript each call — long blog/email sessions make this pay), aligned
with Sunny's cache TTL lever.

Parameters:

  • max_uses (Integer) (defaults to: MAX_USES_PER_REQUEST) —

    per-request allowance; callers pass the
    remaining TURN budget so the per-turn ceiling is hard (3 calls spent →
    the next request gets max_uses: 1, never 2 → a turn can't exceed 4).

Returns:

  • (Hash) 


128
129
130
131
132
133
134
135
136
137
 
# File 'app/services/assistant/advisor_pilot.rb', line 128

def tool_entry(max_uses: MAX_USES_PER_REQUEST)
  {
    type: 'advisor_20260301',
    name: 'advisor',
    model: advisor_model_id,
    max_uses: max_uses.clamp(1, MAX_USES_PER_REQUEST),
    max_tokens: ADVISOR_MAX_TOKENS,
    caching: Assistant::PromptCache.cache_control
  }
end

.turn_call_budget_left? ⇒ Boolean

Whether this turn may still consult the advisor.

Returns:

  • (Boolean) 


109
110
111
 
# File 'app/services/assistant/advisor_pilot.rb', line 109

def turn_call_budget_left?
  remaining_turn_calls.positive?
end