Class: Assistant::TechnicalSupport::CaseReplayRunner

Inherits:
Object
  • Object
show all
Defined in:
app/services/assistant/technical_support/case_replay_runner.rb

Overview

Executes a de-identified replay through the same ChatService and persisted
tool trace used by Sunny. The source Support Case service is intentionally
absent: this is a holdout test of canonical content, not case retrieval.

Defined Under Namespace

Classes: BlankResponse, Result

Constant Summary collapse

DEFAULT_MODEL = 
'gemini-flash'
RECOVERY_MODEL = 
'claude-sonnet'

Instance Method Summary collapse

Constructor Details

#initialize(conversation_factory: nil, chat_service_factory: nil, trace_reader: nil, prompt_trace_reader: nil, grounding_submitter: nil) ⇒ CaseReplayRunner

Builds a runner with optional persistence and chat test doubles.

Parameters:

  • conversation_factory (Proc, nil) (defaults to: nil) —

    Sunny conversation factory

  • chat_service_factory (Proc, nil) (defaults to: nil) —

    ChatService factory

  • trace_reader (Proc, nil) (defaults to: nil) —

    persisted tool-trace reader

  • prompt_trace_reader (Proc, nil) (defaults to: nil) —

    per-turn persisted system-prompt reader

  • grounding_submitter (Proc, nil) (defaults to: nil) —

    completed-turn grounding dispatcher



35
36
37
38
39
40
41
42
43
44
 
# File 'app/services/assistant/technical_support/case_replay_runner.rb', line 35

def initialize(
  conversation_factory: nil, chat_service_factory: nil, trace_reader: nil,
  prompt_trace_reader: nil, grounding_submitter: nil
)
  @conversation_factory = conversation_factory || method(:create_conversation)
  @chat_service_factory = chat_service_factory || ->(arguments) { Assistant::ChatService.new(**arguments) }
  @trace_reader = trace_reader || method(:read_tool_trace)
  @prompt_trace_reader = prompt_trace_reader || method(:read_system_prompt_trace)
  @grounding_submitter = grounding_submitter || method(:submit_response_grounding)
end

Instance Method Details

#call(account:, support_case:, prompts:, tool_services:) ⇒ Result

Runs each replay turn through Sunny's existing ChatService.

Parameters:

  • account (Account) —

    authorized employee replay operator

  • support_case (SupportCase) —

    source case retained for audit metadata

  • prompts (Array<String>) —

    de-identified replay turns

  • tool_services (Array<String>) —

    allowed Sunny service keys

Returns:

  • (Result) —

    conversation, final answer, and persisted tool trace



53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
 
# File 'app/services/assistant/technical_support/case_replay_runner.rb', line 53

def call(account:, support_case:, prompts:, tool_services:)
  conversation = conversation_factory.call(
    user: .party,
    title: "Technical replay: #{support_case.case_number}",
    metadata: {
      conversation_type: 'technical_support_replay',
      source_support_case_id: support_case.id
    }
  )
  conversation.update!(
    tool_services:,
    model_preference: DEFAULT_MODEL
  )

  context = user_context(, tool_services)
  system_prompt_trace = []
  answers = Array(prompts).map.with_index do |prompt, index|
    answer = run_turn(
      conversation:,
      prompt:,
      model: DEFAULT_MODEL,
      tool_services:,
      context:
    )
    if answer.blank?
      answer = recover_blank_turn(
        conversation:,
        prompt:,
        tool_services:,
        context:
      )
    end

    # Recovery uses complete_only, so the persisted user turn is not
    # duplicated. A second blank remains an operational failure rather
    # than being converted into knowledge evidence.
    raise BlankResponse, "Sunny returned no content for replay turn #{index + 1}." if answer.blank?

    system_prompt_trace << prompt_trace_reader.call(conversation, index + 1)
    grounding_submitter.call(conversation)
    answer
  end

  Result.new(
    conversation_id: conversation.id,
    answer: answers.last.to_s,
    tool_trace: trace_reader.call(conversation),
    system_prompt_trace:
  )
end