Class: Heatwave::Crawler::ResultEnricher

Inherits:
Object
  • Object
show all
Defined in:
app/services/heatwave/crawler/result_enricher.rb

Overview

Turns a tier's FetchOutcome into the public Result: extracts
readable content (readability → Markdown) for content modes, attaches
technical-SEO fields for :inspect mode, applies the policy's content
cap, and normalizes failures.

Constant Summary collapse

TRUNCATION_MARKER = 
"\n\n[... content truncated]"

Instance Method Summary collapse

Constructor Details

#initialize(mode:, policy:) ⇒ ResultEnricher

Returns a new instance of ResultEnricher.

Parameters:



13
14
15
16
 
# File 'app/services/heatwave/crawler/result_enricher.rb', line 13

def initialize(mode:, policy:)
  @mode = mode
  @policy = policy
end

Instance Method Details

#failure(url, outcome) ⇒ Heatwave::Crawler::Result

Parameters:

Returns:



35
36
37
38
39
40
41
 
# File 'app/services/heatwave/crawler/result_enricher.rb', line 35

def failure(url, outcome)
  return Result.new(url: url, error: 'no tier attempted') unless outcome

  Result.new(url: url, status: outcome.status, tier: outcome.tier,
             error: outcome.error || 'content not acceptable',
             details: outcome.details.dup)
end

#success(url, outcome) ⇒ Heatwave::Crawler::Result

Parameters:

Returns:



21
22
23
24
25
26
27
28
29
30
 
# File 'app/services/heatwave/crawler/result_enricher.rb', line 21

def success(url, outcome)
  content = cap_content(extract_content(outcome))
  log_acceptance(url, outcome, content)

  Result.new(
    success: true, url: url, final_url: outcome.final_url,
    status: outcome.status, tier: outcome.tier, content: content,
    details: success_details(outcome, content)
  )
end