Class: Heatwave::Crawler::Tiers::OxylabsBase

Inherits:
Object
  • Object
show all
Defined in:
app/services/heatwave/crawler/tiers/oxylabs_base.rb

Overview

Shared plumbing for the Oxylabs-backed tiers (datacenter render,
residential render, Amazon presets): client construction with the
Policy DI seam, the Retailer::DailyCostGuard pre-flight (a fatal
outcome — never escalate to a pricier tier when the budget is gone),
and response unwrapping.

All Oxylabs traffic keeps flowing through Retailer::OxylabsApi so the
budget guard and submit throttling remain the single choke point.

Subclasses implement #tier_name and #payload (or override
#attempt_fetch entirely, as the Amazon preset tier does).

Direct Known Subclasses

Amazon, Oxylabs, OxylabsResidential

Constant Summary collapse

OXYLABS_TIMEOUT = 
120

Instance Method Summary collapse

Constructor Details

#initialize(policy) ⇒ OxylabsBase

Returns a new instance of OxylabsBase.

Parameters:



20
21
22
 
# File 'app/services/heatwave/crawler/tiers/oxylabs_base.rb', line 20

def initialize(policy)
  @policy = policy
end

Instance Method Details

#applies_to?(_url) ⇒ Boolean

Returns:

  • (Boolean) 


24
 
# File 'app/services/heatwave/crawler/tiers/oxylabs_base.rb', line 24

def applies_to?(_url) = true

#attempt(url) ⇒ Heatwave::Crawler::FetchOutcome

Parameters:

  • url (String)

Returns:



28
29
30
 
# File 'app/services/heatwave/crawler/tiers/oxylabs_base.rb', line 28

def attempt(url)
  with_tier_errors(url) { attempt_fetch(url) }
end