Class: Heatwave::Crawler::Policy

Inherits:
Data
  • Object
show all
Defined in:
app/services/heatwave/crawler/policy.rb

Overview

Per-caller configuration for fetch. Immutable.

See Also:

  • doc/tasks/202607181233_SUPERCRAWLERdoc/tasks/202607181233_SUPERCRAWLER.md

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(max_chars: nil, timeout_seconds: 15, user_agent: nil, follow_redirects: true, http_method: :get, tiers: nil, skip_tiers: [], allow_private_hosts: nil, capture_screenshot: nil, residential_geo: 'United States', oxylabs_api: nil, web_unblocker_api: nil) ⇒ Policy

All fields are optional; defaults match the pre-Policy behavior of
the modes they configure. See the field comments above.

Parameters:

  • max_chars (Integer, nil) (defaults to: nil)
  • timeout_seconds (Integer) (defaults to: 15)
  • user_agent (String, nil) (defaults to: nil)
  • follow_redirects (Boolean) (defaults to: true)
  • http_method (Symbol) (defaults to: :get)
  • tiers (Array<Symbol>, nil) (defaults to: nil)
  • skip_tiers (Array<Symbol>) (defaults to: [])
  • allow_private_hosts (Boolean, nil) (defaults to: nil)
  • capture_screenshot (Boolean, nil) (defaults to: nil)
  • residential_geo (String) (defaults to: 'United States')
  • oxylabs_api (Retailer::OxylabsApi, nil) (defaults to: nil)
  • web_unblocker_api (Retailer::WebUnblockerApi, nil) (defaults to: nil) 


36
37
38
39
40
41
42
 
# File 'app/services/heatwave/crawler/policy.rb', line 36

def initialize(max_chars: nil, timeout_seconds: 15, user_agent: nil,
               follow_redirects: true, http_method: :get, tiers: nil,
               skip_tiers: [], allow_private_hosts: nil,
               capture_screenshot: nil, residential_geo: 'United States',
               oxylabs_api: nil, web_unblocker_api: nil)
  super
end

Instance Attribute Details

#allow_private_hosts ⇒ Object (readonly)

Returns the value of attribute allow_private_hosts

Returns:

  • (Object) —

    the current value of allow_private_hosts



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def allow_private_hosts
  @allow_private_hosts
end

#capture_screenshot ⇒ Object (readonly)

Returns the value of attribute capture_screenshot

Returns:

  • (Object) —

    the current value of capture_screenshot



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def capture_screenshot
  @capture_screenshot
end

#follow_redirects ⇒ Object (readonly)

Returns the value of attribute follow_redirects

Returns:

  • (Object) —

    the current value of follow_redirects



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def follow_redirects
  @follow_redirects
end

#http_method ⇒ Object (readonly)

Returns the value of attribute http_method

Returns:

  • (Object) —

    the current value of http_method



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def http_method
  @http_method
end

#max_chars ⇒ Object (readonly)

Returns the value of attribute max_chars

Returns:

  • (Object) —

    the current value of max_chars



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def max_chars
  @max_chars
end

#oxylabs_api ⇒ Object (readonly)

Returns the value of attribute oxylabs_api

Returns:

  • (Object) —

    the current value of oxylabs_api



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def oxylabs_api
  @oxylabs_api
end

#residential_geo ⇒ Object (readonly)

Returns the value of attribute residential_geo

Returns:

  • (Object) —

    the current value of residential_geo



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def residential_geo
  @residential_geo
end

#skip_tiers ⇒ Object (readonly)

Returns the value of attribute skip_tiers

Returns:

  • (Object) —

    the current value of skip_tiers



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def skip_tiers
  @skip_tiers
end

#tiers ⇒ Object (readonly)

Returns the value of attribute tiers

Returns:

  • (Object) —

    the current value of tiers



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def tiers
  @tiers
end

#timeout_seconds ⇒ Object (readonly)

Returns the value of attribute timeout_seconds

Returns:

  • (Object) —

    the current value of timeout_seconds



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def timeout_seconds
  @timeout_seconds
end

#user_agent ⇒ Object (readonly)

Returns the value of attribute user_agent

Returns:

  • (Object) —

    the current value of user_agent



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def user_agent
  @user_agent
end

#web_unblocker_api ⇒ Object (readonly)

Returns the value of attribute web_unblocker_api

Returns:

  • (Object) —

    the current value of web_unblocker_api



7
8
9
 
# File 'app/services/heatwave/crawler/policy.rb', line 7

def web_unblocker_api
  @web_unblocker_api
end