Class: Retailer::BatchPriceChecker

Inherits:
Object
  • Object
show all
Defined in:
app/services/retailer/batch_price_checker.rb

Overview

Batch price checker using Oxylabs Push-Pull API with webhook callbacks.
Submits jobs asynchronously with per-item callback URLs. Results are
processed by WebhookProcessors::OxylabsProcessor when webhooks arrive.

Creates pending WebhookLog entries for each submitted job to enable:

  • Audit trail of all submitted jobs
  • Detection of missed webhook callbacks
  • Retry logic via StaleTranscriptionRecoveryWorker

In development, use the dev tunnel (script/dev_tunnel_api.sh) to receive callbacks.

Examples:

Check all retailers

checker = Retailer::BatchPriceChecker.new
checker.check_all_retailers

Check a single catalog

checker = Retailer::BatchPriceChecker.new
checker.check_catalog(Catalog.find(18))

Constant Summary collapse

BATCH_SIZE =

Submit jobs in batches of 50

50
COSTCO_REQUEST_SPACING =

Seconds to wait between Costco API probes. Costco's price API
rate-limits request bursts; ~8s spacing keeps a full catalog run
within limits.

8

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ BatchPriceChecker

Returns a new instance of BatchPriceChecker.



32
33
34
35
36
37
 
# File 'app/services/retailer/batch_price_checker.rb', line 32

def initialize(options = {})
  @api = options[:api] || Retailer::OxylabsApi.new(timeout: 120)
  @url_constructor = options[:url_constructor] || Retailer::UrlConstructor.new
  @logger = options[:logger] || Rails.logger
  @price_checker = Retailer::PriceChecker.new(api: @api, url_constructor: @url_constructor, logger: @logger)
end

Instance Attribute Details

#api ⇒ Object (readonly)

Returns the value of attribute api.



30
31
32
 
# File 'app/services/retailer/batch_price_checker.rb', line 30

def api
  @api
end

#logger ⇒ Object (readonly)

Returns the value of attribute logger.



30
31
32
 
# File 'app/services/retailer/batch_price_checker.rb', line 30

def logger
  @logger
end

#url_constructor ⇒ Object (readonly)

Returns the value of attribute url_constructor.



30
31
32
 
# File 'app/services/retailer/batch_price_checker.rb', line 30

def url_constructor
  @url_constructor
end

Instance Method Details

#callback_url_for(catalog_item_id) ⇒ String

Generate a callback URL with a time-limited authentication token (24 hours)
Each job gets its own callback URL with the catalog_item_id embedded

Parameters:

  • catalog_item_id (Integer) —

    The catalog item ID to embed in the callback URL

Returns:

  • (String) —

    The callback URL based on environment:

    • Production: api.warmlyyours.com
    • Staging: api.warmlyyours.ws
    • Development: api-hostname.warmlyyours.dev (via Cloudflare tunnel)


46
47
48
49
50
51
52
53
 
# File 'app/services/retailer/batch_price_checker.rb', line 46

def callback_url_for(catalog_item_id)
  if Rails.env.development?
    Retailer::CallbackTokenService.dev_callback_url(catalog_item_id: catalog_item_id)
  else
    # Production and staging use API_HOSTNAME_WITHOUT_PORT constant
    Retailer::CallbackTokenService.callback_url(catalog_item_id: catalog_item_id)
  end
end

#check_all_retailers ⇒ Hash

Check all catalogs that have external_price_check_enabled
Jobs are submitted asynchronously - results arrive via webhook

Returns:

  • (Hash) —

    Summary of submitted jobs by catalog



58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'app/services/retailer/batch_price_checker.rb', line 58

def check_all_retailers
  catalogs = Catalog.where(external_price_check_enabled: true)
  results = { total_submitted: 0, catalogs: {} }

  catalogs.each do |catalog|
    submitted = check_catalog(catalog)

    results[:total_submitted] += submitted
    results[:catalogs][catalog.name] = submitted

    @logger.info "[BatchPriceChecker] #{catalog.name}: #{submitted} jobs submitted"
  end

  @logger.info "[BatchPriceChecker] Total: #{results[:total_submitted]} jobs submitted, awaiting callbacks"
  results
end

#check_catalog(catalog) ⇒ Integer

Check all items in a catalog using batch processing

Parameters:

Returns:

  • (Integer) —

    Number of jobs submitted



78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
 
# File 'app/services/retailer/batch_price_checker.rb', line 78

def check_catalog(catalog)
  items = catalog.catalog_items
                 .where(state: 'active')
                 .where(skip_url_checks: false)
                 .includes(:catalog)
                 .to_a

  return 0 if items.empty?

  # Per-catalog cadence: skip items whose last probe is newer than
  # `probe_cadence_days` ago. Default cadence is 1 day (no filter), but
  # slow-moving catalogs (Houzz, Costco, Lowes Canada) can be set to 3
  # or 7 to cut request volume proportionally. Items that have never
  # been probed (`url_last_checked: nil`) are always included.
  cadence_days = catalog.probe_cadence_days.to_i
  if cadence_days > 1
    original_count = items.size
    cutoff = cadence_days.days.ago
    items = items.reject { |item| item.url_last_checked.present? && item.url_last_checked > cutoff }
    skipped_count = original_count - items.size
    @logger.info "[BatchPriceChecker] Skipped #{skipped_count} items checked within last #{cadence_days}d (cadence=#{cadence_days}d) for #{catalog.name}" if skipped_count > 0
  end

  return 0 if items.empty?

  @logger.info "[BatchPriceChecker] Processing #{items.size} items for #{catalog.name}"

  # Costco uses synchronous JSON APIs (no Oxylabs job / webhook callback).
  # Probe the whole catalog in one pass so the inter-request spacing holds
  # continuously, instead of resetting at every 50-item slice boundary.
  return probe_costco_synchronously(items) if catalog.is_costco?

  # Process in batches and count total submitted
  items.each_slice(BATCH_SIZE).sum do |batch|
    process_batch(batch, catalog)
  end
end

#check_single_item(catalog_item) ⇒ Integer

Check a single catalog item
Only processes active items that aren't set to skip URL checks

Parameters:

Returns:

  • (Integer) —

    1 if submitted, 0 if skipped



120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
 
# File 'app/services/retailer/batch_price_checker.rb', line 120

def check_single_item(catalog_item)
  # Only check active items (skip pending_onboarding, phased_out, etc.)
  unless catalog_item.state == 'active'
    @logger.info "[BatchPriceChecker] Skipping item #{catalog_item.id} - state is '#{catalog_item.state}' (not active)"
    return 0
  end

  # Skip items explicitly marked to not check URLs
  if catalog_item.skip_url_checks?
    @logger.info "[BatchPriceChecker] Skipping item #{catalog_item.id} - skip_url_checks is true"
    return 0
  end

  @logger.info "[BatchPriceChecker] Processing single item #{catalog_item.id}"
  catalog = catalog_item.catalog
  return probe_costco_synchronously([catalog_item]) if catalog.is_costco?

  process_batch([catalog_item], catalog)
end