Class: Retailer::UrlBackfill

Inherits:
Object
  • Object
show all
Defined in:
app/services/retailer/url_backfill.rb

Overview

One-time repair for catalog items that have no product URL and are therefore
never probed at all.

Retailer::UrlConstructor#product_url returns nil when an item has no stored
URL and no safely-synthesizable PDP, BatchPriceChecker drops the
nil payload without a trace, and the item silently leaves the daily run
holding whatever price its last probe wrote — see
ListingIssues::RetailerProbeAdapter's probe_stale issue, which is the
detector for exactly this state. This is the repair half.

It runs the same UrlRediscovery strategies the on-failure path
uses, so a retailer taught to rediscover URLs is repaired in bulk here and
stays repaired there — there is no second, backfill-only notion of "find the
URL" to keep in sync.

Strategy objects are built once per catalog, not once per item: Best Buy's
fetches a product catalogue and rona's fetches sitemaps, and rebuilding those
per item would turn a 240-item catalog into 240 redundant round trips.

Examples:

Preview what would be written

Retailer::UrlBackfill.new.run(dry_run: true).report

Repair one catalog

Retailer::UrlBackfill.new(catalog_ids: [CatalogConstants::BESTBUY_CANADA]).run

Defined Under Namespace

Classes: Outcome, Result

Instance Method Summary collapse

Constructor Details

#initialize(catalog_ids: nil, logger: Rails.logger) ⇒ UrlBackfill

Returns a new instance of UrlBackfill.

Parameters:

  • catalog_ids (Array<Integer>, nil) (defaults to: nil) —

    restrict to these catalogs (nil = all probed)

  • logger (Logger) (defaults to: Rails.logger) 


58
59
60
61
 
# File 'app/services/retailer/url_backfill.rb', line 58

def initialize(catalog_ids: nil, logger: Rails.logger)
  @catalog_ids = catalog_ids
  @logger = logger
end

Instance Method Details

#items ⇒ Array<CatalogItem>

Active, probe-eligible items with nothing for the probe to fetch. Mirrors
BatchPriceChecker#check_catalog's own filters (active + not auto-skipped) so
the repaired set is exactly the set that would then be probed.

Items whose URL is synthesizable from an identifier we already hold (Costco
and Walmart build a deterministic PDP from third_party_part_number) are
excluded — a blank url is not a defect for them, and writing one would
freeze a URL that UrlConstructor otherwise keeps current.

Returns:



84
85
86
87
88
89
 
# File 'app/services/retailer/url_backfill.rb', line 84

def items
  scope = CatalogItem.where(url: [nil, ''], state: 'active', skip_url_checks: false)
                     .joins(:catalog).where(catalogs: { external_price_check_enabled: true })
  scope = scope.where(catalog_id: @catalog_ids) if @catalog_ids
  scope.includes(:catalog).reject { |item| synthesizable?(item) }
end

#run(dry_run: false) ⇒ Result

Parameters:

  • dry_run (Boolean) (defaults to: false) —

    when true, discover and report but write nothing

Returns:



65
66
67
68
69
70
71
72
 
# File 'app/services/retailer/url_backfill.rb', line 65

def run(dry_run: false)
  outcomes = items.group_by(&:catalog_id).flat_map do |_catalog_id, group|
    strategies = Retailer::UrlRediscovery.strategies_for(group.first)
    group.map { |item| process(item, strategies, dry_run) }
  end

  Result.new(outcomes: outcomes, dry_run: dry_run)
end