Module: Retailer::UrlRediscovery

Defined in:
app/services/retailer/url_rediscovery.rb

Overview

Failure-triggered product-URL rediscovery, shared across retailers.

Catalog product URLs are stable, so we discover them once (per-retailer
backfill) rather than on a schedule. But a URL can go stale when a retailer
renames a slug or re-homes a product, and the probe then fails. This is the
"on failure, re-find the URL" path: when a probe comes back as a URL-type
failure, WebhookResultProcessor enqueues
RetailerUrlRediscoveryWorker, which calls UrlRediscovery.attempt here to cheaply re-find
the URL (no Oxylabs spend) and re-probe.

Each retailer plugs in a strategy object responding to
#discover_url(catalog_item) -> String | nil. Strategies are tried in order
and the first non-blank URL wins, so they are listed cheapest-first: every
retailer starts with ProbeHistoryUrlDiscovery, which reads URLs we
already resolved and stored on past probe rows and therefore costs nothing at
all, before any strategy that talks to the retailer.

The same strategies drive the one-time backfill of url-less items — see
UrlBackfill — so a retailer added here is repaired in bulk and
kept repaired on failure by the same code.

Class Method Summary collapse

Class Method Details

.attempt(catalog_item) ⇒ Boolean

Re-find and persist a fresh product URL for the item, if any strategy finds
a different one.

Parameters:

Returns:

  • (Boolean) —

    true when a new, different URL was found and saved



31
32
33
34
35
36
37
38
 
# File 'app/services/retailer/url_rediscovery.rb', line 31

def attempt(catalog_item)
  url = discover(catalog_item)
  return false if url.blank? || url == catalog_item.url

  catalog_item.update_columns(url: url, url_last_checked: Time.current)
  Rails.logger.info "[UrlRediscovery] #{catalog_item.id} (#{catalog_item.catalog&.name}): url -> #{url}"
  true
end

.available_for?(catalog_item) ⇒ Boolean

Whether the item's retailer has any rediscovery strategy (gates the trigger
so the hot probe path stays a no-op for retailers without one).

Parameters:

Returns:

  • (Boolean) 


54
55
56
 
# File 'app/services/retailer/url_rediscovery.rb', line 54

def available_for?(catalog_item)
  strategies_for(catalog_item).any?
end

.discover(catalog_item) ⇒ String?

First URL any strategy can prove belongs to this item, without persisting —
the read-only half of attempt, so a backfill can preview before writing.

Parameters:

Returns:

  • (String, nil) 


45
46
47
 
# File 'app/services/retailer/url_rediscovery.rb', line 45

def discover(catalog_item)
  strategies_for(catalog_item).lazy.filter_map { |s| s.discover_url(catalog_item).presence }.first
end

.retailer_strategies(catalog_id) ⇒ Array<Object>

Parameters:

  • catalog_id (Integer)

Returns:

  • (Array<Object>) 


72
73
74
75
76
77
78
79
 
# File 'app/services/retailer/url_rediscovery.rb', line 72

def retailer_strategies(catalog_id)
  case catalog_id
  when CatalogConstants::RONA_CANADA then [Retailer::RonaUrlDiscovery.new]
  when CatalogConstants::BESTBUY_CANADA then [Retailer::BestBuyUrlDiscovery.new]
  when *Retailer::SearchUrlDiscovery::SEARCH_URL_BUILDERS.keys then [Retailer::SearchUrlDiscovery.new]
  else []
  end
end

.strategies_for(catalog_item) ⇒ Array<Object>

Probe-history recovery leads for EVERY retailer, including ones with no
entry in retailer_strategies — it reads URLs we already resolved and
stored on past probe rows, so it costs nothing and needs no per-retailer
support. That is also why available_for? is true everywhere: any item
with a resolvable history is worth a rediscovery attempt.

Parameters:

Returns:

  • (Array<Object>) —

    strategies responding to #discover_url, cheapest first



66
67
68
 
# File 'app/services/retailer/url_rediscovery.rb', line 66

def strategies_for(catalog_item)
  [Retailer::ProbeHistoryUrlDiscovery.new, *retailer_strategies(catalog_item.catalog_id)].compact
end