Class: Retailer::UrlConstructor

Inherits:
Object
  • Object
show all
Includes:
CatalogConstants
Defined in:
app/services/retailer/url_constructor.rb

Overview

Constructs product URLs for retailer catalog items dynamically.
Prioritizes hardcoded URLs but can generate URLs from product identifiers.

Examples:

Get URL for a catalog item

constructor = Retailer::UrlConstructor.new
url = constructor.product_url(catalog_item)

Reset hardcoded URL to generated

catalog_item.update!(url: constructor.generate_url(catalog_item))

Constant Summary collapse

CATALOG_BUILDERS =

Catalog ID to URL builder method mapping
Uses CatalogConstants for maintainability

Two kinds of entry. A *_url builder synthesizes a DETERMINISTIC direct PDP
from an identifier the retailer itself assigned us (their item number, our
ASIN); stored_url_only means the retailer's only synthesizable fallback was
a search-results page, which is the hollow-probe / wrong-match risk
#generate_url exists to block — for those, probe the stored canonical PDP or
nothing.

Home Depot, Canadian Tire and Best Buy sat in the second group by mistake.
They do expose search endpoints keyed by the same id (/s/{id},
/search?q={id}), but they ALSO expose a direct PDP keyed by it, and the
slug in that PDP path is cosmetic — all three serve the correct product from
the trailing id with a placeholder slug (verified live 2026-08-08). Treating
them as search-only left 348 active items with no probeable URL at all: they
were dropped silently by BatchPriceChecker and froze at their last probed
price. A direct PDP keyed by the retailer's own id is exactly the fallback
#generate_url's gate is meant to let through.

{
  # Home Depot — direct `/p/{slug}/{id}` (US) and `/product/{slug}/{id}` (CA) PDPs
  HOME_DEPOT_USA => :home_depot_usa_url,
  HOME_DEPOT_CANADA => :home_depot_canada_url,
  # Costco — direct `/.product.{id}.html` PDP (also probed via its JSON API)
  COSTCO_CANADA => :costco_canada_url,
  # Wayfair
  WAYFAIR_USA => :wayfair_url,
  WAYFAIR_CANADA => :wayfair_url,
  WAYFAIR_GERMANY => :wayfair_url,
  # Rona / Lowe's — both search-only fallbacks
  RONA_CANADA => :rona_url,
  LOWES_USA => :stored_url_only,
  LOWES_CANADA => :stored_url_only,
  # Build.com (Ferguson Home) — search-only fallback
  BUILD_COM => :stored_url_only,
  # Amazon Seller Marketplaces — direct `/dp/{asin}` PDP
  AMAZON_SC_US_CATALOG_ID => :amazon_url,
  AMAZON_SC_CA_CATALOG_ID => :amazon_url,
  # Amazon European Marketplaces
  AMAZON_SC_FR_CATALOG_ID => :amazon_url,
  AMAZON_SC_IT_CATALOG_ID => :amazon_url,
  AMAZON_SC_ES_CATALOG_ID => :amazon_url,
  AMAZON_SC_DE_CATALOG_ID => :amazon_url,
  AMAZON_SC_NL_CATALOG_ID => :amazon_url,
  AMAZON_SC_PL_CATALOG_ID => :amazon_url,
  AMAZON_SC_UK_CATALOG_ID => :amazon_url,
  AMAZON_SC_SE_CATALOG_ID => :amazon_url,
  AMAZON_SC_BE_CATALOG_ID => :amazon_url,
  # Walmart Seller Marketplaces — direct `/ip/product/{id}` PDP
  WALMART_SELLER_USA => :walmart_usa_url,
  WALMART_SELLER_CANADA => :walmart_canada_url,
  # Canadian Tire — direct `/en/pdp/{slug}-{code}p.html` PDP
  CANADIAN_TIRE => :canadian_tire_url,
  # Best Buy Canada — direct `/en-ca/product/{slug}/{sku}` PDP
  BESTBUY_CANADA => :bestbuy_canada_url
}.freeze
PLACEHOLDER_SLUG =

Slug segment used wherever a retailer's PDP path carries a cosmetic slug it
resolves the product without. Matches the existing Walmart builders'
/ip/product/{id}, so every synthesized URL in this class reads the same.

'product'
SAFE_IDENTIFIER =

Shape a retailer product id may take before we will interpolate it into a
URL path. Every id we actually receive is alphanumeric with at most dashes,
dots or underscores (Home Depot "203363388", Walmart "4KRAWZGZCAO2",
Canadian Tire "774-8471-8").

A value carrying /, ?, # or whitespace would not be escaped by string
interpolation — it would silently restructure the URL and point the probe at
some other page, whose price then feeds repricing and MAP reporting. Bad
data yields no URL rather than a plausible wrong one, which leaves the item
visibly unprobed (probe_stale) instead of quietly mispriced.

/\A[A-Za-z0-9._-]+\z/

Constants included from CatalogConstants

CatalogConstants::ALL_MAIN_CATALOG_IDS, CatalogConstants::AMAZON_CATALOG_IDS, CatalogConstants::AMAZON_CA_CATALOG_IDS, CatalogConstants::AMAZON_EU_CATALOG_IDS, CatalogConstants::AMAZON_NA_SELLER_IDS, CatalogConstants::AMAZON_SC_BE_CATALOG_ID, CatalogConstants::AMAZON_SC_CATALOG_IDS, CatalogConstants::AMAZON_SC_CA_CATALOG_ID, CatalogConstants::AMAZON_SC_DE_CATALOG_ID, CatalogConstants::AMAZON_SC_ES_CATALOG_ID, CatalogConstants::AMAZON_SC_FR_CATALOG_ID, CatalogConstants::AMAZON_SC_IT_CATALOG_ID, CatalogConstants::AMAZON_SC_NL_CATALOG_ID, CatalogConstants::AMAZON_SC_PL_CATALOG_ID, CatalogConstants::AMAZON_SC_SE_CATALOG_ID, CatalogConstants::AMAZON_SC_UK_CATALOG_ID, CatalogConstants::AMAZON_SC_US_CATALOG_ID, CatalogConstants::AMAZON_SELLER_IDS, CatalogConstants::AMAZON_US_CATALOG_IDS, CatalogConstants::AMAZON_VC_CATALOG_IDS, CatalogConstants::AMAZON_VC_CA_CATALOG_ID, CatalogConstants::AMAZON_VC_CA_CATALOG_IDS, CatalogConstants::AMAZON_VC_DIRECT_FULFILLMENT_CATALOG_IDS, CatalogConstants::AMAZON_VC_US_CATALOG_IDS, CatalogConstants::AMAZON_VC_US_WASN4_CATALOG_ID, CatalogConstants::AMAZON_VC_US_WAX7V_CATALOG_ID, CatalogConstants::AMAZON_VC_WAT0F_CA_CATALOG_ID, CatalogConstants::AMAZON_VC_WAT4D_CA_CATALOG_ID, CatalogConstants::AMAZON_VENDOR_CODE_TO_CATALOG_ID, CatalogConstants::BESTBUY_CANADA, CatalogConstants::BUILD_COM, CatalogConstants::CANADIAN_TIRE, CatalogConstants::CA_CATALOG_ID, CatalogConstants::COSTCO_CANADA, CatalogConstants::COSTCO_CATALOGS, CatalogConstants::COSTCO_USA, CatalogConstants::EU_CATALOG_ID, CatalogConstants::HOME_DEPOT_CANADA, CatalogConstants::HOME_DEPOT_CATALOGS, CatalogConstants::HOME_DEPOT_USA, CatalogConstants::HOUZZ, CatalogConstants::LOCALE_TO_CATALOG, CatalogConstants::LOWES_CANADA, CatalogConstants::LOWES_USA, CatalogConstants::RONA_CANADA, CatalogConstants::US_CATALOG_ID, CatalogConstants::WALMART_CATALOGS, CatalogConstants::WALMART_SELLER_CANADA, CatalogConstants::WALMART_SELLER_USA, CatalogConstants::WAYFAIR_CANADA, CatalogConstants::WAYFAIR_CATALOGS, CatalogConstants::WAYFAIR_GERMANY, CatalogConstants::WAYFAIR_USA

Instance Method Summary collapse

Methods included from CatalogConstants

amazon_catalog?, amazon_seller_catalog?, costco_catalog?, home_depot_catalog?, walmart_catalog?, wayfair_catalog?

Instance Method Details

#amazon_domain(catalog) ⇒ String

Returns the Amazon domain for a catalog based on its marketplace

Parameters:

Returns:

  • (String) —

    Domain like 'amazon.com' or 'amazon.ca'



137
138
139
140
141
142
143
144
145
 
# File 'app/services/retailer/url_constructor.rb', line 137

def amazon_domain(catalog)
  marketplace = catalog.amazon_marketplace
  return 'amazon.com' unless marketplace&.url

  # Extract domain from marketplace URL (https://www.amazon.com -> amazon.com)
  URI.parse(marketplace.url).host.gsub(/^www\./, '')
rescue URI::InvalidURIError
  'amazon.com'
end

#can_generate_url?(catalog) ⇒ Boolean

Check if we can generate a URL for this catalog

Parameters:

Returns:

  • (Boolean) 


129
130
131
 
# File 'app/services/retailer/url_constructor.rb', line 129

def can_generate_url?(catalog)
  CATALOG_BUILDERS.key?(catalog.id)
end

#generate_url(catalog_item) ⇒ String?

Generates a URL from product identifiers (ASIN, third_party_part_number, UPC)
Does not use the hardcoded URL field

Parameters:

Returns:

  • (String, nil) 


103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
 
# File 'app/services/retailer/url_constructor.rb', line 103

def generate_url(catalog_item)
  catalog = catalog_item.catalog
  return nil unless catalog

  # Find matching retailer builder by catalog ID
  builder_method = CATALOG_BUILDERS[catalog.id]
  return nil unless builder_method

  url = send(builder_method, catalog_item, catalog)

  # Hard invariant: a *synthesized* fallback URL must resolve to a single
  # product page, never a search-results page. A search URL is built from our
  # own identifiers, so the probe's identity check matches it tautologically
  # and accepts whatever price the results grid happens to render — that fed
  # wrong-product prices into MAP-violation reporting and hollow "no data"
  # probes that burned Oxylabs budget (found 2026-07-09). Stored canonical
  # URLs bypass this in #product_url; they're trusted as-is. Only a
  # deterministic direct-PDP fallback (Costco / Walmart item id, Amazon ASIN)
  # survives this gate; everything else falls through to nil = not probed.
  Retailer::UrlPatterns.direct_product_url?(url) ? url : nil
end

#product_url(catalog_item) ⇒ String?

Returns the best URL for a catalog item
Prefers hardcoded URL if present, otherwise generates from identifiers

Parameters:

Returns:

  • (String, nil) 


94
95
96
 
# File 'app/services/retailer/url_constructor.rb', line 94

def product_url(catalog_item)
  catalog_item.url.presence || generate_url(catalog_item)
end