Class: Retailer::WayfairUrlDiscovery

Inherits:
Object
  • Object
show all
Includes:
CatalogConstants
Defined in:
app/services/retailer/wayfair_url_discovery.rb

Overview

Discovers Wayfair variant URLs for products.

Wayfair uses piid (Product Item ID) parameters in URLs to identify specific variants.
This service navigates to parent product pages and extracts variant data from
the embedded JSON, matching partNumbers to our internal SKUs.

Flow:

  1. Group catalog items by parent_sku
  2. For each parent, search by parent_sku to get the product page
  3. Extract variant JSON from page (contains partNumber and piid for each size)
  4. Match each variant's partNumber to our catalog item's SKU
  5. Store the complete URL (with piid) in catalog_item.url

This is efficient: one page load per parent product, not per variant.

Examples:

Run discovery for all Wayfair Canada variants

discovery = Retailer::WayfairUrlDiscovery.new(catalog_id: 21)
discovery.run

Run discovery for a specific parent SKU

discovery = Retailer::WayfairUrlDiscovery.new(catalog_id: 21)
discovery.run(parent_sku: 'WRM10160')

Constant Summary collapse

WAYFAIR_DOMAINS = 
{
  WAYFAIR_USA => 'www.wayfair.com',
  WAYFAIR_CANADA => 'www.wayfair.ca',
  WAYFAIR_GERMANY => 'www.wayfair.de'
}.freeze

Constants included from CatalogConstants

CatalogConstants::ALL_MAIN_CATALOG_IDS, CatalogConstants::AMAZON_CATALOG_IDS, CatalogConstants::AMAZON_CA_CATALOG_IDS, CatalogConstants::AMAZON_EU_CATALOG_IDS, CatalogConstants::AMAZON_NA_SELLER_IDS, CatalogConstants::AMAZON_SC_BE_CATALOG_ID, CatalogConstants::AMAZON_SC_CATALOG_IDS, CatalogConstants::AMAZON_SC_CA_CATALOG_ID, CatalogConstants::AMAZON_SC_DE_CATALOG_ID, CatalogConstants::AMAZON_SC_ES_CATALOG_ID, CatalogConstants::AMAZON_SC_FR_CATALOG_ID, CatalogConstants::AMAZON_SC_IT_CATALOG_ID, CatalogConstants::AMAZON_SC_NL_CATALOG_ID, CatalogConstants::AMAZON_SC_PL_CATALOG_ID, CatalogConstants::AMAZON_SC_SE_CATALOG_ID, CatalogConstants::AMAZON_SC_UK_CATALOG_ID, CatalogConstants::AMAZON_SC_US_CATALOG_ID, CatalogConstants::AMAZON_SELLER_IDS, CatalogConstants::AMAZON_US_CATALOG_IDS, CatalogConstants::AMAZON_VC_CATALOG_IDS, CatalogConstants::AMAZON_VC_CA_CATALOG_ID, CatalogConstants::AMAZON_VC_CA_CATALOG_IDS, CatalogConstants::AMAZON_VC_DIRECT_FULFILLMENT_CATALOG_IDS, CatalogConstants::AMAZON_VC_US_CATALOG_IDS, CatalogConstants::AMAZON_VC_US_WASN4_CATALOG_ID, CatalogConstants::AMAZON_VC_US_WAX7V_CATALOG_ID, CatalogConstants::AMAZON_VC_WAT0F_CA_CATALOG_ID, CatalogConstants::AMAZON_VC_WAT4D_CA_CATALOG_ID, CatalogConstants::AMAZON_VENDOR_CODE_TO_CATALOG_ID, CatalogConstants::BESTBUY_CANADA, CatalogConstants::BUILD_COM, CatalogConstants::CANADIAN_TIRE, CatalogConstants::CA_CATALOG_ID, CatalogConstants::COSTCO_CANADA, CatalogConstants::COSTCO_CATALOGS, CatalogConstants::COSTCO_USA, CatalogConstants::EU_CATALOG_ID, CatalogConstants::HOME_DEPOT_CANADA, CatalogConstants::HOME_DEPOT_CATALOGS, CatalogConstants::HOME_DEPOT_USA, CatalogConstants::HOUZZ, CatalogConstants::LOCALE_TO_CATALOG, CatalogConstants::LOWES_CANADA, CatalogConstants::LOWES_USA, CatalogConstants::MARKETPLACE_CATALOGS, CatalogConstants::PRICE_CHECK_ENABLED_CATALOGS, CatalogConstants::RONA_CANADA, CatalogConstants::US_CATALOG_ID, CatalogConstants::VENDOR_CATALOGS, CatalogConstants::WALMART_CATALOGS, CatalogConstants::WALMART_SELLER_CANADA, CatalogConstants::WALMART_SELLER_USA, CatalogConstants::WAYFAIR_CANADA, CatalogConstants::WAYFAIR_CATALOGS, CatalogConstants::WAYFAIR_GERMANY, CatalogConstants::WAYFAIR_USA

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from CatalogConstants

amazon_catalog?, amazon_seller_catalog?, costco_catalog?, home_depot_catalog?, marketplace_catalog?, price_check_enabled?, vendor_catalog?, walmart_catalog?, wayfair_catalog?

Constructor Details

#initialize(catalog_id:, logger: Rails.logger) ⇒ WayfairUrlDiscovery

Returns a new instance of WayfairUrlDiscovery.

Raises:

  • (ArgumentError) 


37
38
39
40
41
42
43
44
45
46
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 37

def initialize(catalog_id:, logger: Rails.logger)
  @catalog_id = catalog_id
  @catalog = Catalog.find(catalog_id)
  @domain = WAYFAIR_DOMAINS[catalog_id]
  @logger = logger
  @results = { discovered: 0, failed: 0, skipped: 0, errors: [] }
  @oxylabs_api = Retailer::OxylabsApi.new

  raise ArgumentError, "Unsupported catalog: #{catalog_id}" unless @domain
end

Instance Attribute Details

#catalog ⇒ Object (readonly)

Returns the value of attribute catalog.



35
36
37
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 35

def catalog
  @catalog
end

#catalog_id ⇒ Object (readonly)

Returns the value of attribute catalog_id.



35
36
37
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 35

def catalog_id
  @catalog_id
end

#domain ⇒ Object (readonly)

Returns the value of attribute domain.



35
36
37
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 35

def domain
  @domain
end

#logger ⇒ Object (readonly)

Returns the value of attribute logger.



35
36
37
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 35

def logger
  @logger
end

#oxylabs_api ⇒ Object (readonly)

Returns the value of attribute oxylabs_api.



35
36
37
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 35

def oxylabs_api
  @oxylabs_api
end

#results ⇒ Object (readonly)

Returns the value of attribute results.



35
36
37
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 35

def results
  @results
end

Instance Method Details

#run(parent_sku: nil, dry_run: false) ⇒ Hash

Run discovery for all variant items or a specific parent

Parameters:

  • parent_sku (String, nil) (defaults to: nil) —

    If provided, only process variants of this parent

  • dry_run (Boolean) (defaults to: false) —

    If true, don't save changes to database

Returns:

  • (Hash) —

    Results summary



53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'app/services/retailer/wayfair_url_discovery.rb', line 53

def run(parent_sku: nil, dry_run: false)
  @dry_run = dry_run

  # Load variants and group by parent_sku
  variants = load_variants(parent_sku)
  grouped = variants.group_by(&:parent_sku)

  logger.info "[WayfairUrlDiscovery] Processing #{grouped.size} parent products, #{variants.size} variants"

  # Process each parent product group
  grouped.each_with_index do |(parent, items), index|
    process_parent_group(parent, items)
    sleep(2) if index < grouped.size - 1 # 2 second delay between parent requests
  end

  logger.info "[WayfairUrlDiscovery] Complete: #{results}"
  results
end