Class: Retailer::Extractors::CanadianTire

Inherits:
Base
  • Object
show all
Defined in:
app/services/retailer/extractors/canadian_tire.rb

Overview

Canadian Tire data extractor.

URL format: https://www.canadiantire.ca/en/pdp/{product-slug}-{code}p.html

Reads the schema.org Product JSON-LD the PDP embeds, which carries everything
we need and is stable across their client-side rendering:

"offers": { "price": "739.99", "priceCurrency": "CAD",
"availability": "InStock" }
"sku": "7748612"

That sku is Canadian Tire's own product code — the same one
UrlConstructor#canadian_tire_url derives from our
third_party_part_number ("774-8612-4" → "7748612") — so it doubles as an
identity anchor the page asserts about itself.

== Why this was returning no price at all

Canadian Tire had never yielded a single price in its probe history. The
extractor dispatched on search_results_page?, which tested for the substring
"search-results" anywhere in the body — and a normal PDP contains it three
times, in nav links and scripts. So every real product page took the
search-results branch, looked for product cards that aren't there, and set
product_available = false with a nil price.

The search branch is gone rather than fixed. Since PR #1480 a search URL is
never probed — UrlPatterns rejects one and
UrlConstructor won't synthesize one — so that branch was
unreachable by design and only ever misfired on genuine PDPs.

Constant Summary collapse

RENDER_REQUIRED =

Canadian Tire PDP loads pricing client-side. Keep on.

true
WEB_UNBLOCKER_FALLBACK =

Retry an empty Scraper API response through the Web Unblocker.

Canadian Tire intermittently returns nothing at all: of the 40 items in the
first run after the extractor was fixed, 33 priced and the other 7 recorded
zero bytes — re-fetching those same URLs by hand priced them fine. An empty
body is exactly what the fallback exists for.

true
PRICE_SELECTORS =

Current price markup, checked only when the JSON-LD is missing or malformed.

[
  '[data-testid="priceTotal"]',
  '.nl-price--total',
  '[data-testid="price-value"]',
  '[itemprop="price"]'
].freeze

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.build_payload(url:) ⇒ Hash

Build Oxylabs payload for Canadian Tire product scraping
Uses 'universal' source with JS rendering.

Parameters:

  • url (String) —

    Full product URL

Returns:

  • (Hash) —

    Oxylabs API payload



67
68
69
70
71
72
73
74
75
76
 
# File 'app/services/retailer/extractors/canadian_tire.rb', line 67

def self.build_payload(url:)
  {
    source: 'universal',
    url: url,
    render: render_value,
    context: [
      { key: 'follow_redirects', value: true }
    ]
  }.compact
end

.storefront_geo(_catalog) ⇒ String

Canadian Tire is a Canadian storefront, so the fallback must egress from
Canada — otherwise Web Unblocker's default US state rotation would fetch
(or be refused) from the wrong country. Country-level is also all the
unblocker honours for a generic target, and all Canadian Tire needs: unlike
Home Depot it does not price per store.

Parameters:

Returns:

  • (String) 


52
 
# File 'app/services/retailer/extractors/canadian_tire.rb', line 52

def self.storefront_geo(_catalog) = 'Canada'

Instance Method Details

#catalog_base_url ⇒ Object (protected) 



99
100
101
 
# File 'app/services/retailer/extractors/canadian_tire.rb', line 99

def catalog_base_url
  'https://www.canadiantire.ca'
end

#extract(check, content) ⇒ void

This method returns an undefined value.

Parameters:



81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'app/services/retailer/extractors/canadian_tire.rb', line 81

def extract(check, content)
  return unless valid_html?(content)

  check.scraper_source = source_name
  check.currency = 'CAD'

  doc = parse_html(content)
  @discovered_url = extract_canonical_url(doc)
  offer = product_offer(doc)

  extract_price(check, doc, offer)
  check.product_available = availability(offer, check)
  check.retailer_sku = product_node(doc)&.dig('sku').presence
  check.raw_title = extract_title(doc)
end