Class: Retailer::ProbeHistoryUrlDiscovery

Inherits:
Object
  • Object
show all
Defined in:
app/services/retailer/probe_history_url_discovery.rb

Overview

Recovers a canonical product URL for a url-less catalog item from OUR OWN
past probe rows — no retailer request, no Oxylabs spend.

Why this exists: probes follow redirects and store the URL they actually
landed on. Back when the search-URL fallback was alive, a search for our
retailer part number frequently redirected to the real PDP, and that resolved
URL was written to CatalogItemRetailerProbe url — but never back onto the
catalog item. When PR #1480 killed the search fallback, those items lost the
only URL they had and silently dropped out of the daily run entirely
(ListingIssues::RetailerProbeAdapter reports them as probe_stale). The
resolved PDPs were sitting in our own database the whole time.

== The identity check

A resolved URL is only trustworthy if we can prove it points at OUR product,
and #1480's warning is the reason: a URL built from our own identifiers
matches itself tautologically. That warning applies with full force here —
Canadian Tire's resolved URLs carry a ?rq=774-8471-8 parameter that is
nothing but an echo of the search query WE sent, so a naive substring check
against the whole URL would "verify" every wrong-product landing.

So identity is asserted against the URL PATH ONLY (query and fragment are
stripped before matching), and the path must contain the retailer's own part
number for this item in one of three forms:

  1. verbatim — Home Depot: /p/{slug}/203363388
  2. separators stripped — 774-8471-8 → 77484718
  3. stripped, minus a trailing
    check digit — 774-8471-8 → 7748471, which is what
    Canadian Tire puts in -7748471p.html

Form 3 drops one character, so it is gated at MIN_KEY_LENGTH to keep the
match specific. An item with no third_party_part_number has nothing to
assert identity with and is skipped outright rather than guessed at.

Examples:

Recover one item's URL

Retailer::ProbeHistoryUrlDiscovery.new.discover_url(catalog_item)
#=> "https://www.homedepot.ca/product/warmlyyours-21-bar-vida.../1001318763"

See Also:

Constant Summary collapse

CANDIDATE_LIMIT =

Newest probe rows to consider. Product URLs are stable, so the most recent
resolved URL is the best candidate; the window only exists so an item with
thousands of probe rows doesn't pull them all.

200
MIN_KEY_LENGTH =

Shortest identity key we will accept. Guards form 3 (check digit dropped)
from degenerating into a match on a two-or-three digit fragment that could
appear anywhere in a slug.

6

Instance Method Summary collapse

Instance Method Details

#discover_url(catalog_item) ⇒ String?

Returns a canonical PDP URL proven to be this item's, or nil.

Parameters:

Returns:

  • (String, nil) —

    a canonical PDP URL proven to be this item's, or nil



57
58
59
60
61
62
 
# File 'app/services/retailer/probe_history_url_discovery.rb', line 57

def discover_url(catalog_item)
  keys = identity_keys(catalog_item)
  return nil if keys.empty?

  candidate_urls(catalog_item).find { |url| matches_identity?(url, keys) }
end