Class: EdiStalledFeedSweep

Inherits:
Object
  • Object
show all
Includes:
Sidekiq::Job
Defined in:
app/workers/edi_stalled_feed_sweep.rb

Overview

Hourly sweep for EDI feeds that were BUILT but never SENT.

The gap this closes: a sender that RAISES moves its EdiCommunicationLog to
exception, which emails the EDI admin from the state machine. A sender that
HANGS moves nothing — the ECL sits in ready with NULL notes and
updated_at == created_at forever, and no layer notices. Menard sat like that
from 2026-07-13 to 08-03 (doc/troubleshooting/MENARD_PLAYWRIGHT_OUTAGE_2026_07.md),
and the sweep that found it was a human running a GROUP BY three weeks late.

Alerts per partner+category GROUP, not per row: one wedged sender produces
hundreds of identical rows, and the actionable fact is "this feed stopped",
not each snapshot it failed to ship. Gated on a Rails.cache flag per group so
a known-broken feed re-nudges every ALERT_CACHE_TTL rather than hourly —
the MissedFreightPickupSweep convention.

Scheduled hourly via sidekiq-scheduler — see
config/sidekiq_production_schedule.yml. PRODUCTION ONLY: staging runs no EDI
flows, so every row in a prod-restored edi_communication_logs there would
look stalled.

See Also:

Constant Summary collapse

STALE_AFTER =

How long a feed may sit in each state before it counts as stalled, and which
column dates its ARRIVAL in that state.

ready — staged for a sender, which is what the insert records, so
created_at is the arrival time. Every EDI flow runs at least daily and most
run hourly, so 6h without a state change means nobody picked it up.

processing — handed to the partner, awaiting their result. Measured from
updated_at, not created_at: a log can sit in ready for days before a
sender takes it, and dating that from creation would alert the instant it was
finally picked up. The state machine saves on transition, so updated_at is
the arrival time; an unrelated later write can only push it forward, which
delays an alert rather than inventing one. Walmart sets the threshold — it
calls its own feeds abandoned after failure_timeout_in_minutes (24h), so
past 26h a feed is stalled by Walmart's definition, not ours.

{
  'ready'      => { age: 6.hours,  column: :created_at },
  'processing' => { age: 26.hours, column: :updated_at }
}.freeze
ALERT_CACHE_TTL =

Constant.

3.days
UNDELIVERED_CATEGORIES =

Categories this sweep must ignore, because a leftover ready row in them is
DEBRIS, not a pending feed.

These are inbound retrieval records. Their processor takes the ECL as an
ARGUMENT (ListingItemInformationProcessor#process(ecl, locale:, …)) and runs
inline in the same pass that retrieved it — nothing anywhere scans for ready
rows in these categories, so one that survives an interrupted pass will sit
there forever and this sweep would shout about it forever. An interrupted
12-hour batch on 2026-07-28 left 2,939 such rows behind, which is 92% of
everything the sweep currently reports.

Replaying them is not the answer either: the processor writes
catalog_item.update_columns(retailer_information: …, amazon_info_datetime: Time.current), so a stale row would overwrite current Amazon state AND stamp
it as fresh.

ponytail: a denylist, not a derived "has a queue-draining consumer" check —
the set is three categories and stable, and deriving it is real machinery for
no gain. The better fix is upstream: have the inline pass archive its own ECL
on completion, so a leftover ready genuinely means "this pass died" and can
be swept like everything else. Delete this constant when that lands.

%w[
  listing_item_information
  catalog_item_information
  listing_item_schema
].freeze

Instance Method Summary collapse

Instance Method Details

#perform ⇒ Object

Runs the job.

Returns:

  • (Object) —

    the result



82
83
84
85
86
87
88
89
90
 
# File 'app/workers/edi_stalled_feed_sweep.rb', line 82

def perform
  STALE_AFTER.each do |state, rule|
    stalled_groups(state, rule).each do |(partner, category), count, oldest|
      next unless claim_alert!(state, partner, category)

      alert!(state, partner, category, count, oldest)
    end
  end
end