Class: ShipmentEvent

Inherits:
ApplicationRecord show all
Defined in:
app/models/shipment_event.rb

Overview

Typed projection of ShipEngine tracking webhook events into indexed
columns. One row per carrier scan emitted in the data.events[] array
of each tracking webhook (POST /webhooks/v1/shipengine).

Populated by WebhookProcessors::ShipengineProcessor from WebhookLog
records ingested at Webhooks::V1::ShipengineController#create. The
roll-up status (latest status_code, estimated_delivery_date, etc.)
continues to live as columns on Shipment; this table is the per-scan
audit trail used by the Tracking Events tab and future sweep workers.

Idempotency: ShipEngine does not stamp events with a stable ID and each
webhook re-sends the cumulative events[] array. We synthesise a
natural key from (tracking_number, occurred_at, event_code, status_code).
Replay safety relies on the unique index on idempotency_key.

Mirrors FreightEvent (the Freightquote/CHR Navisphere projection) in
structure and conventions.

See Also:

Constant Summary collapse

STATUS_CODE_LABELS =

ShipEngine normalised status codes (mirrors
WebhookProcessors::ShipengineProcessor::TRACKING_STATUS_CODES).

{
  'UN' => 'unknown',
  'AC' => 'accepted',
  'IT' => 'in_transit',
  'DE' => 'delivered',
  'EX' => 'exception',
  'AT' => 'delivery_attempt',
  'NY' => 'not_yet_in_system',
  'SP' => 'delivered_to_collection_location'
}.freeze
MOVEMENT_STATUS_CODES =

Status codes where the carrier has physically accepted the package
(i.e. past "label created"). These are the events that meaningfully
advance a shipment's lifecycle. Mirrors
WebhookProcessors::ShipengineProcessor::MOVEMENT_STATUS_CODES.

%w[IT DE EX AT SP].freeze
DELIVERED_STATUS_CODES =

Terminal happy-path code; reaching this is the strongest signal a
package was delivered.

%w[DE SP].freeze
EXCEPTION_STATUS_CODES =

Carrier status codes indicating a delivery exception or attention needed.

%w[EX AT].freeze
RETURN_TO_SENDER_PATTERN =

"Returning package to shipper" (FedEx), "UPS has returned your package
to the sender", "Returned to sender/shipper" — the scan that flips a
shipment's story from "delivering" to "coming back". A DE scan after
one of these is the parcel arriving back at ORIGIN, not the customer.

/\breturn\w*(?:\s+\w+){0,2}\s+to\s+(?:the\s+)?(?:sender|shipper)\b/i
DEVIATION_PATTERN =

Softer trouble than #exception?: the carrier is still moving the
parcel, but the delivery has deviated from plan — a delay, an address
correction, a reroute, a reschedule, a customs hold. These arrive as
plain IT scans, so neither the status code nor infer_status_code
(whose IT catch-all swallows them) can distinguish them.

Matched on descriptive text rather than a code. UPS does tag these
event_code: "X", but that code is NOT a usable marker: across 45
days of prod its single most common description is a perfectly benign
"On the Way" (32 of ~180), so keying on it would mislabel healthy
parcels. FedEx uses HLD/X inconsistently and USPS uses numeric codes,
so text is the only signal that generalizes across carriers.

Every pattern below was taken from descriptions actually observed in
shipment_events — not from carrier documentation.

/
  \bdelay(?:ed|s|ing)?\b
  | \barriving\s+late\b | \blater\s+than\s+expected\b
  | \baddress\s+(?:corrected|changed|information\s+required)
  | \baddressing\s+error\b | \bstreet\s+name\s+is\s+incorrect\b
  | \bwe(?:'ve|\s+have)\s+corrected\s+the\s+postal\s+code\b
  | \bre-?rout(?:e|ed)\b | \breschedul(?:e|ed|ing)\b
  | \bclearance\b | \bcustoms\b | \baccess\s+point\b
  | \bsevere\s+weather\b | \bweather\s+may\s+cause\b
/xi

Constants included from Models::Schedulable

Models::Schedulable::SIMPLE_FORM_OPTIONS

Instance Attribute Summary collapse

Belongs to collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from ApplicationRecord

ransackable_associations, ransackable_attributes, ransackable_scopes, ransortable_attributes, #to_relation

Methods included from Models::Schedulable

config

Methods included from Models::AfterCommittable

#after_commit

Methods included from Models::EventPublishable

#publish_event

Instance Attribute Details

#idempotency_key ⇒ Object (readonly)

Validates the idempotency key is present and unique (dedupes webhook redeliveries).

Validations:



133
 
# File 'app/models/shipment_event.rb', line 133

validates :idempotency_key, presence: true, uniqueness: true

#occurred_at ⇒ Object (readonly)

Validates the event timestamp is present.

Validations:



129
 
# File 'app/models/shipment_event.rb', line 129

validates :occurred_at, presence: true

#tracking_number ⇒ Object (readonly)

Validates the tracking number is present.

Validations:



131
 
# File 'app/models/shipment_event.rb', line 131

validates :tracking_number, presence: true

Class Method Details

.chronological ⇒ ActiveRecord::Relation<ShipmentEvent>

A relation of ShipmentEvents that are chronological. Active Record Scope

Returns:

See Also:



140
 
# File 'app/models/shipment_event.rb', line 140

scope :chronological,       -> { order(:occurred_at) }

.delivered ⇒ ActiveRecord::Relation<ShipmentEvent>

A relation of ShipmentEvents that are delivered. Active Record Scope

Returns:

See Also:



138
 
# File 'app/models/shipment_event.rb', line 138

scope :delivered,           -> { where(status_code: DELIVERED_STATUS_CODES) }

.derive_carrier_code(payload) ⇒ String?

Pull the carrier_code out of the resource_url query string when
present (e.g. /v1/tracking?carrier_code=usps&tracking_number=...).
ShipEngine does not surface carrier_code in the structured payload,
so we sniff it from the URL — best-effort, returns nil when absent.

Parameters:

  • payload (Hash) —

    ShipEngine webhook payload

Returns:

  • (String, nil) —

    carrier code (e.g. 'usps')



181
182
183
184
185
186
187
188
189
190
191
 
# File 'app/models/shipment_event.rb', line 181

def self.derive_carrier_code(payload)
  url = payload['resource_url'].to_s
  return nil if url.blank?

  query = URI.parse(url).query
  return nil if query.blank?

  URI.decode_www_form(query).to_h['carrier_code']
rescue URI::InvalidURIError
  nil
end

.exceptions ⇒ ActiveRecord::Relation<ShipmentEvent>

A relation of ShipmentEvents that are exceptions. Active Record Scope

Returns:

See Also:



139
 
# File 'app/models/shipment_event.rb', line 139

scope :exceptions,          -> { where(status_code: EXCEPTION_STATUS_CODES) }

.for_tracking_number ⇒ ActiveRecord::Relation<ShipmentEvent>

A relation of ShipmentEvents that are for tracking number. Active Record Scope

Returns:

See Also:



135
 
# File 'app/models/shipment_event.rb', line 135

scope :for_tracking_number, ->(tracking_number) { where(tracking_number: tracking_number) }

.idempotency_key_for(tracking_number:, occurred_at:, event_code: nil, status_code: nil) ⇒ String

Build the natural-key string used for replay-safe upserts. Mirrors
FreightEvent.idempotency_key_for in shape — UTC-normalised time so
the same payload arriving as a string or a Time produces an identical
key.

Parameters:

  • tracking_number (String)
  • occurred_at (Time, String)
  • event_code (String, nil) (defaults to: nil)
  • status_code (String, nil) (defaults to: nil)

Returns:

  • (String) 


154
155
156
157
158
 
# File 'app/models/shipment_event.rb', line 154

def self.idempotency_key_for(tracking_number:, occurred_at:, event_code: nil, status_code: nil)
  time = occurred_at.is_a?(String) ? Time.zone.parse(occurred_at) : occurred_at
  time_str = time&.utc&.iso8601(6)
  [tracking_number, time_str, event_code, status_code].map(&:to_s).join('|')
end

.infer_status_code(description) ⇒ String?

Maps a carrier-side description (e.g. "Our Package was Delivered",
"Out for Delivery-Almost There!", "Picked Up and On the Move!") to
one of the STATUS_CODE_LABELS keys (UN/AC/IT/DE/EX/AT/NY/SP).
Returns nil when no pattern matches; the field then stays nil.

Used as a fallback by parse_event when ShipEngine leaves the
per-event status_code nil (Canpar consistently does this) and by
Shipping::ShipengineBase#recover_actual_delivery_date_from_events!
so the polling tracking path also recognizes delivered scans from
description text.

Parameters:

  • description (String, nil) —

    carrier-side event description

Returns:

  • (String, nil) —

    one of STATUS_CODE_LABELS.keys, or nil



93
94
95
96
97
98
99
100
101
 
# File 'app/models/shipment_event.rb', line 93

def self.infer_status_code(description)
  return nil if description.blank?

  text = description.to_s
  STATUS_INFERENCE_PATTERNS.each do |code, pattern|
    return code if pattern.match?(text)
  end
  nil
end

.movement ⇒ ActiveRecord::Relation<ShipmentEvent>

A relation of ShipmentEvents that are movement. Active Record Scope

Returns:

See Also:



137
 
# File 'app/models/shipment_event.rb', line 137

scope :movement,            -> { where(status_code: MOVEMENT_STATUS_CODES) }

.parse_event(event, tracking_number:, carrier_code:) ⇒ Hash

Parse one entry from data.events[] into our column shape.

Parameters:

  • event (Hash) —

    one entry from the events array

  • tracking_number (String)
  • carrier_code (String, nil)

Returns:

  • (Hash) —

    attribute hash; occurred_at stays a string for AR coercion



199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
 
# File 'app/models/shipment_event.rb', line 199

def self.parse_event(event, tracking_number:, carrier_code:)
  # Synthesize status_code from the carrier description when ShipEngine
  # leaves the per-event field nil (Canpar's consistent behavior).
  status_code = event['status_code'].presence ||
                infer_status_code(event['event_description'].presence || event['description'])

  {
    tracking_number: tracking_number,
    carrier_code: carrier_code,
    occurred_at: event['occurred_at'],
    status_code: status_code,
    status_description: STATUS_CODE_LABELS[status_code],
    carrier_status_code: event['carrier_status_code'],
    carrier_status_description: event['carrier_status_description'],
    event_code: event['event_code'],
    event_description: event['event_description'],
    description: event['description'],
    carrier_detail_code: event['carrier_detail_code'],
    city_locality: event['city_locality'].presence,
    state_province: event['state_province'].presence,
    postal_code: event['postal_code'].presence,
    country_code: event['country_code'].presence,
    company_name: event['company_name'].presence,
    signer: event['signer'].presence,
    latitude: event['latitude'],
    longitude: event['longitude'],
    payload: event
  }
end

.parse_payload(payload) ⇒ Array<Hash>

Parse one ShipEngine tracking webhook into an array of attribute
hashes — one per entry in data.events[]. The roll-up status_code
and tracking_number live at data.*; per-scan fields are extracted
from each event in the array.

Parameters:

  • payload (Hash) —

    full webhook body (with data envelope)

Returns:

  • (Array<Hash>) —

    one ShipmentEvent attribute hash per scan



167
168
169
170
171
172
173
 
# File 'app/models/shipment_event.rb', line 167

def self.parse_payload(payload)
  data = payload['data'] || {}
  tracking_number = data['tracking_number']
  carrier_code = derive_carrier_code(payload)
  events = Array(data['events'])
  events.map { |event| parse_event(event, tracking_number: tracking_number, carrier_code: carrier_code) }
end

.since ⇒ ActiveRecord::Relation<ShipmentEvent>

A relation of ShipmentEvents that are since. Active Record Scope

Returns:

See Also:



136
 
# File 'app/models/shipment_event.rb', line 136

scope :since,               ->(time)            { where('occurred_at >= ?', time) }

Instance Method Details

#delivered? ⇒ Boolean

Returns true if this scan is a terminal delivered event.

Returns:

  • (Boolean) —

    true if this scan is a terminal delivered event.



248
249
250
 
# File 'app/models/shipment_event.rb', line 248

def delivered?
  DELIVERED_STATUS_CODES.include?(status_code)
end

#deviation? ⇒ Boolean

Returns true if this scan says the delivery deviated from
plan (delayed / readdressed / rerouted / rescheduled / held in
customs) without being a hard exception.

Returns:

  • (Boolean) —

    true if this scan says the delivery deviated from
    plan (delayed / readdressed / rerouted / rescheduled / held in
    customs) without being a hard exception.



293
294
295
 
# File 'app/models/shipment_event.rb', line 293

def deviation?
  DEVIATION_PATTERN.match?(descriptive_text)
end

#exception? ⇒ Boolean

Returns true if this scan is an exception / failed-attempt.
Falls back to description inference because carriers sometimes
normalize a failed attempt as a plain IT scan — e.g. Purolator's
event_code: "Undeliverable" / "Attempted delivery - receiver
unavailable" arrives with status_code: "IT" (DE796177, 2026-07).

Returns:

  • (Boolean) —

    true if this scan is an exception / failed-attempt.
    Falls back to description inference because carriers sometimes
    normalize a failed attempt as a plain IT scan — e.g. Purolator's
    event_code: "Undeliverable" / "Attempted delivery - receiver
    unavailable" arrives with status_code: "IT" (DE796177, 2026-07).



257
258
259
260
261
262
 
# File 'app/models/shipment_event.rb', line 257

def exception?
  return true if EXCEPTION_STATUS_CODES.include?(status_code)
  return false if delivered?

  EXCEPTION_STATUS_CODES.include?(self.class.infer_status_code(descriptive_text))
end

#movement? ⇒ Boolean

Returns true if this scan is one of the movement-stage codes
(carrier has physically accepted the package).

Returns:

  • (Boolean) —

    true if this scan is one of the movement-stage codes
    (carrier has physically accepted the package).



243
244
245
 
# File 'app/models/shipment_event.rb', line 243

def movement?
  MOVEMENT_STATUS_CODES.include?(status_code)
end

#return_to_sender? ⇒ Boolean

Returns true if this scan indicates the parcel is being (or
has been) sent back to the shipper.

Returns:

  • (Boolean) —

    true if this scan indicates the parcel is being (or
    has been) sent back to the shipper.



237
238
239
 
# File 'app/models/shipment_event.rb', line 237

def return_to_sender?
  RETURN_TO_SENDER_PATTERN.match?(descriptive_text)
end

#shipment ⇒ Shipment

Shipment this event belongs to.

Returns:

See Also:



117
 
# File 'app/models/shipment_event.rb', line 117

belongs_to :shipment, inverse_of: :shipment_events, optional: true

#webhook_log ⇒ WebhookLog

Webhook log entry the event was ingested from.

Returns:

See Also:



119
 
# File 'app/models/shipment_event.rb', line 119

belongs_to :webhook_log, optional: true