Module: Durability

Defined in:
app/services/durability.rb

Overview

Durability — guards inbound ingestion against silent "fake-success" commits.

A poisoned or desynced pooled database connection can make +save!+/+create!+
return WITHOUT raising while the row never durably commits — during the June
2026 EDI incident the INSERT even reported an unrelated row's id. Any
ingestion path that ACKNOWLEDGES its source — deletes the SFTP file, returns
200 to a partner that pushed an order, returns 200 to a webhook provider —
BEFORE the write is durable can therefore lose the message with no trace and
no error.

Call Durability.confirm_persisted! AFTER the transaction has closed and BEFORE the ack.
It re-reads the row by its NATURAL KEY — never by id alone, which a stale-id
fake success would defeat — on the current (writing) connection with the query
cache bypassed, so it sees the real committed table state. If the row is
absent it fires a critical alert and raises CommitLostError, so the caller's
rescue leaves the source un-acked and the partner / SFTP / provider
re-delivers instead of the message vanishing.

NOTE: must run on the writing connection (do not wrap in
+connected_to(role: :reading)+) or replica lag would cause false positives.

Defined Under Namespace

Classes: CommitLostError

Class Method Summary collapse

Class Method Details

.confirm_persisted!(klass, natural_key, context: {}) ⇒ true

Returns when the row is confirmed durable.

Parameters:

  • klass (Class<ActiveRecord::Base>) —

    the model to re-read

  • natural_key (Hash) —

    business-key attributes that uniquely identify the
    row (e.g. +category:, file_name:+ or +external_id:+).
    Must contain at least one non-+:id+ attribute — verifying by id alone would
    be fooled by the stale-id failure mode.

  • context (Hash) (defaults to: {}) —

    extra context attached to the critical alert

Returns:

  • (true) —

    when the row is confirmed durable

Raises:

  • (ArgumentError) —

    if +natural_key+ has no non-+:id+ attribute

  • (CommitLostError) —

    if no committed row matches +natural_key+



39
40
41
42
43
44
45
46
47
48
49
50
51
52
 
# File 'app/services/durability.rb', line 39

def confirm_persisted!(klass, natural_key, context: {})
  natural_key = natural_key.compact
  raise ArgumentError, "Durability.confirm_persisted! needs a natural key beyond :id (got #{natural_key.inspect})" if natural_key.except(:id).empty?

  # Bypass the query cache so we read the real committed state, not a cached
  # or in-transaction view, on the writing connection.
  return true if klass.uncached { klass.where(natural_key).exists? }

  ErrorReporting.critical(
    CommitLostError.new("Durable write lost: #{klass.name} #{natural_key.inspect} absent after commit"),
    { source: :background, model: klass.name, natural_key: natural_key.inspect }.merge(context)
  )
  raise CommitLostError, "#{klass.name} #{natural_key.inspect} did not durably commit"
end