Class: InvalidDigitalAssetPurgeWorker

Inherits:
Object
  • Object
show all
Includes:
Sidekiq::Job
Defined in:
app/workers/invalid_digital_asset_purge_worker.rb

Overview

Nightly purge — the tail end of the broken-asset lifecycle.

Lifecycle: ImageFullAnalysisWorker stamps invalid_at on a permanent embedding
failure (404/410 URL, "invalid image" 400) → DigitalAsset.invalidate! excludes
the asset from every .active fetch and the embedding candidate scope → after
RETENTION (a recovery window) the unreferenced ones are hard-deleted here.

Safety:

  • Only assets quarantined for >= RETENTION (30-day window to re-upload/recover).
  • destroy! is attempted per asset; those still referenced by a RESTRICT FK
    (e.g. a product's primary_image_id) raise InvalidForeignKey and are SKIPPED
    • logged for manual recovery — never force-deleted or orphaned.
  • Capped per run (BATCH_LIMIT) and count-logged (bulk-op protocol).

Scheduled nightly via sidekiq-cron (config/sidekiq_production_schedule.yml).

Constant Summary collapse

RETENTION =

Recovery window before a quarantined asset becomes eligible for hard delete.

30.days
BATCH_LIMIT =

Max assets to purge per run — keeps the nightly sweep bounded.

500

Instance Method Summary collapse

Instance Method Details

#perform ⇒ Object

Runs the job.

Returns:

  • (Object) —

    the result



31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
 
# File 'app/workers/invalid_digital_asset_purge_worker.rb', line 31

def perform
  cutoff = RETENTION.ago
  assets = DigitalAsset.invalidated
                       .where(invalid_at: ..cutoff)
                       .order(:invalid_at)
                       .limit(BATCH_LIMIT)
                       .to_a
  return log_info('Nothing to purge') if assets.empty?

  deleted = 0
  retained = []

  assets.each do |asset|
    # Lock the row so the reference check and the destroy are atomic: a new
    # reference (product primary image / article preview / video poster)
    # attached mid-loop can't slip in between externally_referenced? and
    # destroy! and then be silently detached (nullify) + deleted.
    asset.with_lock do
      if asset.externally_referenced?
        retained << asset.id
        log_warn("Retained ##{asset.id} (#{asset.class.name}) — still referenced (product/article/video), needs manual recovery")
      else
        asset.destroy!
        deleted += 1
      end
    end
  rescue ActiveRecord::RecordNotFound
    log_info("Skipped ##{asset.id} — already removed by a concurrent run")
  rescue ActiveRecord::InvalidForeignKey, ActiveRecord::RecordNotDestroyed => e
    # Backstop: any other RESTRICT FK not covered by externally_referenced?.
    retained << asset.id
    log_warn("Retained ##{asset.id} (#{asset.class.name}) — destroy blocked by FK: #{e.message.truncate(120)}")
  end

  overflow = retained.size > 20 ? ', …' : ''
  detail   = retained.any? ? " [#{retained.first(20).join(', ')}#{overflow}]" : ''
  log_info("Purged #{deleted} quarantined asset(s) (> #{RETENTION.inspect} old); " \
           "retained #{retained.size} still-referenced#{detail}")
end