Class: Shipping::LabelBarcodeReader

Inherits:
Object
  • Object
show all
Defined in:
app/services/shipping/label_barcode_reader.rb

Overview

Reads the barcodes off a shipping label PDF and returns the ones that
are valid tracking numbers.

Why not Edi::Commercehub::PackingSlipReaderThdUs's approach: that
reader uses PDF::Inspector::Text.analyze, which works because
CommerceHub packing slips are generated PDFs carrying a real text
layer (89 and 136 extractable strings in the THD US / CA samples).
ShipStation's Canpar label is a single flattened image — both
PDF::Inspector and PDF::Reader extract zero strings from it — so the
number only exists as bars. (Canada Post's label does carry a text
layer, so text extraction would work there; barcodes are the one
technique that works for every label, which is why we do it this way
for all of them.)

Pipeline: render PDFs at DENSITY DPI through the bounded Poppler
subprocess, then hand the PNG to zbarimg. Libvips remains available
only for ordinary image inputs, whose loaders are not on Rails'
process-wide untrusted list. A decoder gem would be the heavier option;
zbar-tools is the one package this adds to the image.

Examples:

Shipping::LabelBarcodeReader.new(path).tracking_numbers
# => [{ number: "D576002440000001133001", carrier: "Canpar" }]

Defined Under Namespace

Classes: DecodeError

Constant Summary collapse

DENSITY =

Thermal labels rasterize badly below ~300 DPI; 400 decodes reliably
on the Canpar and Canada Post samples without being slow.

400
MAX_PAGES =

Label PDFs are routinely multi-page and the ship label is routinely
not on page 1: an Amazon FBA transfer interleaves carton labels with
UPS labels, so page 1 is an FBA carton barcode and the first ship
label — the one carrying the master tracking number — is page 2.
Pages are scanned in order and results returned in that order, so
callers can take the first hit as the master.

40

Instance Method Summary collapse

Constructor Details

#initialize(file_path, carrier: nil) ⇒ LabelBarcodeReader

Returns a new instance of LabelBarcodeReader.

Parameters:

  • file_path (String) —

    path to a label PDF (or image)

  • carrier (String, nil) (defaults to: nil) —

    the carrier the shipment is going out
    with, free-form (shipments.carrier). Pass it whenever it's known
    — a label sheet carries barcodes that validate as some carrier's
    number without being a tracking number at all. A UPS label's GS1
    420<zip> postal routing code is 12 digits and passes FedEx's
    check digit; scoping to the expected carrier drops it, and every
    other carrier's stray codes with it.



52
53
54
55
56
57
58
59
60
61
 
# File 'app/services/shipping/label_barcode_reader.rb', line 52

def initialize(file_path, carrier: nil)
  @file_path = file_path
  # Honour the hint only when it names a carrier we hold specs for.
  # Callers pass `shipments.carrier` straight through, and on a manual
  # delivery that is routinely a placeholder ("override", "Standard").
  # Treating those as a scope would filter out every hit instead of
  # falling back to sniffing the number, which is the useful default.
  normalized = Heatwave::Normalizers.shipping_carrier(carrier.to_s).presence
  @carrier = normalized if PARCEL_CARRIER_INTERNAL_TO_TRACKING_NUMBER_GEM.key?(normalized)
end

Instance Method Details

#candidates_in(raw) ⇒ Array<String>

Readings to try from one barcode payload or text string. A label
never presents the number in just one shape:

  • Code 39 payloads arrive wrapped in their * start/stop delimiter.
  • GS1-128 payloads pack several fields separated by the GS control
    character — USPS labels carry 420<zip> ␝ 9434…, and only the
    second field is the tracking number.
  • Text is printed in readable groups ("8390 2796 4580 8746") or
    behind a caption ("PUROLATOR PIN: 520669613470"), so both the
    packed whole and each individual token have to be tried.

Parameters:

  • raw (String)

Returns:

  • (Array<String>) 


133
134
135
136
137
138
139
 
# File 'app/services/shipping/label_barcode_reader.rb', line 133

def candidates_in(raw)
  raw.split(/[]/).flat_map do |field|
    field = field.gsub(/\A\*|\*\z/, '')
    packed = Heatwave::Normalizers.tracking_number(field).to_s
    [packed, *packed.scan(/[0-9A-Z]+/)]
  end.reject(&:empty?).uniq
end

#decode ⇒ Array<String>

Every barcode payload on the label, tracking number or not. Labels
carry plenty that aren't: the Canpar label's second barcode is the
16-digit one this whole exercise is about, and cartons carry SSCCs.

Returns:

  • (Array<String>) 


99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
 
# File 'app/services/shipping/label_barcode_reader.rb', line 99

def decode
  return [] unless File.exist?(@file_path)

  Dir.mktmpdir do |dir|
    decoded = []
    page_count.times do |page|
      decoded.concat(decode_page(page, dir))
    rescue Pdf::Utility::CliRasterizer::RasterizationError => e
      raise unless @page_count_fallback && page.positive?

      Rails.logger.info "[LabelBarcodeReader] stopped fallback scan at page #{page + 1}: #{e.message}"
      break
    end
    decoded
  end
rescue StandardError => e
  Rails.logger.error "[LabelBarcodeReader] #{@file_path}: #{e.message}"
  ErrorReporting.error(e)
  []
end

#decode_page(page, dir) ⇒ Array<String>

Parameters:

  • page (Integer) —

    zero-based

  • dir (String) —

    scratch directory

Returns:

  • (Array<String>) 


215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
 
# File 'app/services/shipping/label_barcode_reader.rb', line 215

def decode_page(page, dir)
  rasterized_page = nil
  raster = if pdf_file?
             rasterized_page = Pdf::Utility::CliRasterizer.render_page(@file_path, dpi: DENSITY, page: page)
             rasterized_page.path
           else
             rasterize_image(dir, page)
           end

  # zbarimg exits 4 when it finds nothing, which is not an error here.
  out, err, status = Heatwave::BoundedSubprocess.capture('zbarimg', '-q', '--raw', raster)
  return [] if status.exitstatus == 4

  unless status.success?
    detail = err.to_s.strip.presence
    raise DecodeError, "zbarimg failed (exit #{status.exitstatus || 'unknown'})#{": #{detail}" if detail}"
  end

  out.to_s.split("\n").map(&:strip).reject(&:empty?)
ensure
  rasterized_page&.close
  rasterized_page&.unlink
  FileUtils.rm_f(raster) if raster && rasterized_page.nil?
end

#drop_fragments(hits) ⇒ Array<Hash>

Discards a hit whose number is contained in a longer one. Text
extraction wraps long numbers across lines, and a wrapped remainder
can be a valid number in its own right: Canada Post's
2007 1528 9968 8344 breaks such that 152899688344 is left on its
own line, and 12 digits with a good check digit is a valid FedEx
number. The full number is always also present, so preferring the
longer containing match drops the artefact.

Parameters:

  • hits (Array<Hash>)

Returns:

  • (Array<Hash>) 


186
187
188
189
190
 
# File 'app/services/shipping/label_barcode_reader.rb', line 186

def drop_fragments(hits)
  hits.reject do |hit|
    hits.any? { |other| other[:number] != hit[:number] && other[:number].include?(hit[:number]) }
  end
end

#page_count ⇒ Integer

Returns pages to scan, capped at MAX_PAGES.

Returns:

  • (Integer) —

    pages to scan, capped at MAX_PAGES



201
202
203
204
205
206
207
208
209
210
 
# File 'app/services/shipping/label_barcode_reader.rb', line 201

def page_count
  @page_count_fallback = false
  return 1 unless pdf_file?

  pages = Pdf::Utility::CliRasterizer.page_count(@file_path)
  pages.clamp(1, MAX_PAGES)
rescue StandardError
  @page_count_fallback = true
  MAX_PAGES # render until the first missing page, still bounded
end

#pdf_file? ⇒ Boolean

Returns whether the input's content is a PDF.

Returns:

  • (Boolean) —

    whether the input's content is a PDF



241
242
243
244
245
246
247
 
# File 'app/services/shipping/label_barcode_reader.rb', line 241

def pdf_file?
  return @pdf_file if defined?(@pdf_file)

  @pdf_file = Pdf::Utility::FileNormalizer.call(@file_path)
rescue StandardError
  @pdf_file = false
end

#rasterize_image(dir, page) ⇒ String

Normalize an ordinary image input to PNG for zbarimg.

Parameters:

  • dir (String) —

    scratch directory

  • page (Integer) —

    zero-based page index (always zero for images)

Returns:

  • (String) —

    path to the PNG



254
255
256
257
258
259
260
261
262
 
# File 'app/services/shipping/label_barcode_reader.rb', line 254

def rasterize_image(dir, page)
  require 'vips'

  raster = File.join(dir, "label-#{page}.png")
  image = Vips::Image.new_from_file(@file_path)
  image = image.flatten(background: [255, 255, 255]) if image.has_alpha?
  image.write_to_file(raster)
  raster
end

#specs_for_carrier(carrier) ⇒ Array<Heatwave::TrackingNumber::Spec>

Parameters:

  • carrier (String) —

    canonical internal carrier name

Returns:



194
195
196
197
198
 
# File 'app/services/shipping/label_barcode_reader.rb', line 194

def specs_for_carrier(carrier)
  TrackingNumberFormatValidator.specs_for(
    TrackingNumberFormatValidator.catalog_courier_code(carrier)
  )
end

#text_fragments ⇒ Array<String>

Lines from the PDF's text layer, each considered on its own — line by
line, never concatenated, because joining them invents numbers that
span two unrelated fields.

Uses poppler's pdftotext rather than the PDF::Inspector::Text
approach Edi::Commercehub::PackingSlipReaderThdUs takes on packing
slips. Both read a text layer, but PDF::Inspector (via pdf-reader) is
strict about structure and raises MalformedPDFError on carrier
labels that every viewer renders fine — a Purolator label printed
from Amazon, whose "PUROLATOR PIN: 520620274255" line is perfectly
legible on screen, is the case that caught this. pdftotext recovered
the text from that file and from every other sample, including two
PDF::Inspector reported as having no text layer at all. poppler-utils
is already installed for other PDF work.

Returns:

  • (Array<String>) 


280
281
282
283
284
285
286
 
# File 'app/services/shipping/label_barcode_reader.rb', line 280

def text_fragments
  out, _err, _status = Heatwave::BoundedSubprocess.capture('pdftotext', '-q', @file_path, '-')
  out.to_s.split("\n").map(&:strip).reject(&:empty?)
rescue StandardError => e
  Rails.logger.info "[LabelBarcodeReader] no text layer on #{@file_path}: #{e.message}"
  []
end

#tracking_numbers ⇒ Array<Hash>

Candidates on the label that parse as a tracking number for a carrier
we know. Barcodes first (the only technique that works on a flattened
label), then the text layer — Canada Post's label is the inverse of
Canpar's: its barcode is a GS1 routing code while the tracking number
sits in the text as "2007 1528 9968 8344".

Returns:

  • (Array<Hash>) —

    [{ number:, carrier:, source: }] where source
    is :barcode or :text — which technique found it, worth knowing
    when auditing a correction. Empty when the label can't be read or
    carries no recognisable tracking number.



73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
 
# File 'app/services/shipping/label_barcode_reader.rb', line 73

def tracking_numbers
  sourced = decode.product([:barcode]) + text_fragments.product([:text])

  sourced.flat_map { |raw, source| candidates_in(raw).product([source]) }.filter_map do |candidate, source|
    # The sniffer proposes a carrier from the number's shape; it is
    # deliberately lenient (it accepts a bad check digit, so a typed
    # number still routes somewhere). Reading a label is the opposite
    # situation — we want only confident reads — so the carrier's own
    # spec has to accept the candidate outright. Without this the
    # caption on a Purolator label ("PUROLATOR PIN: 520669613470",
    # packed to PUROLATORPIN:520669613470) comes back as a number.
    carrier = @carrier || Heatwave::Normalizers.shipping_carrier_from_tracking_number(candidate)
    next if carrier.blank?

    specs = specs_for_carrier(carrier)
    next unless specs.any? { |spec| spec.valid?(candidate) }

    { number: unwrap(candidate, specs), carrier: carrier, source: source }
  end.uniq { |hit| hit[:number] }.then { |hits| drop_fragments(hits) }
end

#unwrap(number, specs) ⇒ String

Some barcodes carry the tracking number inside a larger structure.
FedEx's 34-digit GS1-128 is the case that matters here: it validates
as a FedEx number in its own right, but the number on the package —
and the one the customer tracks — is the serial the catalog decodes
of it (9622001900005141075700382843936101 → 382843936101).

Only unwraps when the inner number is shorter and also valid for
the same carrier, which is what keeps it from mangling formats whose
serial is meaningless on its own: UPS's serial drops the 1Z prefix
and Canpar's drops the D, so neither re-validates and both are left
exactly as scanned.

Deliberately no window-scanning fallback. Purolator's 34-digit
barcode also embeds its PIN, but four different 12-digit windows of
it pass the Luhn check — there is no way to tell which is the real
one, so those labels yield nothing from the barcode and fall through
to the text layer, or to typed entry guarded by
TrackingNumberFormatValidator.

Parameters:

Returns:

  • (String) 


163
164
165
166
167
168
169
170
171
172
173
174
 
# File 'app/services/shipping/label_barcode_reader.rb', line 163

def unwrap(number, specs)
  inner = specs.filter_map do |spec|
    next unless spec.valid?(number)

    decoded = spec.decode(number)
    next if decoded[:serial_number].blank?

    "#{decoded[:serial_number]}#{decoded[:check_digit]}".sub(/\A0+/, '')
  end.uniq.find { |c| c.length < number.length && specs.any? { |spec| spec.valid?(c) } }

  inner || number
end