Class: Edi::Amazon::ReturnsReportRetriever

Inherits:
BaseEdiService show all
Defined in:
app/services/edi/amazon/returns_report_retriever.rb

Overview

Service object: returns report retriever.

Pulls Amazon Seller Central customer-return data via the SP-API Reports
API (+GET_FLAT_FILE_RETURNS_DATA_BY_RETURN_DATE+) for a date window and
lands the flat-file TSV on an +EdiCommunicationLog+ (category
+return_batch+) for ReturnsReportProcessor to apply.

The flow is deliberately synchronous — createReport, poll getReport until
DONE, getReportDocument, download — and NOT the REPORT_PROCESSING_FINISHED
SQS notification path: that notification fires for every report ANY
authorized app runs on the seller account with no requester field to
correlate on (see lib/tasks/amazon_notifications.rake).

Constant Summary collapse

REPORT_TYPE =

The SP-API report type for customer returns by return date.

'GET_FLAT_FILE_RETURNS_DATA_BY_RETURN_DATE'
REPORTS_API_VERSION =

Reports API version segment.

'2021-06-30'
MAX_POLL_ATTEMPTS =

getReport polling budget: 10 x 30s = 5 minutes worst case before giving up.

10
POLL_INTERVAL_SECONDS = 
30

Constants included from RequestIdentifiable

RequestIdentifiable::REQUEST_ID_HEADERS

Constants included from Edi::AddressAbbreviator

Edi::AddressAbbreviator::MAX_LENGTH

Instance Attribute Summary

Attributes inherited from BaseEdiService

#orchestrator

Attributes inherited from BaseService

#options

Instance Method Summary collapse

Methods inherited from BaseEdiService

#amazon_feed_product_type, #duplicate_po_already_notified?, #initialize, #mark_duplicate_po_as_notified, #onboard_ordered_catalog_items, #report_order_creation_issues, #safe_process_edi_communication_log

Methods included from RequestIdentifiable

#partner_request_id

Methods included from Edi::AddressAbbreviator

#abbreviate_street, #collect_street_originals, #record_address_abbreviation_notes

Methods inherited from BaseService

#initialize, #log_debug, #log_error, #log_info, #log_warning, #logger, #tagged_logger

Constructor Details

This class inherits a constructor from Edi::BaseEdiService

Instance Method Details

#api_host ⇒ String

Returns the SP-API host from the orchestrator's transport profile.

Returns:

  • (String) —

    the SP-API host from the orchestrator's transport profile.



227
228
229
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 227

def api_host
  Heatwave::Configuration.fetch(orchestrator.transporter_profile&.to_sym, :api_host)
end

#create_report(transport, start_time, end_time) ⇒ String?

Requests report generation from the Reports API.

Parameters:

Returns:

  • (String, nil) —

    the reportId, or nil when the request failed
    (rate limited or API error — next scheduled run retries).



97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 97

def create_report(transport, start_time, end_time)
  body = {
    reportType: REPORT_TYPE,
    marketplaceIds: [orchestrator.marketplace],
    dataStartTime: start_time.utc.iso8601,
    dataEndTime: end_time.utc.iso8601
  }.to_json
  res = transport.send_data(body, reports_url, 'POST', { 'content-type' => 'application/json' })
  report_id = res[:success] ? sp_api_payload(res).dig(:reportId) : nil
  logger.warn "createReport for #{REPORT_TYPE} failed for #{orchestrator.partner}: #{res[:http_result]&.body}" if report_id.blank?
  report_id.presence
rescue HTTP::RateLimitExceededError => e
  logger.warn "Amazon SP-API rate limited on createReport for #{orchestrator.partner}. Skipping run; next scheduled run will retry. #{e.message}"
  nil
end

#documents_url ⇒ String

Returns the getReportDocument endpoint base.

Returns:

  • (String) —

    the getReportDocument endpoint base.



208
209
210
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 208

def documents_url
  "https://#{api_host}/reports/#{REPORTS_API_VERSION}/documents"
end

#download_document(transport, document_id) ⇒ String?

Fetches the document's pre-signed URL via getReportDocument and
downloads (and, when Amazon says so, gunzips) the flat file.

Parameters:

Returns:

  • (String, nil) —

    the TSV payload, or nil on failure.



156
157
158
159
160
161
162
163
164
165
166
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 156

def download_document(transport, document_id)
  res = transport.send_data('', "#{documents_url}/#{document_id}", 'GET')
  payload = res[:success] ? sp_api_payload(res) : {}
  url = payload[:url]
  if url.blank?
    logger.warn "getReportDocument #{document_id} failed: #{res[:http_result]&.body}"
    return nil
  end
  body = Faraday.get(url).body
  payload[:compressionAlgorithm] == 'GZIP' ? ActiveSupport::Gzip.decompress(body) : body
end

#instantiate_transporter(transporter, transporter_profile = nil) ⇒ Object 



231
232
233
234
235
236
237
238
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 231

def instantiate_transporter(transporter, transporter_profile = nil)
  case transporter
  when :http_seller_api
    Transport::HttpSellerApiConnection.new({ profile: transporter_profile })
  else
    raise "Unknown transporter: #{transporter}"
  end
end

#poll_for_document(transport, report_id) ⇒ String, ...

Polls getReport until the report reaches a terminal state, bounded by
MAX_POLL_ATTEMPTS x POLL_INTERVAL_SECONDS.

Parameters:

Returns:

  • (String, nil, Symbol) —

    the reportDocumentId on DONE with data;
    nil on CANCELLED or an empty window (DONE with no document);
    +:fatal+ on FATAL (dead report — start a fresh one), +:failed+ on
    timeout or rate limit (resume polling the same report).



122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 122

def poll_for_document(transport, report_id)
  MAX_POLL_ATTEMPTS.times do |attempt|
    res = transport.send_data('', "#{reports_url}/#{report_id}", 'GET')
    payload = res[:success] ? sp_api_payload(res) : {}
    status = payload[:processingStatus]
    if status == 'DONE'
      document_id = payload[:reportDocumentId].presence
      logger.info "Returns report #{report_id} DONE with no document — no returns in window" unless document_id
      return document_id
    elsif status == 'FATAL'
      # FATAL is a report-generation failure — retryable, but re-polling
      # THIS report is pointless; the caller drops it and starts fresh.
      logger.warn "Returns report #{report_id} finished FATAL; will retry with a new report next run"
      return :fatal
    elsif status == 'CANCELLED'
      logger.warn "Returns report #{report_id} finished CANCELLED; nothing to import"
      return nil
    end
    logger.debug { "Returns report #{report_id} status #{status || 'unknown'} (attempt #{attempt + 1}/#{MAX_POLL_ATTEMPTS})" }
    sleep(POLL_INTERVAL_SECONDS)
  end
  logger.warn "Returns report #{report_id} not DONE after #{MAX_POLL_ATTEMPTS} polls; giving up (next run will retry the window)"
  :failed
rescue HTTP::RateLimitExceededError => e
  logger.warn "Amazon SP-API rate limited polling getReport #{report_id}. Giving up this run; next run will retry the window. #{e.message}"
  :failed
end

#process(start_time: nil, end_time: nil) ⇒ Array<EdiCommunicationLog>?

Retrieves the returns report for the window and stores it as an ECL.

Idempotent per partner + window: the ECL file_name is deterministic
(+returns_YYYY-MM-DD_YYYY-MM-DD.tsv+), so a re-run over the same window
returns the already-stored log instead of duplicating it.

Parameters:

  • start_time (Time, Date, String, nil) (defaults to: nil) —

    window start (default:
    previous day, beginning of day); parameterizable for backfills.

  • end_time (Time, Date, String, nil) (defaults to: nil) —

    window end (default: end of
    the start day).

Returns:

  • (Array<EdiCommunicationLog>, nil) —

    the stored (or pre-existing)
    logs; empty for a valid empty window (CANCELLED/DONE without
    document); nil on failure (API error, FATAL, timeout, rate limit) so
    the caller can leave the window retryable instead of gating it.



39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 39

def process(start_time: nil, end_time: nil)
  start_time, end_time = resolve_window(start_time, end_time)
  file_name = "returns_#{start_time.to_date.iso8601}_#{end_time.to_date.iso8601}.tsv"
  if (existing = EdiCommunicationLog.find_by(partner: orchestrator.partner, category: 'return_batch', file_name: file_name))
    logger.info "#{file_name} already stored as edi log #{existing.id}; skipping"
    return [existing]
  end

  transport = instantiate_transporter(orchestrator.transporter, orchestrator.transporter_profile)
  # createReport has no idempotency key: after a mid-flow failure the
  # retry must resume polling the SAME report, not submit another one
  # for the same window (Amazon happily generates duplicates).
  pending_report_key = "edi/amazon/returns_report/pending_report/#{orchestrator.partner}/#{file_name}"
  report_id = Rails.cache.read(pending_report_key) || create_report(transport, start_time, end_time)
  return nil unless report_id

  Rails.cache.write(pending_report_key, report_id, expires_in: 24.hours)
  document_id = poll_for_document(transport, report_id)
  if document_id == :fatal # dead report — next run starts a fresh one
    Rails.cache.delete(pending_report_key)
    return nil
  end
  return nil if document_id == :failed # timeout/rate-limit — resume polling this report next run

  Rails.cache.delete(pending_report_key) if document_id.nil? # CANCELLED/DONE-no-document is terminal
  return [] if document_id.nil? # valid empty window

  tsv = download_document(transport, document_id)
  return nil if tsv.blank?

  Rails.cache.delete(pending_report_key)
  [store_log(tsv, file_name, start_time, end_time, report_id, document_id)]
rescue StandardError => e
  # Transport errors (timeouts, connection failures) and store failures
  # must also leave the window retryable, not escape as an unhandled
  # exception — the caller only pins/gates on a nil result.
  logger.warn "Returns retrieval failed for #{orchestrator.partner} (#{file_name}): #{e.class}: #{e.message} — leaving the window retryable"
  nil
end

#reports_url ⇒ String

Returns the createReport/getReport endpoint base.

Returns:

  • (String) —

    the createReport/getReport endpoint base.



203
204
205
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 203

def reports_url
  "https://#{api_host}/reports/#{REPORTS_API_VERSION}/reports"
end

#resolve_window(start_time, end_time) ⇒ Array(Time, Time)

Resolves the report window, defaulting to the previous day (UTC).

Parameters:

  • start_time (Time, Date, String, nil) —

    explicit window start.

  • end_time (Time, Date, String, nil) —

    explicit window end.

Returns:

  • (Array(Time, Time)) —

    the resolved [start, end] window.



84
85
86
87
88
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 84

def resolve_window(start_time, end_time)
  start_time = (start_time.presence || 1.day.ago.beginning_of_day).to_time
  end_time = (end_time.presence || start_time.end_of_day).to_time
  [start_time, end_time]
end

#sp_api_payload(res) ⇒ HashWithIndifferentAccess

Parses a Reports API response body. The 2021-06-30 Reports API returns
its fields top-level ("..."); older SP-API endpoints wrap
them in a +payload+ key, which we tolerate for safety.

Parameters:

  • res (Hash) —

    the transport's result envelope.

Returns:

  • (HashWithIndifferentAccess) —

    the parsed payload (empty on
    unparseable body).



219
220
221
222
223
224
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 219

def sp_api_payload(res)
  parsed = JSON.parse(res[:http_result].body.to_s).with_indifferent_access
  parsed[:payload].presence || parsed
rescue JSON::ParserError
  {}
end

#store_log(tsv, file_name, start_time, end_time, report_id, document_id) ⇒ EdiCommunicationLog

Lands the TSV on an +EdiCommunicationLog+ for the processor to pick up.

Parameters:

  • tsv (String) —

    the report payload.

  • file_name (String) —

    deterministic per-window file name.

  • start_time (Time) —

    window start.

  • end_time (Time) —

    window end.

  • report_id (String) —

    the SP-API reportId.

  • document_id (String) —

    the SP-API reportDocumentId.

Returns:



177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
 
# File 'app/services/edi/amazon/returns_report_retriever.rb', line 177

def store_log(tsv, file_name, start_time, end_time, report_id, document_id)
  edi_log = EdiCommunicationLog.create!(
    partner: orchestrator.partner,
    category: 'return_batch',
    data: tsv.encode('UTF-8', invalid: :replace, replace: ''),
    data_type: 'csv',
    file_name: file_name,
    file_info: {
      report_id: report_id,
      report_document_id: document_id,
      window_start: start_time.utc.iso8601,
      window_end: end_time.utc.iso8601
    },
    transmit_datetime: Time.current
  )
  logger.info "#{file_name} saved to edi log #{edi_log.id}"
  edi_log
rescue ActiveRecord::RecordNotUnique
  # A concurrent run stored this window first (partial unique index
  # idx_ecl_return_batch_partner_file_name_uniq) — reuse it. If the
  # winner vanished in between, re-raise the original RecordNotUnique
  # rather than masking it as RecordNotFound.
  EdiCommunicationLog.find_by(partner: orchestrator.partner, category: 'return_batch', file_name: file_name) || raise
end