Class: ApplicationMailbox

Inherits:
ActionMailbox::Base
  • Object
show all
Defined in:
app/mailboxes/application_mailbox.rb

Overview

Test at https://api.warmlyyours.me:3000/rails/conductor/action_mailbox/inbound_emails
Troubleshoot action inbox
ActionMailbox::InboundEmail.failed.first
and for each one of the failed ones you update their status to 'pending'
ActionMailbox::InboundEmail.where(status: 'failed').update_all(status:  'pending')
and now you reprocess each one by doing
SupportsMailbox.new(ActionMailbox::InboundEmail.find(:id)).process
Call Route on email

On production to download an existing email and test it locally
https://world.hey.com/robzolkos/debugging-production-actionmailbox-issues-in-development-f5886579
Find the email: m = ActionMailbox::InboundEmail.find(:id)
Download the raw source and copy it to clipboard.
puts m.mail
Go to your local: https://api.warmlyyours.me:3000/en-US/rails/conductor/action_mailbox/inbound_emails/sources/new
Paste the raw source.

Constant Summary collapse

FETCH_MAX_SIZE =

Maximum download size for fetch_linked_files (25 MB — above this Google Drive shows a warning page).

25 * 1024 * 1024
FETCHABLE_MIME_TYPES =

MIME types accepted when downloading a linked file.

%r{\A(image/|application/(pdf|zip|x-zip|msword|vnd\.openxmlformats|vnd\.ms-))}i
DIRECT_FILE_EXTENSIONS =

Direct file extensions.

/\.(jpe?g|png|gif|webp|svg|pdf|docx?|xlsx?|pptx?|zip|csv)\z/i
GDRIVE_FILE_RE =

Gdrive file re.

%r{https://drive\.google\.com/file/d/([^/?#\s]+)}
GDRIVE_OPEN_RE =

Gdrive open re.

%r{https://drive\.google\.com/open\?id=([^&\s]+)}
MAX_FILENAME_LENGTH =

Sanitize and truncate attachment filenames to prevent ENAMETOOLONG errors
Filesystem limit is typically 255 bytes; we use 200 to be safe with encoding

200

Instance Method Summary collapse

Instance Method Details

#attempt_url_download(url) ⇒ Upload?

Attempt to download a URL, returning an Upload on success or nil on any failure.
Checks the Content-Type before accepting the response to avoid saving HTML auth pages.

Parameters:

  • url (String) —

    the URL to download

Returns:

  • (Upload, nil) —

    the created upload, or nil on failure



133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'app/mailboxes/application_mailbox.rb', line 133

def attempt_url_download(url)
  require 'down'

  tempfile = Down.download(url, max_size: FETCH_MAX_SIZE) { |client| client.timeout(connect: 5, read: 15) }
  content_type = tempfile.content_type.to_s

  return nil if content_type.start_with?('text/html', 'text/plain')
  return nil unless FETCHABLE_MIME_TYPES.match?(content_type)

  filename = sanitize_attachment_filename(
    tempfile.original_filename.presence ||
      begin
        raw = URI.parse(url).path.to_s.split('?').first
        CGI.unescape(File.basename(raw)).presence
      rescue StandardError
        nil
      end ||
      "file_#{SecureRandom.hex(4)}"
  )

  Upload.uploadify(tempfile.path, 'email_attachment', nil, filename)
rescue Down::TooLarge
  Rails.logger.info "[fetch_linked_files] Skipped (too large): #{url}"
  nil
rescue Down::Error, Errno::ECONNREFUSED, SocketError, Timeout::Error, HTTP::Error => e
  Rails.logger.info "[fetch_linked_files] Skipped (#{e.class}): #{url}"
  nil
ensure
  tempfile&.close
  tempfile&.unlink
end

#collect_fetchable_urls(doc) ⇒ Hash{String => String}

Returns a hash of { original_url => download_url } for all candidate URLs
found in the HTML document. Google Drive share links are transformed to
direct-download equivalents. External img src are included as-is.

Parameters:

  • doc (Nokogiri::HTML::Document) —

    the parsed email body

Returns:

  • (Hash{String => String}) —

    map of original URL to download URL



171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
 
# File 'app/mailboxes/application_mailbox.rb', line 171

def collect_fetchable_urls(doc)
  url_map = {}
  our_host_pattern = /warmlyyours\.(com|me)\z/i

  extract_urls = lambda do |raw|
    return unless raw.to_s.start_with?('https://', 'http://')

    uri = URI.parse(raw.to_s.strip)
    return if our_host_pattern.match?(uri.host.to_s)

    if (m = GDRIVE_FILE_RE.match(raw))
      url_map[raw] = "https://drive.google.com/uc?export=download&id=#{m[1]}"
    elsif (m = GDRIVE_OPEN_RE.match(raw))
      url_map[raw] = "https://drive.google.com/uc?export=download&id=#{m[1]}"
    elsif uri.path.to_s.match?(DIRECT_FILE_EXTENSIONS)
      url_map[raw] = raw
    end
  rescue URI::InvalidURIError
    nil
  end

  doc.css('a[href]').each { |a| extract_urls.call(a['href']) }
  doc.css('img[src]').each { |img| extract_urls.call(img['src']) unless url_map.key?(img['src']) }

  url_map
end

#create_activity(resource, act_type, assigned_resource, comm, complete_act) ⇒ void

This method returns an undefined value.

Creates a follow-up Activity on the resource for the inbound email,
targeting end of the next business day, and optionally completing it.

Parameters:

  • resource (ActiveRecord::Base) —

    the resource the activity belongs to

  • act_type (String) —

    the ActivityType task_type (e.g. 'EMAILIN', 'EMAILOUT')

  • assigned_resource (Party, nil) —

    the party the activity is assigned to

  • comm (Communication) —

    the communication created for the email

  • complete_act (Boolean) —

    whether to immediately complete the activity



342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
 
# File 'app/mailboxes/application_mailbox.rb', line 342

def create_activity(resource, act_type, assigned_resource, comm, complete_act)
  note = +''
  note << "<b style='color:red'>THIS EMAIL IS OLD. Probably processed after a technical issue.</b>\n\n" if inbound_email.created_at < 2.days.ago
  note << "Email received on #{inbound_email.created_at} and automatically processed."
  # Due date will be set to end of business day.
  target_datetime = inbound_email.created_at.to_datetime
  target_datetime = WorkingHours.advance_to_working_time(target_datetime) # First get to the first working time
  target_datetime = WorkingHours.advance_to_closing_time(target_datetime) # Then get to the closing time of that time
  a = resource.activities.create!(
    activity_type: ActivityType.find_by(task_type: act_type),
    new_note: note,
    target_datetime: target_datetime,
    original_target_datetime: target_datetime,
    assigned_resource: assigned_resource,
    communication_id: comm.id
  )
  a.complete(closed_by_id: assigned_resource) if complete_act
  Rails.logger.info "Activity created from #{resource.class} Mailbox. Activity ID #{a.id}"
end

#create_communication(resource, mail, note, comm_state, comm_direction) ⇒ Communication

Builds and saves a Communication for the resource from the inbound email,
including recipients (to/cc) and sender party resolution.

Parameters:

  • resource (ActiveRecord::Base) —

    the resource the communication belongs to

  • mail (Mail::Message) —

    the inbound email message

  • note (String) —

    the extracted body content

  • comm_state (String) —

    the communication state ('sent' or 'received')

  • comm_direction (String) —

    the communication direction ('outbound' or 'inbound')

Returns:



294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
 
# File 'app/mailboxes/application_mailbox.rb', line 294

def create_communication(resource, mail, note, comm_state, comm_direction)
  sender = mail.from&.first
  sender_party = find_sender_party(mail)

  comm = Communication.new(
    resource: resource,
    subject: mail.subject || 'No subject',
    body: note,
    sender: sender,
    sender_party_id: sender_party&.id,
    state: comm_state,
    triggered_by_mailbox: true,
    mailbox_inbound_email_id: inbound_email.id,
    reply_to_full_name: sender_party&.full_name || mail&.from_address&.name,
    direction: comm_direction
  )

  comm.transmit_at = Time.current if comm_direction == 'outbound'

  comm.build_recipients mail&.to&.join(',')
  comm.build_recipients mail&.cc&.join(','), 'cc'
  comm.save!
  Rails.logger.info "Communication created from #{resource.class} Mailbox. Communication ID #{comm.id}"
  comm
end

#fetch_linked_files(communication) ⇒ void

This method returns an undefined value.

Scans the HTML body for external file links and attempts to download each one,
saving successful downloads as Upload records and rewriting the URL in the body.

Supported patterns:

  • Google Drive share links → transformed to uc?export=download
  • Any / pointing to a URL with a known file extension

Silently skips on errors (auth walls, timeouts, oversized files, HTML responses).

Parameters:

  • communication (Communication) —

    the communication whose body is scanned



99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
 
# File 'app/mailboxes/application_mailbox.rb', line 99

def fetch_linked_files(communication)
  body = communication.body.to_s
  return unless body.include?('http')

  doc = Nokogiri::HTML(body)
  url_map = collect_fetchable_urls(doc)
  return if url_map.empty?

  body_changed = false
  current_body = body

  url_map.each do |original_url, download_url|
    upload = attempt_url_download(download_url)
    next unless upload

    communication.uploads << upload
    asset_url = upload.attachment.url
    if asset_url.present? && original_url != download_url
      # Replace the original share link with our permanent asset URL
      current_body = current_body.gsub(original_url, asset_url)
      body_changed = true
    end
  rescue StandardError => e
    Rails.logger.info "[fetch_linked_files] Error for #{original_url}: #{e.class}: #{e.message}"
  end

  communication.update_column(:body, current_body) if body_changed
end

#find_sender_party(mail) ⇒ Party?

Finds the Party (preferring active records) whose contact point detail
matches the email sender address.

Parameters:

  • mail (Mail::Message) —

    the inbound email message

Returns:

  • (Party, nil) —

    the sender's party, if found



325
326
327
328
329
330
331
 
# File 'app/mailboxes/application_mailbox.rb', line 325

def find_sender_party(mail)
  return unless (sender_email = mail.from&.first&.downcase)

  sender = Party.active.order(updated_at: :desc).joins(:contact_points).find_by(contact_points: { detail: sender_email })
  sender ||= Party.order(updated_at: :desc).joins(:contact_points).find_by(contact_points: { detail: sender_email })
  sender
end

#get_resource_id ⇒ Integer

Extracts and decrypts the target resource ID from the matching recipient address.

Returns:

  • (Integer) —

    the decrypted resource ID



36
37
38
39
40
41
42
43
 
# File 'app/mailboxes/application_mailbox.rb', line 36

def get_resource_id
  Rails.logger.info "Application Mailbox, retrieving resource ID. Recipients are: #{mail.recipients.join(', ')}"
  recipient = mail.recipients.find { |r| self.class::RECIPIENT_FORMAT.match?(r) }
  resource_id_encrypted = recipient[self.class::RECIPIENT_FORMAT, 1]
  resource_id = Encryption.decrypt_string(resource_id_encrypted).to_i
  ErrorReporting.error("Incoming mail couldn't be processed. Recipient: #{recipient}") if resource_id.nil? || resource_id.zero?
  resource_id
end

#inline_or_attachment_part?(part) ⇒ Boolean

Returns true for MIME parts that should be saved as Upload records:
regular attachments (Content-Disposition: attachment or has a filename)
AND inline images identified by Content-ID (nested in multipart/related).

Parameters:

  • part (Mail::Part) —

    the MIME part to check

Returns:

  • (Boolean) —

    true when the part should be saved



204
205
206
207
208
209
 
# File 'app/mailboxes/application_mailbox.rb', line 204

def inline_or_attachment_part?(part)
  return false if %r{\Atext/}i.match?(part.content_type.to_s)
  return false if %r{\Amultipart/}i.match?(part.content_type.to_s)

  part.attachment? || part.content_id.present?
end

#process_attachments(communication, mail) ⇒ void

This method returns an undefined value.

Saves all attachments and inline images as Upload records linked to the
communication, then attempts to download files linked in the body.

Parameters:

  • communication (Communication) —

    the communication to attach uploads to

  • mail (Mail::Message) —

    the inbound email message



51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
 
# File 'app/mailboxes/application_mailbox.rb', line 51

def process_attachments(communication, mail)
  # mail.attachments misses inline images nested inside multipart/related.
  # Collect all MIME parts that are attachments OR carry a Content-ID (inline images).
  parts_to_save = mail.all_parts.select { |p| inline_or_attachment_part?(p) }.uniq

  cid_to_upload = {}

  parts_to_save.each do |part|
    filename = sanitize_attachment_filename(part.filename.presence || "attachment_#{SecureRandom.hex(4)}")
    upload = Upload.uploadify_from_data(file_name: Addressable::URI.escape(filename), data: part.decoded, category: 'email_attachment')
    if upload.present?
      communication.uploads << upload
      cid = part.content_id.to_s.delete('<>').presence
      cid_to_upload[cid] = upload if cid
    end
  rescue StandardError => e
    ErrorReporting.error(e, inbound_email_id: inbound_email.id)
  end

  replace_cid_references(communication, cid_to_upload) if cid_to_upload.any?

  # Best-effort: try to download files linked in the body (e.g. Google Drive
  # share links, direct image/PDF URLs). Silently skipped on any failure.
  fetch_linked_files(communication)
end

#process_content(mail) ⇒ String

Extracts the note body from the email, preferring the HTML part (which
preserves inline images and formatting) over the sanitized text part.

Parameters:

  • mail (Mail::Message) —

    the inbound email message

Returns:

  • (String) —

    the extracted body content



261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
 
# File 'app/mailboxes/application_mailbox.rb', line 261

def process_content(mail)
  # Prefer HTML when available: it preserves inline images (cid: references),
  # formatting, and links. Text parts only have [image: Image] placeholders.
  if mail.html_part.present?
    html = mail.html_part.decoded.force_encoding('utf-8')
    doc = Nokogiri::HTML(html)
    return doc.to_s if doc.content.strip.present?
  end

  if mail.text_part.present?
    # https://stackoverflow.com/questions/21371258/invalid-byte-sequence-using-html-sanitizer
    raw_body = mail.text_part.body.decoded.force_encoding('utf-8').encode('UTF-16', invalid: :replace, replace: '').encode('UTF-8')
    raw_body = EmailReplyParser.parse_reply(raw_body)
    note = Rails::Html::WhiteListSanitizer.new.sanitize(raw_body, tags: ['img']).presence
    return note if note.present?
  end

  begin
    mail.decoded.force_encoding('utf-8')
  rescue StandardError
    'This email has no content'
  end
end

#replace_cid_references(communication, cid_to_upload) ⇒ void

This method returns an undefined value.

After inline images are saved as uploads, replace cid: references in the
communication body with the Dragonfly URL so browsers can render them.

Parameters:

  • communication (Communication) —

    the communication whose body is rewritten

  • cid_to_upload (Hash{String => Upload}) —

    map of Content-ID to saved upload



217
218
219
220
221
222
223
224
225
226
227
228
229
 
# File 'app/mailboxes/application_mailbox.rb', line 217

def replace_cid_references(communication, cid_to_upload)
  body = communication.body.to_s
  return unless body.include?('cid:')

  cid_to_upload.each do |cid, upload|
    url = upload.attachment.url
    next if url.blank?

    body = body.gsub("cid:#{cid}", url)
  end

  communication.update_column(:body, body)
end

#sanitize_attachment_filename(original_filename) ⇒ String

Sanitizes an attachment filename (colons removed) and truncates it to
MAX_FILENAME_LENGTH bytes while preserving the extension.

Parameters:

  • original_filename (String, nil) —

    the original attachment filename

Returns:

  • (String) —

    the sanitized filename



240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
 
# File 'app/mailboxes/application_mailbox.rb', line 240

def sanitize_attachment_filename(original_filename)
  filename = original_filename.to_s.tr(':', '-') # Remove colons - they cause path issues

  return filename if filename.length <= MAX_FILENAME_LENGTH

  # Preserve extension when truncating
  extension = File.extname(filename)
  basename = File.basename(filename, extension)

  # Truncate basename, keeping room for extension
  max_basename_length = MAX_FILENAME_LENGTH - extension.length
  truncated_basename = basename[0, max_basename_length]

  "#{truncated_basename}#{extension}"
end