Class: ApplicationMailbox

Inherits:
ActionMailbox::Base
  • Object
show all
Defined in:
app/mailboxes/application_mailbox.rb

Overview

Test at https://api.warmlyyours.me:3000/rails/conductor/action_mailbox/inbound_emails
Troubleshoot action inbox
ActionMailbox::InboundEmail.failed.first
and for each one of the failed ones you update their status to 'pending'
ActionMailbox::InboundEmail.where(status: 'failed').update_all(status:  'pending')
and now you reprocess each one by doing
SupportsMailbox.new(ActionMailbox::InboundEmail.find(:id)).process
Call Route on email

On production to download an existing email and test it locally
https://world.hey.com/robzolkos/debugging-production-actionmailbox-issues-in-development-f5886579
Find the email: m = ActionMailbox::InboundEmail.find(:id)
Download the raw source and copy it to clipboard.
puts m.mail
Go to your local: https://api.warmlyyours.me:3000/en-US/rails/conductor/action_mailbox/inbound_emails/sources/new
Paste the raw source.

Constant Summary collapse

FETCH_MAX_SIZE =

Scan the HTML body for external file links and attempt to download each one,
saving successful downloads as Upload records and rewriting the URL in the body.

Supported patterns:

  • Google Drive share links → transformed to uc?export=download
  • Any / pointing to a URL with a known file extension

Silently skips on errors (auth walls, timeouts, oversized files, HTML responses).

25 * 1024 * 1024
FETCHABLE_MIME_TYPES =

25 MB — above this Google Drive shows a warning page

/\A(image\/|application\/(pdf|zip|x-zip|msword|vnd\.openxmlformats|vnd\.ms-))/i
DIRECT_FILE_EXTENSIONS = 
/\.(jpe?g|png|gif|webp|svg|pdf|docx?|xlsx?|pptx?|zip|csv)\z/i
GDRIVE_FILE_RE = 
%r{https://drive\.google\.com/file/d/([^/?#\s]+)}
GDRIVE_OPEN_RE = 
%r{https://drive\.google\.com/open\?id=([^&\s]+)}
MAX_FILENAME_LENGTH =

Sanitize and truncate attachment filenames to prevent ENAMETOOLONG errors
Filesystem limit is typically 255 bytes; we use 200 to be safe with encoding

200

Instance Method Summary collapse

Instance Method Details

#attempt_url_download(url) ⇒ Object

Attempt to download a URL, returning an Upload on success or nil on any failure.
Checks the Content-Type before accepting the response to avoid saving HTML auth pages.



117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
 
# File 'app/mailboxes/application_mailbox.rb', line 117

def attempt_url_download(url)
  require 'down'

  tempfile = Down.download(url, max_size: FETCH_MAX_SIZE) { |client| client.timeout(connect: 5, read: 15) }
  content_type = tempfile.content_type.to_s

  return nil if content_type.start_with?('text/html', 'text/plain')
  return nil unless FETCHABLE_MIME_TYPES.match?(content_type)

  filename = sanitize_attachment_filename(
    tempfile.original_filename.presence ||
      begin
        raw = URI.parse(url).path.to_s.split('?').first
        CGI.unescape(File.basename(raw)).presence
      rescue StandardError
        nil
      end ||
      "file_#{SecureRandom.hex(4)}"
  )

  Upload.uploadify(tempfile.path, 'email_attachment', nil, filename)
rescue Down::TooLarge
  Rails.logger.info "[fetch_linked_files] Skipped (too large): #{url}"
  nil
rescue Down::Error, Errno::ECONNREFUSED, SocketError, Timeout::Error, HTTP::Error => e
  Rails.logger.info "[fetch_linked_files] Skipped (#{e.class}): #{url}"
  nil
ensure
  tempfile&.close
  tempfile&.unlink
end

#collect_fetchable_urls(doc) ⇒ Object

Returns a hash of { original_url => download_url } for all candidate URLs
found in the HTML document. Google Drive share links are transformed to
direct-download equivalents. External img src are included as-is.



152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
 
# File 'app/mailboxes/application_mailbox.rb', line 152

def collect_fetchable_urls(doc)
  url_map = {}
  our_host_pattern = /warmlyyours\.(com|me)\z/i

  extract_urls = lambda do |raw|
    return unless raw.to_s.start_with?('https://', 'http://')

    uri = URI.parse(raw.to_s.strip)
    return if our_host_pattern.match?(uri.host.to_s)

    if (m = GDRIVE_FILE_RE.match(raw))
      url_map[raw] = "https://drive.google.com/uc?export=download&id=#{m[1]}"
    elsif (m = GDRIVE_OPEN_RE.match(raw))
      url_map[raw] = "https://drive.google.com/uc?export=download&id=#{m[1]}"
    elsif uri.path.to_s.match?(DIRECT_FILE_EXTENSIONS)
      url_map[raw] = raw
    end
  rescue URI::InvalidURIError
    nil
  end

  doc.css('a[href]').each { |a| extract_urls.call(a['href']) }
  doc.css('img[src]').each { |img| extract_urls.call(img['src']) unless url_map.key?(img['src']) }

  url_map
end

#create_activity(resource, act_type, assigned_resource, comm, complete_act) ⇒ Object 



283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
 
# File 'app/mailboxes/application_mailbox.rb', line 283

def create_activity(resource, act_type, assigned_resource, comm, complete_act)
  note = +''
  note << "<b style='color:red'>THIS EMAIL IS OLD. Probably processed after a technical issue.</b>\n\n" if inbound_email.created_at < 2.days.ago
  note << "Email received on #{inbound_email.created_at} and automatically processed."
  # Due date will be set to end of business day.
  target_datetime = inbound_email.created_at.to_datetime
  target_datetime = WorkingHours.advance_to_working_time(target_datetime) # First get to the first working time
  target_datetime = WorkingHours.advance_to_closing_time(target_datetime) # Then get to the closing time of that time
  a = resource.activities.create!(
    activity_type: ActivityType.find_by(task_type: act_type),
    new_note: note,
    target_datetime: target_datetime,
    original_target_datetime: target_datetime,
    assigned_resource: assigned_resource,
    communication_id: comm.id
  )
  a.complete(closed_by_id: assigned_resource) if complete_act
  Rails.logger.info "Activity created from #{resource.class} Mailbox. Activity ID #{a.id}"
end

#create_communication(resource, mail, note, comm_state, comm_direction) ⇒ Object 



249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
 
# File 'app/mailboxes/application_mailbox.rb', line 249

def create_communication(resource, mail, note, comm_state, comm_direction)
  sender = mail.from&.first
  sender_party = find_sender_party(mail)

  comm = Communication.new(
    resource: resource,
    subject: mail.subject || 'No subject',
    body: note,
    sender: sender,
    sender_party_id: sender_party&.id,
    state: comm_state,
    triggered_by_mailbox: true,
    mailbox_inbound_email_id: inbound_email.id,
    reply_to_full_name: sender_party&.full_name || mail&.from_address&.name,
    direction: comm_direction
  )

  comm.transmit_at = Time.current if comm_direction == 'outbound'

  comm.build_recipients mail&.to&.join(',')
  comm.build_recipients mail&.cc&.join(','), 'cc'
  comm.save!
  Rails.logger.info "Communication created from #{resource.class} Mailbox. Communication ID #{comm.id}"
  comm
end

#fetch_linked_files(communication) ⇒ Object 



86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
 
# File 'app/mailboxes/application_mailbox.rb', line 86

def fetch_linked_files(communication)
  body = communication.body.to_s
  return unless body.include?('http')

  doc = Nokogiri::HTML(body)
  url_map = collect_fetchable_urls(doc)
  return if url_map.empty?

  body_changed = false
  current_body = body

  url_map.each do |original_url, download_url|
    upload = attempt_url_download(download_url)
    next unless upload

    communication.uploads << upload
    asset_url = upload.attachment.url
    if asset_url.present? && original_url != download_url
      # Replace the original share link with our permanent asset URL
      current_body = current_body.gsub(original_url, asset_url)
      body_changed = true
    end
  rescue StandardError => e
    Rails.logger.info "[fetch_linked_files] Error for #{original_url}: #{e.class}: #{e.message}"
  end

  communication.update_column(:body, current_body) if body_changed
end

#find_sender_party(mail) ⇒ Object 



275
276
277
278
279
280
281
 
# File 'app/mailboxes/application_mailbox.rb', line 275

def find_sender_party(mail)
  return unless (sender_email = mail.from&.first&.downcase)

  sender = Party.active.order(updated_at: :desc).joins(:contact_points).find_by(contact_points: { detail: sender_email })
  sender ||= Party.order(updated_at: :desc).joins(:contact_points).find_by(contact_points: { detail: sender_email })
  sender
end

#get_resource_id ⇒ Object

Convenience methods for mailers



35
36
37
38
39
40
41
42
 
# File 'app/mailboxes/application_mailbox.rb', line 35

def get_resource_id
  Rails.logger.info "Application Mailbox, retrieving resource ID. Recipients are: #{mail.recipients.join(', ')}"
  recipient = mail.recipients.find { |r| self.class::RECIPIENT_FORMAT.match?(r) }
  resource_id_encrypted = recipient[self.class::RECIPIENT_FORMAT, 1]
  resource_id = Encryption.decrypt_string(resource_id_encrypted).to_i
  ErrorReporting.error("Incoming mail couldn't be processed. Recipient: #{recipient}") if resource_id.nil? || resource_id.zero?
  resource_id
end

#inline_or_attachment_part?(part) ⇒ Boolean

Returns true for MIME parts that should be saved as Upload records:
regular attachments (Content-Disposition: attachment or has a filename)
AND inline images identified by Content-ID (nested in multipart/related).

Returns:

  • (Boolean) 


182
183
184
185
186
187
 
# File 'app/mailboxes/application_mailbox.rb', line 182

def inline_or_attachment_part?(part)
  return false if part.content_type.to_s =~ /\Atext\//i
  return false if part.content_type.to_s =~ /\Amultipart\//i

  part.attachment? || part.content_id.present?
end

#process_attachments(communication, mail) ⇒ Object 



44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'app/mailboxes/application_mailbox.rb', line 44

def process_attachments(communication, mail)
  # mail.attachments misses inline images nested inside multipart/related.
  # Collect all MIME parts that are attachments OR carry a Content-ID (inline images).
  parts_to_save = mail.all_parts.select { |p| inline_or_attachment_part?(p) }.uniq

  cid_to_upload = {}

  parts_to_save.each do |part|
    begin
      filename = sanitize_attachment_filename(part.filename.presence || "attachment_#{SecureRandom.hex(4)}")
      upload = Upload.uploadify_from_data(file_name: Addressable::URI.escape(filename), data: part.decoded, category: 'email_attachment')
      if upload.present?
        communication.uploads << upload
        cid = part.content_id.to_s.delete('<>').presence
        cid_to_upload[cid] = upload if cid
      end
    rescue StandardError => e
      ErrorReporting.error(e, inbound_email_id: inbound_email.id)
    end
  end

  replace_cid_references(communication, cid_to_upload) if cid_to_upload.any?

  # Best-effort: try to download files linked in the body (e.g. Google Drive
  # share links, direct image/PDF URLs). Silently skipped on any failure.
  fetch_linked_files(communication)
end

#process_content(mail) ⇒ Object 



225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
 
# File 'app/mailboxes/application_mailbox.rb', line 225

def process_content(mail)
  # Prefer HTML when available: it preserves inline images (cid: references),
  # formatting, and links. Text parts only have [image: Image] placeholders.
  if mail.html_part.present?
    html = mail.html_part.decoded.force_encoding('utf-8')
    doc = Nokogiri::HTML(html)
    return doc.to_s if doc.content.strip.present?
  end

  if mail.text_part.present?
    # https://stackoverflow.com/questions/21371258/invalid-byte-sequence-using-html-sanitizer
    raw_body = mail.text_part.body.decoded.force_encoding('utf-8').encode('UTF-16', invalid: :replace, replace: '').encode('UTF-8')
    raw_body = EmailReplyParser.parse_reply(raw_body)
    note = Rails::Html::WhiteListSanitizer.new.sanitize(raw_body, tags: ['img']).presence
    return note if note.present?
  end

  begin
    mail.decoded.force_encoding('utf-8')
  rescue StandardError
    'This email has no content'
  end
end

#replace_cid_references(communication, cid_to_upload) ⇒ Object

After inline images are saved as uploads, replace cid: references in the
communication body with the Dragonfly URL so browsers can render them.



191
192
193
194
195
196
197
198
199
200
201
202
203
 
# File 'app/mailboxes/application_mailbox.rb', line 191

def replace_cid_references(communication, cid_to_upload)
  body = communication.body.to_s
  return unless body.include?('cid:')

  cid_to_upload.each do |cid, upload|
    url = upload.attachment.url
    next unless url.present?

    body = body.gsub("cid:#{cid}", url)
  end

  communication.update_column(:body, body)
end

#sanitize_attachment_filename(original_filename) ⇒ Object 



209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
 
# File 'app/mailboxes/application_mailbox.rb', line 209

def sanitize_attachment_filename(original_filename)
  filename = original_filename.to_s.tr(':', '-') # Remove colons - they cause path issues

  return filename if filename.length <= MAX_FILENAME_LENGTH

  # Preserve extension when truncating
  extension = File.extname(filename)
  basename = File.basename(filename, extension)

  # Truncate basename, keeping room for extension
  max_basename_length = MAX_FILENAME_LENGTH - extension.length
  truncated_basename = basename[0, max_basename_length]

  "#{truncated_basename}#{extension}"
end