Class: Communication::ClickBotScorer

Inherits:
BaseService show all
Defined in:
app/services/communication/click_bot_scorer.rb

Overview

Scores a set of CommunicationRecipients' link clicks to separate
security-scanner traffic from human clicks, persisting the verdict to
CommunicationRecipient#machine_clicked.

Corporate link scanners (Proofpoint, Mimecast, Microsoft Defender Safe Links,
Barracuda, …) fetch every URL in an email, inflating click rates. Unlike opens,
clicks carry no SendGrid sg_machine_open flag, so the signal is behavioral and
needs cross-recipient / cross-link context — it can't be decided per event at
ingestion. This service aggregates a set of clicks and flags three scanner
signatures:

  1. Shared-IP fan-out — one IP clicking on behalf of many recipients in the
    scored set is a corporate scanner egress, not a person
    (SHARED_IP_RECIPIENTS+ recipients).
  2. Rapid multi-link burst — BURST_CLICKS+ distinct links clicked within
    BURST_WINDOWs by one recipient; no human reads and opens that fast.
  3. Scanner-IP registry — IPs proven to be scanners by an invisible-canary
    trip anywhere in the last REGISTRY_WINDOW (ClickBotScorer.known_scanner_ips), passed in
    as +extra_scanner_ips+. Lets a transactional send — which carries no canary of
    its own — inherit scanner IPs learned from marketing / campaign sends.

Campaign sends are scored per-send by CampaignEmailClickScoringWorker a few
hours after transmission (ClickBotScorer.for_campaign); non-campaign / transactional sends are
swept by NonCampaignClickScoringWorker.

A recipient is machine_clicked when every click is attributable to a scanner
signal; a single clean click marks them human. Recipients with no clicks are left
nil (unscored), which reporting treats as human — so legacy/unscored sends keep
their existing numbers.

Examples:

Communication::ClickBotScorer.for_campaign(campaign_email).process
# => { scored: 380, machine: 250, human: 130, scanner_ips: 7 }

Constant Summary collapse

SHARED_IP_RECIPIENTS =

Distinct recipients an IP must touch in one send to be treated as a scanner.

3
BURST_CLICKS =

Clicks within BURST_WINDOW that mark a recipient's session as a scanner burst.

3
BURST_WINDOW =

Burst detection window, in seconds.

5
REGISTRY_WINDOW =

How far back a canary trip still vouches that an IP is a scanner.

30.days

Instance Attribute Summary

Attributes inherited from BaseService

#options

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from BaseService

#log_debug, #log_error, #log_info, #log_warning, #logger, #tagged_logger

Constructor Details

#initialize(recipients, extra_scanner_ips: [], **options) ⇒ ClickBotScorer

Returns a new instance of ClickBotScorer.

Parameters:

  • recipients (ActiveRecord::Relation<CommunicationRecipient>) —

    the set to score

  • extra_scanner_ips (Enumerable<String>) (defaults to: []) —

    IPs already known to be scanners
    (e.g. known_scanner_ips) — merged into this set's own scanner signals so a
    send that carries no canary can still inherit a learned scanner IP

  • options (Hash) —

    forwarded to BaseService



51
52
53
54
55
 
# File 'app/services/communication/click_bot_scorer.rb', line 51

def initialize(recipients, extra_scanner_ips: [], **options)
  super(options)
  @recipients = recipients
  @extra_scanner_ips = extra_scanner_ips.to_set
end

Class Method Details

.for_campaign(campaign_email) ⇒ Communication::ClickBotScorer

Score a campaign send. Behaviour is unchanged from the original per-send scorer.

Parameters:

Returns:



61
62
63
 
# File 'app/services/communication/click_bot_scorer.rb', line 61

def self.for_campaign(campaign_email, **)
  new(campaign_email.communication_recipients, **)
end

.known_scanner_ips(since: REGISTRY_WINDOW.ago) ⇒ Set<String>

IPs proven to be security scanners by an invisible-canary trip in the last
+since+ window — a fleet-wide registry usable to score sends that carry no
canary of their own (transactional email). A canary fetch is provable: no human
can see or click an invisible, display:none link, so the IP is a scanner.

Parameters:

  • since (ActiveSupport::TimeWithZone) (defaults to: REGISTRY_WINDOW.ago)

Returns:

  • (Set<String>) 


72
73
74
75
76
77
 
# File 'app/services/communication/click_bot_scorer.rb', line 72

def self.known_scanner_ips(since: REGISTRY_WINDOW.ago)
  CommunicationRecipient
    .where(canary_tripped_at: since..)
    .where.not(canary_ip: nil)
    .distinct.pluck(:canary_ip).to_set
end

Instance Method Details

#process ⇒ Hash

Score the send and persist CommunicationRecipient#machine_clicked.

Returns:

  • (Hash) —

    summary counts (:scored, :machine, :human, :scanner_ips)



82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
 
# File 'app/services/communication/click_bot_scorer.rb', line 82

def process
  rows = click_rows
  return { scored: 0, machine: 0, human: 0, scanner_ips: 0 } if rows.empty?

  scanners = scanner_ips(rows)
  machine_ids = []
  human_ids = []

  rows.group_by { |r| r[:recipient_id] }.each do |recipient_id, clicks|
    (machine?(clicks, scanners) ? machine_ids : human_ids) << recipient_id
  end

  persist(machine_ids, human_ids)
  { scored: machine_ids.size + human_ids.size, machine: machine_ids.size,
    human: human_ids.size, scanner_ips: scanners.size }
end