Class: Retailer::DailyCostGuard

Inherits:
Object
  • Object
show all
Defined in:
app/services/retailer/daily_cost_guard.rb

Overview

Daily request-volume circuit breaker for the Oxylabs API.

In May 2026 we blew past our $49/mo Oxylabs budget mid-month with no early
warning. This guard caps daily request volume so a runaway worker, infinite
retry loop, or new caller can't drain the budget unattended.

Backed by Rails.cache (Redis) with a per-day key. On the first request of
a UTC day the counter is initialized with a 36h TTL; subsequent requests
atomically increment. When the limit is crossed we raise BudgetExceeded,
which OxylabsApi catches and converts into a normal failure
Result so callers see "API unavailable" rather than a crash.

Tunable via the oxylabs.daily_request_limit credential. Disabled in tests
to avoid touching the cache.

Examples:

Retailer::DailyCostGuard.check_and_increment!  # raises BudgetExceeded if over

Defined Under Namespace

Classes: BudgetExceeded

Constant Summary collapse

DEFAULT_DAILY_LIMIT =

Default daily ceiling. Picked to be ~10% above the historical p95
(~2,400/day) so genuine peak load isn't blocked, but a runaway loop is.

2_750
KEY_PREFIX =

Cache key prefix; full key includes the UTC date.

'oxylabs:daily_request_count'
KEY_TTL =

Key TTL — covers the full day plus a 12h safety margin.

36.hours

Class Method Summary collapse

Class Method Details

.check_and_increment!(count: 1) ⇒ Integer

Increment today's counter and raise if we crossed the limit.

Parameters:

  • count (Integer) (defaults to: 1) —

    how many requests this call represents (default 1)

Returns:

  • (Integer) —

    the new counter value

Raises:

  • (BudgetExceeded) —

    when the post-increment value exceeds the limit



41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
 
# File 'app/services/retailer/daily_cost_guard.rb', line 41

def check_and_increment!(count: 1)
  return 0 if disabled?

  # Bypass the RailsBrotliCache wrapper around Rails.cache: it compresses
  # values with Brotli, which makes Redis INCR fail with "value is not an
  # integer". The underlying RedisCacheStore's #increment is atomic INCRBY
  # and seeds the key to 0 itself when missing.
  new_count = counter_store.increment(redis_key, count, expires_in: KEY_TTL).to_i

  if new_count > daily_limit
    Rails.logger.error("[DailyCostGuard] Oxylabs daily limit exceeded: #{new_count}/#{daily_limit}")
    raise BudgetExceeded, "Daily Oxylabs request budget exceeded (#{new_count}/#{daily_limit})"
  end

  new_count
end

.current_count ⇒ Integer

Read today's counter without modifying it. Useful for diagnostics and
admin pages.

Returns:

  • (Integer) 


91
92
93
94
95
 
# File 'app/services/retailer/daily_cost_guard.rb', line 91

def current_count
  return 0 if disabled?

  (counter_store.read(redis_key, raw: true) || 0).to_i
end

.daily_limit ⇒ Integer

Returns:

  • (Integer) 


104
105
106
107
108
 
# File 'app/services/retailer/daily_cost_guard.rb', line 104

def daily_limit
  Heatwave::Configuration.fetch(:oxylabs, :daily_request_limit) || DEFAULT_DAILY_LIMIT
rescue StandardError
  DEFAULT_DAILY_LIMIT
end

.disabled? ⇒ Boolean

Returns true in test env or when the credential opts out.

Returns:

  • (Boolean) —

    true in test env or when the credential opts out



111
112
113
114
115
116
117
 
# File 'app/services/retailer/daily_cost_guard.rb', line 111

def disabled?
  return true if Rails.env.test?

  Heatwave::Configuration.fetch(:oxylabs, :daily_request_limit_disabled) == true
rescue StandardError
  false
end

.ensure_within_budget! ⇒ void

This method returns an undefined value.

Pre-flight ceiling check that does NOT consume budget. Raises when
today's count has already reached the limit; otherwise returns. Pair it
with record! (called only once a request is actually accepted) so
rejected/throttled submissions don't burn budget, while a genuine
runaway loop is still blocked the moment the ceiling is reached.

Raises:



66
67
68
69
70
71
72
73
74
 
# File 'app/services/retailer/daily_cost_guard.rb', line 66

def ensure_within_budget!
  return if disabled?

  count = current_count
  return if count < daily_limit

  Rails.logger.error("[DailyCostGuard] Oxylabs daily limit reached: #{count}/#{daily_limit}")
  raise BudgetExceeded, "Daily Oxylabs request budget exceeded (#{count}/#{daily_limit})"
end

.record!(count: 1) ⇒ Integer

Count count accepted request(s) against today's budget. Call this only
after Oxylabs actually accepts the work (a queued async job or a completed
realtime query) so the counter tracks real spend, not submission attempts.

Parameters:

  • count (Integer) (defaults to: 1) —

    how many accepted requests to record (default 1)

Returns:

  • (Integer) —

    the new counter value



82
83
84
85
86
 
# File 'app/services/retailer/daily_cost_guard.rb', line 82

def record!(count: 1)
  return 0 if disabled?

  counter_store.increment(redis_key, count, expires_in: KEY_TTL).to_i
end

.reset! ⇒ void

This method returns an undefined value.

Test helper: clears today's counter.



99
100
101
 
# File 'app/services/retailer/daily_cost_guard.rb', line 99

def reset!
  counter_store.delete(redis_key)
end