Redis Cache Migration

Overview

Migrated from Memcached (Dalli) to Redis for all caching needs, consolidating all caching infrastructure into Redis with separate databases for different cache types.

Problem Solved

  • Before: Memcached + Redis for different purposes
  • After: Redis-only for all caching, sessions, and background jobs

Changes Made

1. Cache Store Configuration (config/initializers/redis_cache.rb)

  • Replaced MemCacheStore with RedisCacheStore
  • Maintained RailsBrotliCache wrapper for compression
  • Added separate API cache store for cached_resource

2. Database Assignments

  • Database 4: Rails cache (fragment caching, view caching)
  • Database 5: API cache (cached_resource, API responses)

3. Updated Models

  • File: app/models/api/image.rb
  • Change: Uses Rails.application.config.api_cache_store instead of MemCacheStore
  • Benefit: Consistent Redis-based caching across all API resources

4. Gem Dependencies

  • Removed: dalli gem (Memcached client)
  • Existing: redis, redis-rails, redis-namespace already present

Configuration Details

Rails Cache Store

config.cache_store = RailsBrotliCache::Store.new(
  ActiveSupport::Cache::RedisCacheStore.new(
    url: RedisConfig.rails_cache_url,
    namespace: 'rails_cache',
    expires_in: 1.hour,
    race_condition_ttl: 10.seconds
  )
)

API Cache Store

Rails.application.config.api_cache_store = ActiveSupport::Cache::RedisCacheStore.new(
  url: RedisConfig.api_cache_url,
  namespace: 'api_cache',
  expires_in: 1.hour,
  race_condition_ttl: 10.seconds
)

Benefits

1. Unified Infrastructure

  • Single Redis instance for all caching needs
  • Consistent configuration and monitoring
  • Easier deployment and maintenance

2. Better Performance

  • Redis is generally faster than Memcached
  • Better memory efficiency
  • Advanced data structures and operations

3. Enhanced Features

  • Automatic expiration with TTL
  • Race condition protection
  • Namespace isolation
  • Better persistence options

4. Simplified Operations

  • One less service to monitor (no Memcached)
  • Unified Redis monitoring
  • Consistent backup/restore procedures

Database Usage

Database Service Purpose TTL
0 Sessions Rails session storage 7 days
1 Geocoder Location caching 7 days
2 Action Cable WebSocket connections Session-based
3 Sidekiq Background jobs Job-specific
4 Rails Cache Fragment/view caching 1 hour default
5 API Cache API response caching 1 hour default

Migration Steps

1. Deploy Configuration

# Deploy the new cache configuration
git add config/initializers/redis_cache.rb
git add app/models/api/image.rb
git commit -m "Migrate from Memcached to Redis cache"
git push

2. Update Production

# Deploy to production
cap production deploy

3. Test Configuration

# Test Redis cache configuration
rails runner script/test_redis_cache.rb

4. Remove Memcached

# Stop Memcached service (if running)
sudo systemctl stop memcached
sudo systemctl disable memcached

# Remove Memcached package (optional)
sudo apt-get remove memcached

Testing

1. Cache Functionality

# Test basic cache operations
rails runner script/test_redis_cache.rb

2. API Cache

# Test API cache in console
rails console
Api::Image.clear_cache

3. Rails Cache

# Test Rails cache in console
rails console
Rails.cache.write('test', 'value')
Rails.cache.read('test')

Monitoring

Redis Memory Usage

# Check Redis memory usage
redis-cli info memory

Cache Key Monitoring

# Check cache keys by namespace
redis-cli keys "rails_cache:*" | wc -l
redis-cli keys "api_cache:*" | wc -l

Cache Performance

# Monitor cache hit/miss rates
redis-cli info stats | grep keyspace

Rollback Plan

If issues arise, rollback by:

  1. Restore Memcached Configuration:
# config/application.rb
config.cache_store = RailsBrotliCache::Store.new(
  ActiveSupport::Cache::MemCacheStore.new
)
  1. Restore API Cache:
# app/models/api/image.rb
cached_resource enabled: true, ttl: 1.hour.to_i, 
  cache: ActiveSupport::Cache::MemCacheStore.new(namespace: 'image_api')
  1. Add Dalli Gem Back:
# Gemfile
gem 'dalli'

Performance Considerations

Memory Usage

  • Redis uses more memory per key than Memcached
  • Monitor memory usage and adjust Redis maxmemory settings
  • Consider using Redis eviction policies

Network Latency

  • Redis may have slightly higher latency than Memcached
  • Consider Redis clustering for high-availability setups
  • Monitor Redis performance metrics

Persistence

  • Redis can persist data to disk (Memcached cannot)
  • Configure appropriate persistence settings
  • Consider RDB snapshots and AOF logging

This migration consolidates all caching infrastructure into Redis, providing better performance, consistency, and maintainability while eliminating the need for Memcached.