Class: EmbeddingBatchQueueWorker

Inherits:
Object
  • Object
show all
Includes:
Sidekiq::Job, Workers::StatusBroadcastable
Defined in:
app/workers/embedding_batch_queue_worker.rb

Overview

Background worker that queues embedding jobs in batches.
This avoids timeout issues when queueing large numbers of records from the web UI.

Examples:

Queue all images for full analysis

EmbeddingBatchQueueWorker.perform_async('Image')

Queue articles of a specific STI type

EmbeddingBatchQueueWorker.perform_async('Article', article_type: 'ArticleFaq')

Instance Attribute Summary

Attributes included from Workers::StatusBroadcastable

#broadcast_status_updates

Instance Method Summary collapse

Methods included from Workers::StatusBroadcastable::Overrides

#at, #store, #total

Instance Method Details

#perform(type, options = {}) ⇒ Object

Parameters:

  • type (String) —

    The embeddable type to queue

  • options (Hash) (defaults to: {}) —

    Additional options

Options Hash (options):

  • :article_type (String) —

    Article subtype filter

  • :force (Boolean) —

    Force regeneration even if already embedded



23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
 
# File 'app/workers/embedding_batch_queue_worker.rb', line 23

def perform(type, options = {})
  store started_at: Time.current.iso8601
  store type: type

  count = case type
          when 'Post' then queue_posts
          when 'Article' then queue_articles(options[:article_type])
          when 'Showcase' then queue_showcases
          when 'Video' then queue_videos
          when 'Image' then queue_images
          when 'SiteMap' then queue_sitemaps
          when 'ReviewsIo' then queue_reviews
          when 'Item' then queue_items
          when 'CallRecord' then queue_call_records
          when 'ProductLine' then queue_product_lines
          when 'Activity' then queue_activities
          when 'Communication' then queue_communications
          when 'AssistantBrainEntry' then queue_brain_entries
          when 'ImageFingerprints' then queue_image_fingerprints
          else
            store error: "Unknown type: #{type}"
            0
          end

  store completed_at: Time.current.iso8601
  store queued_count: count

  Rails.logger.info "[EmbeddingBatchQueueWorker] Queued #{count} #{type} records"
end