How to implement simple queue limiting/throttling for Sidekiq

The sidekiq-rate-limiter Show archive.org snapshot gem allows rate-limiting Sidekiq jobs and works like a charm. However, it needs to be integrated on a per-worker basis.

If you want to limit a whole queue instead, and if your requirements are simple enough, you can do it via a Sidekiq middleware Show archive.org snapshot yourself.

Here is an example that limits concurrency of the "mailers" queue to 1. It uses a database mutex via the with_advisory_lock Show archive.org snapshot gem but you could easily use something else.

Copyclass SidekiqMailersThrottle
  def call(worker, job_options, queue)
    if queue == 'mailers'
      ApplicationRecord.with_advisory_lock('mailers-throttle') do
        yield
      end
    else
      yield
    end
  end
end

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add SidekiqMailersThrottle
  end
end

Some hints:

• Using this approach, workers will be blocked while jobs are waiting for the mutex to free up. The sidekiq-rate-limiter gem re-queues blocked jobs so that other jobs can be worked off in the meantime (which is smart). Depending on your use case, blocking workers might or might not be an issue.
• To increase concurrency, you might do something as simple as naming your lock "mailers-throttle-#{rand(4)}" (not perfect, but might be good enough).
• For rate-limiting (e.g. only 1 job per 5 seconds) you could name it "mailers-throttle-#{(Time.now.to_i / 5.0).to_i}".

Also note that Sidekiq allows configuring queue weights Show archive.org snapshot . This is a different approach and will not throttle a queue, but instead allow some queues to be picked up more often than others.
Finally, it might be an option to spawn a separate Sidekiq process to work off only your a specific queue, and limit its worker count.