The sidekiq-rate-limiter Show archive.org snapshot gem allows rate-limiting Sidekiq jobs and works like a charm. However, it needs to be integrated on a per-worker basis.
If you want to limit a whole queue instead, and if your requirements are simple enough, you can do it via a Sidekiq middleware Show archive.org snapshot yourself.
Here is an example that limits concurrency of the "mailers" queue to 1. It uses a database mutex via the with_advisory_lock Show archive.org snapshot gem but you could easily use something else.
class SidekiqMailersThrottle
def call(worker, job_options, queue)
if queue == 'mailers'
ApplicationRecord.with_advisory_lock('mailers-throttle') do
yield
end
else
yield
end
end
end
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add SidekiqMailersThrottle
end
end
Some hints:
"mailers-throttle-#{rand(4)}"
(not perfect, but might be good enough)."mailers-throttle-#{(Time.now.to_i / 5.0).to_i}"
.Also note that Sidekiq
allows configuring queue weights
Show archive.org snapshot
. This is a different approach and will not throttle a queue, but instead allow some queues to be picked up more often than others.
Finally, it might be an option to spawn a separate Sidekiq process to work off only your a specific queue, and limit its worker count.