Read more

Heads up: Sidekiq per default silently fails when retries are exhausted!

Deleted user #4117
August 29, 2018Software engineer

For Sidekiq to be able to retry your jobs it has to be able to catch errors that occur while a job is executed.

Illustration UI/UX Design

UI/UX Design by makandra brand

We make sure that your target audience has the best possible experience with your digital product. You get:

  • Design tailored to your audience
  • Proven processes customized to your needs
  • An expert team of experienced designers
Read more Show archive.org snapshot

Per default, Sidekiq will not raise / notify you if the retry count is exhausted. It will only copy the job to the dead queue (see wiki Show archive.org snapshot ).

If you want to get notified, you have to implement it in your worker explicitly with a sidekiq_retries_exhausted-block, e.g. like this:

class DownloadWorker
  include Sidekiq::Worker

  # Import jobs are retried a few times without sending exception notifications.
  # Only after the configured retries are exhausted we send a notification. That
  # way we are only notified about permanent issues.
  sidekiq_retries_exhausted do |error_info, exception|
    ExceptionNotifier.notify_exception exception, data: { sidekiq: error_info }
  end

  def perform(record_class, record_id)
    # ...
  end

end

Since v 5.1, there is also a way to configure this globally Show archive.org snapshot .

Be aware, that if you use the standard count for retries it will take a while until you are notified of your error (~21 days). You could also send a notification each third retry or use less retries.

Also, if you make thousands of calls to an API it may not be wise to send emails if the API is down. Instead you could implement sending an error notification if the data (in the db) is too old.

Testing:

# Use this worker to test if the exception notifier for sidekiq works as expected and the retries total duration fits.
# Example: TestWorker.perform_async('foo', 'bar')

class TestWorker
  include Sidekiq::Worker

  def perform(*args)
    raise 'This is a test error. All is well.'
  end
end
Posted to makandra dev (2018-08-29 11:18)