Posted almost 6 years ago. Visible to the public.

Background processing [2d]

Some tasks in a web application are better not done live when a user request a page, but in the background. Examples are

  • longer running tasks
  • tasks that are not tied to user interaction
  • tasks that can fail, and may need to be retried

Our two main mechanisms for background processing are

Learn about cronjobs

Learn about Sidekiq

  • Read Sidekiq's Readme
  • Read Sidekiq best practices
  • Understand how to schedule jobs, and work them off
  • Understand how ActiveJob integrates with Sidekiq
  • What happens when an exception occurs? How do we get notified?
  • Sidekiq uses threads
    • Discuss advantages and drawbacks with your mentor
    • How problematic do you think this is? To what do you have to pay attention?
  • Sidekiq needs Redis
    • For what?
    • Why Redis (and not MySQL, or memcache)?
  • Sidekiq jobs should never be enqueued in an after_create or after_save callback. Can you find out why?
  • Can cronjobs and Sidekiq be combined? Name at least 2 reasons why this would make sense.
  • Find out how Sidekiq fits into our infrastructure. Is there failover? Where does it run?

Exercise: Move API calls to a Sidekiq job

You've previously implemented calls to external APIs in your MovieDB.

  • Choose one of those and move it into a background job using Sidekiq. The automatic fetching of the release year is a good candidate.
  • Make sure your code is (still) tested. Read the Sidekiq testing guide for some pointers.
  • Sidekiq comes with a Web UI. Mount it into your MovieDB, so you can see your Sidekiq jobs under /sidekiq.
  • Simulate the API being unreliable. Perhaps add a random TimeoutError. Can you see what happens to your job in the Web UI?
  • If you want to start your Sidekiq during deployment, use makandra-sidekiq.

Owner of this card:

Tobias Kraze
Last edit:
3 months ago
by Henning Koch
Posted by Tobias Kraze to makandra Curriculum
This website uses short-lived cookies to improve usability.
Accept or learn more