Posted about 3 years ago. Visible to the public.

Background processing

Some tasks in a web application are better not done live when a user request a page, but in the background. Examples are

  • longer running tasks
  • tasks that are not tied to user interaction
  • tasks that can fail, and may need to be retried

Our two main mechanisms for background processing are

Learn about cronjobs

  • Read HowTo: Add Jobs To cron Under Linux or UNIX?
  • Understand how we manage cronjobs with whenever.
  • Find a project that uses whenever. Find out what kinds of work is done in those cronjobs. Can you find the code? The tests?
  • What happens when an error occurs? How do we get notified?

Learn about Sidekiq

  • Read Sidekiq's Readme
  • Read Sidekiq best practices
  • Understand how to schedule jobs, and work them off
  • What happens when an exception occurs? How do we get notified?
  • Sidekiq uses threads
    • Discuss advantages and drawbacks with your mentor
    • How problematic do you think this is? To what do you have to pay attention?
  • Sidekiq needs Redis
    • For what?
    • Why Redis (and not MySQL, or memcache)?
  • Sidekiq jobs should never be enqueued in an after_create or after_save callback. Can you find out why?
  • Can cronjobs and Sidekiq be combined? Name at least 2 reasons why this would make sense.
  • Find out how Sidekiq fits into our infrastructure. Is there failover? Where does it run?

Exercise: Move API calls to a Sidekiq job

You've previously implemented calls to external APIs in your MovieDB.

  • Choose one of those and move it into a background job using Sidekiq.
  • Make sure your code is (still) tested. Read the Sidekiq testing guide for some pointers.
  • Sidekiq comes with a Web UI. Mount it into your MovieDB, so you can see your Sidekiq jobs under /sidekiq.
  • Simulate the API being unreliable. Perhaps add a random TimeoutError. Can you see what happens to your job in the Web UI?

Owner of this card:

Avatar
Tobias Kraze
Last edit:
about 3 years ago
by Tobias Kraze
Posted by Tobias Kraze to makandra Curriculum
This website uses cookies to improve usability and analyze traffic.
Accept or learn more