Exclusive cronjobs with flock and whenever

I had a very frequent cronjob that in rare cases could be relatively slow. To avoid multiple instances of this cronjob running in parallel, I decided to use flock to ensure that only one instance could run at a time.

flock works by acquiring a lock to a file, and if it can do so running a command. In order not to wait but simply give up when the file is locked, you can add -n:

Copyflock /tmp/my.task.lock -n -c "bin/my-long-running-job"

Using whenever Show archive.org snapshot , and since this was a rake task, the following did the trick:

Copyjob_type :exclusive_rake, "cd :path && flock -n 'tmp/pids/cron_:task.lock' -c ':environment_variable=:environment bundle exec rake :task --silent --backtrace :output'"

every 1.minute do
  exclusive_rake 'my:job:name'
end

Don't let flock confuse you: The lock file will not be removed when the job is done. It simply won't be locked. The --backtrace option is explained here Adjust cron jobs to allow full backtraces for rake tasks.