I had a very frequent cronjob that in rare cases could be relatively slow. To avoid multiple instances of this cronjob running in parallel, I decided to use flock to ensure that only one instance could run at a time.
flock works by acquiring a lock to a file, and if it can do so running a command. In order not to wait but simply give up when the file is locked, you can add -n:
flock /tmp/my.task.lock -n -c "bin/my-long-running-job"
Using whenever Show archive.org snapshot , and since this was a rake task, the following did the trick:
job_type :exclusive_rake, "cd :path && flock -n 'tmp/pids/cron_:task.lock' -c ':environment_variable=:environment bundle exec rake :task --silent --backtrace :output'"
every 1.minute do
exclusive_rake 'my:job:name'
end
Don't let flock confuse you: The lock file will not be removed when the job is done. It simply won't be locked. The --backtrace option is explained here Adjust cron jobs to allow full backtraces for rake tasks.
Posted by Tobias Kraze to makandra dev (2015-11-17 16:24)