You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
105 lines
5.0 KiB
105 lines
5.0 KiB
<h1>Monitoring Cron Jobs</h1>
|
|
<p>SITE_NAME is perfectly suited for monitoring cron jobs. All you have to do is
|
|
update your cron job command to send an HTTP request to SITE_NAME
|
|
after completing the job.</p>
|
|
<p>Let's look at an example:</p>
|
|
<div class="highlight"><pre><span></span><code>$ crontab -l
|
|
<span class="c1"># m h dom mon dow command</span>
|
|
<span class="m">8</span> <span class="m">6</span> * * * /home/user/backup.sh
|
|
</code></pre></div>
|
|
|
|
<p>The above job runs <code>/home/user/backup.sh</code> every day at 6:08. The backup
|
|
script is presumably a headless, background process. Even if it works
|
|
correctly currently, it can start silently failing in the future without
|
|
anyone noticing.</p>
|
|
<p>You can set up SITE_NAME to notify you whenever the backup script does not
|
|
run on time, or it does not complete successfully. Here are the steps to do that.</p>
|
|
<ol>
|
|
<li>
|
|
<p>If you have not already, sign up for a free SITE_NAME account.</p>
|
|
</li>
|
|
<li>
|
|
<p>In your SITE_NAME account, <strong>add a new check</strong>.</p>
|
|
</li>
|
|
<li>
|
|
<p>Give the check <strong>a meaningful name</strong>. Good naming will become
|
|
increasingly important as you add more checks to your account.</p>
|
|
</li>
|
|
<li>
|
|
<p>Edit the check's <strong>schedule</strong>:</p>
|
|
<ul>
|
|
<li>change its type from "Simple" to "Cron"</li>
|
|
<li>enter <code>8 6 * * *</code> in the cron expression field</li>
|
|
<li>set the timezone to match your machine's timezone</li>
|
|
</ul>
|
|
</li>
|
|
<li>
|
|
<p>Take note of your check's unique <strong>ping URL</strong>.</p>
|
|
</li>
|
|
</ol>
|
|
<p>Finally, edit your cron job definition and append a curl or wget call
|
|
after the command:</p>
|
|
<div class="highlight"><pre><span></span><code>$ crontab -e
|
|
<span class="c1"># m h dom mon dow command</span>
|
|
<span class="m">8</span> <span class="m">6</span> * * * /home/user/backup.sh <span class="o">&&</span> curl -fsS --retry <span class="m">5</span> -o /dev/null PING_URL
|
|
</code></pre></div>
|
|
|
|
<p>Now, each time your cron job runs, it will send an HTTP request to the ping URL.
|
|
Since SITE_NAME knows your cron job's schedule, it can calculate
|
|
the dates and times when the job should run. As soon as your cron job doesn't
|
|
report at an expected time, SITE_NAME will send you a notification.</p>
|
|
<p>This monitoring technique takes care of various failure scenarios that could
|
|
potentially go unnoticed otherwise:</p>
|
|
<ul>
|
|
<li>The whole machine goes down (power outage, janitor stumbles on wires, VPS provider problems, etc.)</li>
|
|
<li>the cron daemon is not running or has an invalid configuration</li>
|
|
<li>cron does start your task, but the task exits with a non-zero exit code</li>
|
|
</ul>
|
|
<h2>Curl Options</h2>
|
|
<p>The extra options in the above example tell curl to retry failed HTTP requests, and
|
|
silence output unless there is an error. Feel free to adjust the curl options to
|
|
suit your needs.</p>
|
|
<dl>
|
|
<dt><strong>&&</strong></dt>
|
|
<dd>Run curl only if <code>/home/user/backup.sh</code> exits with an exit code 0.</dd>
|
|
<dt><strong>-f, --fail</strong></dt>
|
|
<dd>Makes curl treat non-200 responses as errors.</dd>
|
|
<dt><strong>-s, --silent</strong></dt>
|
|
<dd>Silent or quiet mode. Hides the progress meter, but also hides error messages.</dd>
|
|
<dt><strong>-S, --show-error</strong></dt>
|
|
<dd>Re-enables error messages when -s is used.</dd>
|
|
<dt><strong>--retry <num></strong></dt>
|
|
<dd>If a transient error is returned when curl tries to perform a
|
|
transfer, it will retry this number of times before giving up.
|
|
Setting the number to 0 makes curl do no retries (which is the default).
|
|
Transient error is a timeout or an HTTP 5xx response code.</dd>
|
|
<dt><strong>-o /dev/null</strong></dt>
|
|
<dd>Redirect curl's stdout to /dev/null (error messages still go to stderr).</dd>
|
|
</dl>
|
|
<h2>Looking up Your Machine's Time Zone</h2>
|
|
<p>If your cron job consistently pings SITE_NAME an hour early or an hour late,
|
|
the likely cause is a timezone mismatch: your machine may be using a timezone
|
|
different from what you have configured on SITE_NAME.</p>
|
|
<p>On modern GNU/Linux systems, you can look up the time zone using the
|
|
<code>timedatectl status</code> command and looking for "Time zone" in its output:</p>
|
|
<div class="highlight"><pre><span></span><code>$ timedatectl status
|
|
|
|
Local time: C 2020-01-23 12:35:50 EET
|
|
Universal time: C 2020-01-23 10:35:50 UTC
|
|
RTC time: C 2020-01-23 10:35:50
|
|
<span class="hll"> Time zone: Europe/Riga (EET, +0200)
|
|
</span>System clock synchronized: yes
|
|
NTP service: active
|
|
RTC in local TZ: no
|
|
</code></pre></div>
|
|
|
|
<h2>Viewing Cron Logs Using <code>journalctl</code></h2>
|
|
<p>On a systemd-based system, you can use the <code>journalctl</code> utility to see system logs,
|
|
including logs from the cron daemon.</p>
|
|
<p>To see live logs:</p>
|
|
<div class="highlight"><pre><span></span><code>journalctl -f
|
|
</code></pre></div>
|
|
|
|
<p>To see the logs from e.g. the last hour, and only from the cron daemon:</p>
|
|
<div class="highlight"><pre><span></span><code>journalctl --since <span class="s2">"1 hour ago"</span> -t CRON
|
|
</code></pre></div>
|