You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

104 lines
5.0 KiB

4 years ago
4 years ago
4 years ago
  1. <h1>Monitoring Cron Jobs</h1>
  2. <p>SITE_NAME is perfectly suited for monitoring cron jobs. All you have to do is
  3. update your cron job command to send an HTTP request to SITE_NAME
  4. after completing the job.</p>
  5. <p>Let's look at an example:</p>
  6. <div class="highlight"><pre><span></span><code>$ crontab -l
  7. <span class="c1"># m h dom mon dow command</span>
  8. <span class="m">8</span> <span class="m">6</span> * * * /home/user/backup.sh
  9. </code></pre></div>
  10. <p>The above job runs <code>/home/user/backup.sh</code> every day at 6:08. The backup
  11. script is presumably a headless, background process. Even if it works
  12. correctly currently, it can start silently failing in the future without
  13. anyone noticing.</p>
  14. <p>You can set up SITE_NAME to notify you whenever the backup script does not
  15. run on time, or it does not complete successfully. Here are the steps to do that.</p>
  16. <ol>
  17. <li>
  18. <p>If you have not already, sign up for a free SITE_NAME account.</p>
  19. </li>
  20. <li>
  21. <p>In your SITE_NAME account, <strong>add a new check</strong>.</p>
  22. </li>
  23. <li>
  24. <p>Give the check <strong>a meaningful name</strong>. Good naming will become
  25. increasingly important as you add more checks to your account.</p>
  26. </li>
  27. <li>
  28. <p>Edit the check's <strong>schedule</strong>:</p>
  29. <ul>
  30. <li>change its type from "Simple" to "Cron"</li>
  31. <li>enter <code>8 6 * * *</code> in the cron expression field</li>
  32. <li>set the timezone to match your machine's timezone</li>
  33. </ul>
  34. </li>
  35. <li>
  36. <p>Take note of your check's unique <strong>ping URL</strong>.</p>
  37. </li>
  38. </ol>
  39. <p>Finally, edit your cron job definition and append a curl or wget call
  40. after the command:</p>
  41. <div class="highlight"><pre><span></span><code>$ crontab -e
  42. <span class="c1"># m h dom mon dow command</span>
  43. <span class="m">8</span> <span class="m">6</span> * * * /home/user/backup.sh <span class="o">&amp;&amp;</span> curl -fsS --retry <span class="m">5</span> -o /dev/null PING_URL
  44. </code></pre></div>
  45. <p>Now, each time your cron job runs, it will send an HTTP request to the ping URL.
  46. Since SITE_NAME knows your cron job's schedule, it can calculate
  47. the dates and times when the job should run. As soon as your cron job doesn't
  48. report at an expected time, SITE_NAME will send you a notification.</p>
  49. <p>This monitoring technique takes care of various failure scenarios that could
  50. potentially go unnoticed otherwise:</p>
  51. <ul>
  52. <li>The whole machine goes down (power outage, janitor stumbles on wires, VPS provider problems, etc.)</li>
  53. <li>the cron daemon is not running or has an invalid configuration</li>
  54. <li>cron does start your task, but the task exits with a non-zero exit code</li>
  55. </ul>
  56. <h2>Curl Options</h2>
  57. <p>The extra options in the above example tell curl to retry failed HTTP requests, and
  58. silence output unless there is an error. Feel free to adjust the curl options to
  59. suit your needs.</p>
  60. <dl>
  61. <dt><strong>&amp;&amp;</strong></dt>
  62. <dd>Run curl only if <code>/home/user/backup.sh</code> exits with an exit code 0.</dd>
  63. <dt><strong>-f, --fail</strong></dt>
  64. <dd>Makes curl treat non-200 responses as errors.</dd>
  65. <dt><strong>-s, --silent</strong></dt>
  66. <dd>Silent or quiet mode. Hides the progress meter, but also hides error messages.</dd>
  67. <dt><strong>-S, --show-error</strong></dt>
  68. <dd>Re-enables error messages when -s is used.</dd>
  69. <dt><strong>--retry &lt;num&gt;</strong></dt>
  70. <dd>If a transient error is returned when curl tries to perform a
  71. transfer, it will retry this number of times before giving up.
  72. Setting the number to 0 makes curl do no retries (which is the default).
  73. Transient error is a timeout or an HTTP 5xx response code.</dd>
  74. <dt><strong>-o /dev/null</strong></dt>
  75. <dd>Redirect curl's stdout to /dev/null (error messages still go to stderr).</dd>
  76. </dl>
  77. <h2>Looking up Your Machine's Time Zone</h2>
  78. <p>If your cron job consistently pings SITE_NAME an hour early or an hour late,
  79. the likely cause is a timezone mismatch: your machine may be using a timezone
  80. different from what you have configured on SITE_NAME.</p>
  81. <p>On modern GNU/Linux systems, you can look up the time zone using the
  82. <code>timedatectl status</code> command and looking for "Time zone" in its output:</p>
  83. <div class="highlight"><pre><span></span><code>$ timedatectl status
  84. Local time: C  2020-01-23 12:35:50 EET
  85. Universal time: C  2020-01-23 10:35:50 UTC
  86. RTC time: C  2020-01-23 10:35:50
  87. <span class="hll"> Time zone: Europe/Riga (EET, +0200)
  88. </span>System clock synchronized: yes
  89. NTP service: active
  90. RTC in local TZ: no
  91. </code></pre></div>
  92. <h2>Viewing Cron Logs Using <code>journalctl</code></h2>
  93. <p>On a systemd-based system, you can use the <code>journalctl</code> utility to see system logs,
  94. including logs from the cron daemon.</p>
  95. <p>To see live logs:</p>
  96. <div class="highlight"><pre><span></span><code>journalctl -f
  97. </code></pre></div>
  98. <p>To see the logs from e.g. the last hour, and only from the cron daemon:</p>
  99. <div class="highlight"><pre><span></span><code>journalctl --since <span class="s2">&quot;1 hour ago&quot;</span> -t CRON
  100. </code></pre></div>