healthchecks

healthchecks is a watchdog for your cron jobs. It's a web server that listens for pings from your cron jobs, plus a web interface.

It is live here: http://healthchecks.io/

The building blocks are:

• Python 2 or Python 3
• Django 1.9
• PostgreSQL or MySQL

Docker quick start

Quick start:

1. Get Docker

2. Start full stack using docker-compose:

curl -Lo docker-compose.yaml https://raw.githubusercontent.com/honestbee/healthchecks/master/docker-compose.yml
docker-compose -p healthchecks up -d
# monitor boot process of healthchecks app
docker-compose -p healthchecks logs -f hc

# once ready:
# open healthchecks on localhost
open http://localhost:9090

# open mailhog for testing
open http://localhost:8025

To deploy the application, set environment variables

Admin Access

By default no admin user is created. To create your first admin user connect to your running docker container:

docker-compose -p healthchecks exec hc bash

and run:

./manage.py createsuperuser

You will now be able to login to the admin at http://localhost:9090/admin.

N.B. There is no validation so ensure you correctly set an email address and password or you may find yourself unable to login.

Configuration

The Docker images are built to take basic configuration from environment variables:

Variable	Default
HEALTHCHECKS_DEBUG	False
HEALTHCHECKS_HOST	localhost
HEALTHCHECKS_SITE_ROOT	"http://localhost:9090"
HEALTHCHECKS_DB	mysql
HEALTHCHECKS_DB_HOST	localhost
HEALTHCHECKS_DB_USER	root
HEALTHCHECKS_DB_PASSWORD	pa55word
HEALTHCHECKS_EMAIL_FROM	"[email protected]"
HEALTHCHECKS_EMAIL_HOST	localhost
HEALTHCHECKS_EMAIL_PORT	25
HEALTHCHECKS_EMAIL_USER	""
HEALTHCHECKS_EMAIL_PASSWORD	""
MYSQL_ROOT_PASSWORD	pa55word
MYSQL_USER	hc
MYSQL_PASSWORD	pa55word
MYSQL_DATABASE	hc

Docker Compose can take environment varialbes from env_files or overwritten using the -f docker-compose.production.yaml flag (refer to docker-compose documentation)

Review the configuration with:

docker-compose config

Building

The following command will build the application with the image tag specified in the docker-compose.yaml file.

docker-compose build

Setting Up for Local Development

These are instructions for setting up HealthChecks Django app in development environment.

• prepare directory for project code and virtualenv:

    $ mkdir -p ~/webapps
    $ cd ~/webapps
  

• prepare virtual environment (with virtualenv you get pip, we'll use it soon to install requirements):

    $ virtualenv --python=python3 hc-venv
    $ source hc-venv/bin/activate
  

• check out project code:

    $ git clone https://github.com/healthchecks/healthchecks.git
  

• install requirements (Django, ...) into virtualenv:

    $ pip install -r healthchecks/requirements.txt
  

• healthchecks is configured to use a SQLite database by default. To use PostgreSQL or MySQL database, create and edit hc/local_settings.py file. There is a template you can copy and edit as needed:

    $ cd ~/webapps/healthchecks
    $ cp hc/local_settings.py.example hc/local_settings.py
  

• create database tables and the superuser account:

    $ cd ~/webapps/healthchecks
    $ ./manage.py migrate
    $ ./manage.py createsuperuser
  

• run development server:

    $ ./manage.py runserver
  

The site should now be running at http://localhost:8080 To log into Django administration site as a super user, visit http://localhost:8080/admin

Configuration

Site configuration is kept in hc/settings.py. Additional configuration is loaded from hc/local_settings.py file, if it exists. You can create this file (should be right next to settings.py in the filesystem) and override settings as needed.

Some useful settings keys to override are:

SITE_ROOT is used to build fully qualified URLs for pings, and for use in emails and notifications. Example:

SITE_ROOT = "https://my-monitoring-project.com"

SITE_NAME has the default value of "healthchecks.io" and is used throughout the templates. Replace it with your own name to personalize your installation. Example:

SITE_NAME = "My Monitoring Project"

REGISTRATION_OPEN controls whether site visitors can create new accounts. Set it to False if you are setting up a private healthchecks instance, but it needs to be publicly accessible (so, for example, your cloud services can send pings).

If you close new user registration, you can still selectively invite users to your team account.

Database Configuration

Database configuration is stored in hc/settings.py and can be overriden in hc/local_settings.py. The default database engine is SQLite. To use PostgreSQL, create hc/local_settings.py if it does not exist, and put the following in it, changing it as neccessary:

DATABASES = {
    'default': {
        'ENGINE':   'django.db.backends.postgresql',
        'NAME':     'your-database-name-here',
        'USER':     'your-database-user-here',
        'PASSWORD': 'your-database-password-here',
        'TEST': {'CHARSET': 'UTF8'}
    }
}

For MySQL:

DATABASES = {
    'default': {
        'ENGINE':   'django.db.backends.mysql',
        'NAME':     'your-database-name-here',
        'USER':     'your-database-user-here',
        'PASSWORD': 'your-database-password-here',
        'TEST': {'CHARSET': 'UTF8'}
    }
}

You can also use hc/local_settings.py to read database configuration from environment variables like so:

import os

DATABASES = {
    'default': {
        'ENGINE':   os.environ['DB_ENGINE'],
        'NAME':     os.environ['DB_NAME'],
        'USER':     os.environ['DB_USER'],
        'PASSWORD': os.environ['DB_PASSWORD'],
        'TEST': {'CHARSET': 'UTF8'}
    }
}

Sending Emails

healthchecks must be able to send email messages, so it can send out login links and alerts to users. Put your SMTP server configuration in hc/local_settings.py like so:

EMAIL_HOST = "your-smtp-server-here.com"
EMAIL_PORT = 587
EMAIL_HOST_USER = "username"
EMAIL_HOST_PASSWORD = "password"
EMAIL_USE_TLS = True

For more information, have a look at Django documentation, Sending Email section.

Sending Status Notifications

healtchecks comes with a sendalerts management command, which continuously polls database for any checks changing state, and sends out notifications as needed. Within an activated virtualenv, you can manually run the sendalerts command like so:

$ ./manage.py sendalerts

In a production setup, you will want to run this command from a process manager like supervisor or systemd.

Database Cleanup

With time and use the healthchecks database will grow in size. You may decide to prune old data: inactive user accounts, old checks not assigned to users, records of outgoing email messages and records of received pings. There are separate Django management commands for each task:

• Remove old records from api_ping table. For each check, keep 100 most recent pings:

  $ ./manage.py prunepings
  

• Remove checks older than 2 hours that are not assigned to users. Such checks are by-products of random visitors and robots loading the welcome page and never setting up an account:

  $ ./manage.py prunechecks
  

• Remove records of sent email messages older than 7 days.

  $ ./manage.py pruneemails
  

• Remove old records of sent notifications. For each check, remove notifications that are older than the oldest stored ping for same check.

  $ ./manage.py prunenotifications
  

• Remove user accounts that match either of these conditions:

• Account was created more than 6 months ago, and user has never logged in. These can happen when user enters invalid email address when signing up.

• Last login was more than 6 months ago, and the account has no checks. Assume the user doesn't intend to use the account any more and would probably want it removed.

  $ ./manage.py pruneusers
  

When you first try these commands on your data, it is a good idea to test them on a copy of your database, not on the live database right away. In a production setup, you should also have regular, automated database backups set up.

Integrations

Pushover

To enable Pushover integration, you will need to:

• register a new application on https://pushover.net/apps/build
• enable subscriptions in your application and make sure to enable the URL subscription type
• add the application token and subscription URL to hc/local_settings.py, as PUSHOVER_API_TOKEN and PUSHOVER_SUBSCRIPTION_URL