Do Something and Record It
Python HTML Makefile
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
doc
dsari
.gitignore
HISTORY.md
LICENSE
MANIFEST.in
Makefile
README
README.md
dsari-daemon
dsari-info
dsari-prometheus-exporter
dsari-render
setup.py

README.md

dsari - Do Something and Record It

dsari is a lightweight continuous integration (CI) system. It provides scheduling, concurrency management and trigger capabilities, and is easy to configure. Job scheduling is handled via dsari-daemon, while dsari-render may be used to format job run information as HTML.

Requirements

dsari requires Python 3.4 or later, and will run on Unix-based platforms. It may use the following non-core packages:

  • croniter, for parsing cron-style schedule definitions
  • python-dateutil, for parsing iCalendar RRULE-style schedule definitions
  • Jinja2, for rendering HTML reports
  • IPython, for better dsari-info shell interaction
  • termcolor, for colorized dsari-info TTY output
  • psycopg2, for PostgreSQL database support
  • mysqlclient (mysqldb), for MySQL database support
  • pymongo, for MongoDB database support

All non-core packages are optional, with the following limitations:

  • If neither croniter nor python-dateutil are installed, dsari-daemon will run, but it will not process scheduled runs (i.e. manual triggers only).
  • Jinja2 is only required if you intend to use dsari-render.
  • psycopg2, mysqlclient or pymongo are only required if you intend to use dsari with an alternative database. By default, dsari uses a SQLite 3 database.

Installation

dsari may be installed as any normal Python package:

$ sudo python3 setup.py install

When this is done, dsari will expect its configuration file -- dsari.json -- in /usr/local/etc/dsari/, and will store its data in /usr/local/lib/dsari/.

These locations may be customized by passing -c argument to dsari-* to specify the configuration directory, and the data_dir configuration option, respectively.

When dsari is installed directly in /usr/ (i.e. as part of distribution packaging), the default configuration and data directories will be /etc/dsari/ and /var/lib/dsari/, respectively.

dsari does not need to be installed at all, it can be run directly from the repository directory. In this case, the default configuration and data directories will be ~/.dsari/etc/ and ~/.dsari/var/, respectively.

The rest of these documents assume a locally-running setup, i.e. ~/.dsari/.

Configuration

A basic configuration for dsari.json looks as follows:

{
    "jobs": {
        "sample-job": {
            "command": ["/usr/bin/env"],
            "schedule": "H/5 * * * *"
        }
    }
}

This defines a job named "sample-job", which is run every 5 minutes. Many more configuration options are available in the doc/ directory.

Running

Once dsari is configured, run dsari-daemon. By default, dsari-daemon will run in the foreground, and can be used with a supervisor (upstart, systemd, supervisord, etc). If given -d, it will daemonize.

When a job is scheduled to be run, it produces a "run". Runs are identified by a UUID, the run output is stored in ~/.dsari/var/runs/, and data related to the run (start time, stop time, exit code, etc) is stored in a SQLite database at ~/.dsari/var/dsari.sqlite3.

When a run is executed, several environment variables are passed to the program to be run:

JOB_NAME=sample-job
RUN_ID=fa0490b8-7a8e-4f6b-b73c-160199a9ff75
PREVIOUS_RUN_ID=e5bd61b3-27f3-46ca-8169-372433056fc2
PREVIOUS_SCHEDULE_TIME=1437004689.27
PREVIOUS_START_TIME=1437004689.65
PREVIOUS_STOP_TIME=1437004689.71
PREVIOUS_EXIT_CODE=0

PREVIOUS_* variables are not set if there is no previous run. In addition, several extra environment variables are present, if the job's jenkins_environment option is set, to aid with migrations from Jenkins setups:

BUILD_NUMBER=fa0490b8-7a8e-4f6b-b73c-160199a9ff75
BUILD_ID=fa0490b8-7a8e-4f6b-b73c-160199a9ff75
BUILD_URL=file:///home/user/.dsari/var/runs/sample-job/fa0490b8-7a8e-4f6b-b73c-160199a9ff75/
NODE_NAME=master
BUILD_TAG=dsari-sample-job-fa0490b8-7a8e-4f6b-b73c-160199a9ff75
JENKINS_URL=file:///home/user/.dsari/var/
EXECUTOR_NUMBER=0
WORKSPACE=/home/user/.dsari/var/runs/sample-job/fa0490b8-7a8e-4f6b-b73c-160199a9ff75

Reports

To render HTML reports, run dsari-render occasionally. This will produce a series of HTML files in ~/.dsari/var/html/. You may then serve these files, rsync them to a remote server, etc.

The dsari-info command may be used to retrieve information about jobs and runs.

The dsari-prometheus-exporter command may be used to start a metrics daemon suitable for ingestion into Prometheus.

License

dsari - Do Something and Record It

Copyright (C) 2015-2016 Ryan Finnie

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.