No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cron
docker
docs
helm/docker-flow-cron
server
.gitignore
.travis.yml
Dockerfile
Dockerfile.docs
Dockerfile.test
Jenkinsfile
LICENSE.md
README.md
docker-compose-test.yml
docker-compose.example.yml
main.go
mkdocs.yml
proxy-key.enc
stack-docs.yml
stack.yml

README.md

Docker Flow Cron

THIS PROJECT IS IN DESIGN PHASE, DO NOT USE IT IN PRODUCTION YET!

Docker Swarm services are designed to the long lasting processes that, potentially, live forever. Docker does not have a mechanism to schedule jobs based on time interval or, to put it in other words, it does not have the ability to use cron-like syntax for time-based scheduling.

Docker Flow Cron is designed to overcome some of the limitations behind Docker Swarm services and provide cron-like time-based scheduling while maintaining fault tolerance features available in Docker Swarm.

Please visit the project documentation for more info or join the #df-cron Slack channel in DevOps20 if you have any questions, suggestions, or problems.

Usage

There is currently two different ways of using Docker Flow Cron

Docker Flow Cron API

Put Job

Adds a job to docker-flow-cron

The following body parameters can be used to send a create job PUT request to Docker Flow Cron. They should be added to the base address [CRON_IP]:[CRON_PORT]/v1/docker-flow-cron/job/[jobName].

param Description Mandatory Example
image Docker image. yes alpine
serviceName Docker service name no my-cronjob
command The command that will be executed when a job is created. no echo "hello World"
schedule The schedule that defines the frequency of the job execution.Check the scheduling section for more info. yes @every 15s
args The list of arguments that can be used with the docker service create command.

--restart-condition cannot be set to any. If not specified, it will be set to none.
--name argument is not allowed. Use serviceName param instead

Any other argument supported by docker service create is allowed.
no TODO

TODO: Example

Get All Jobs

Gets all scheduled jobs

The following GET request [CRON_IP]:[CRON_PORT]/v1/docker-flow-cron/job. can be used to get all scheduled jobs from Docker Flow Cron.

Get Job

Gets a job from docker-flow-cron

The following GET request [CRON_IP]:[CRON_PORT]/v1/docker-flow-cron/[jobName]. can be used to get a job from Docker Flow Cron.

Delete Job

Deletes a job from docker-flow-cron

The following DELETE request [CRON_IP]:[CRON_PORT]/v1/docker-flow-cron/[jobName]. can be used to delete a job from Docker Flow Cron.

Docker Flow Swarm Listener support

Using the Docker Flow Swarm Listener support, Docker Services can schedule jobs. Docker Flow Swarm Listener listens to Docker Swarm events and sends requests to Docker Flow Cron when changes occurs, every time a service is created or deleted Docker Flow Cron gets notified and manages job scheduling.

A Docker Service is created with the following syntax:

docker service create [OPTIONS] IMAGE [COMMAND] [ARG...]

Check the documentation for more information.

The same syntax is used to schedule a job in Docker Flow Cron:

  • IMAGE specifies the Docker Image
  • [COMMAND] specifies the command to schedule.
  • [OPTIONS] specifies some necessary options making scheduling Docker Services possible.

The following Docker Service args [OPTIONS] should be used for scheduled Docker Services:

arg Description Mandatory Example
--replicas Set 0 to prevent the service from running immedietely. Set to 1 to run command on service creation. yes 0 or 1
--restart-condition Set to none to prevent the service from using Docker Swarm's ability to autorestart exited services. yes none

The following Docker Service labels [OPTIONS] needs to be used for scheduled Docker Services:

label Description Prefix Mandatory Example
cron Enable scheduling com.df yes true
image Docker image. com.df.cron yes alpine
name Cronjob name. com.df.cron yes my-cronjob
schedule The schedule that defines the frequency of the job execution. Check the scheduling section for more info. com.df.cron yes @every 15s
command The command that is scheduled, only used for Docker Flow Cron registration. Use the same command you set for your docker service to run. com.df.cron No echo Hello World

All labels needs to be prefixed

Examples:

  • --labels "com.df.cron=true"
  • --labels "com.df.cron.name=my-job"

Tutorial

Docker Flow Cron using Docker Stacks

Create the overlay network

docker network create -d overlay cron

Create the cron stack

docker-compose.yml

version: "3"

services:
  cron:
    image: vfarcic/docker-flow-cron
    networks:
      - cron
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - 8080:8080
    deploy:
      placement:
        constraints: [node.role == manager]
  
  swarm-listener:
    image: vfarcic/docker-flow-swarm-listener
    networks:
      - cron
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DF_NOTIFY_CREATE_SERVICE_URL=http://cron:8080/v1/docker-flow-cron/job/create
      - DF_NOTIFY_REMOVE_SERVICE_URL=http://cron:8080/v1/docker-flow-cron/job/remove
    deploy:
      placement:
        constraints: [node.role == manager]
networks:
  cron:
    external: true

Deploy the stack above by executing docker stack deploy -c docker-compose.yml cron

Docker Flow Cron using Docker Services

Create Docker Flow Swarm Listener

docker service create --name swarm-listener \
    --network cron \
    --mount "type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock" \
    -e DF_NOTIFY_CREATE_SERVICE_URL=http://cron:8080/v1/docker-flow-cron/create \
    -e DF_NOTIFY_REMOVE_SERVICE_URL=http://cron:8080/v1/docker-flow-cron/remove \
    --constraint 'node.role==manager' \
    vfarcic/docker-flow-swarm-listener

Create Docker Flow Cron

docker service create --name cron \
    --network cron \
    --mount "type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock" \
    -p 8080:8080 \
    --constraint 'node.role==manager' \
    vfarcic/docker-flow-cron

Example: Scheduling a Docker Service

Now that the cron stack is up and running we can add our first scheduled Docker Service. The example service below will run echo "Hello World" every 10 seconds using an Alpine image.

Example: Scheduled job using Docker Services

docker service create --name cronjob \
    --network cron --replicas 0 \
    --restart-condition=none \
    -l "com.df.notify=true" \
    -l "com.df.cron=true" \
    -l "com.df.cron.name=cronjob" \
    -l "com.df.cron.image=alpine" \
    -l "com.df.cron.command=echo Hello World" \
    -l "com.df.cron.schedule=@every 10s" \
    alpine \
    echo Hello world

The example below is a more realistic use case for Docker Flow Cron. It will remove unusued docker data by running docker system prune every day.

Example: Scheduling a docker command to run on the host

docker service create --name docker_cleanup \
    --network cron --replicas 0 \
    --restart-condition=none \
    --mount "type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock" \
    -l "com.df.notify=true" \
    -l "com.df.cron=true" \
    -l "com.df.cron.name=docker_cleanup" \
    -l "com.df.cron.image=docker" \
    -l "com.df.cron.command=docker system prune -f" \
    -l "com.df.cron.schedule=@daily" \
    docker \
    docker system prune -f

Scheduling

Docker Flow Cron uses the library robfig/cron to provide a simple cron syntax for scheduling.

Examples

0 30 * * * *        Every hour on the half hour
@hourly             Every hour
@every 1h30m        Every hour thirty

Predefined schedules

You may use one of several pre-defined schedules in place of a cron expression.

Entry                  | Description                                | Equivalent To
-----                  | -----------                                | -------------
@yearly (or @annually) | Run once a year, midnight, Jan. 1st        | 0 0 0 1 1 *
@monthly               | Run once a month, midnight, first of month | 0 0 0 1 * *
@weekly                | Run once a week, midnight on Sunday        | 0 0 0 * * 0
@daily (or @midnight)  | Run once a day, midnight                   | 0 0 0 * * *
@hourly                | Run once an hour, beginning of hour        | 0 0 * * * *

Intervals

You may also schedule a job to execute at fixed intervals

@every <duration>
@every 2h30m15s

Check the library documentation for more information.