Skip to content
Pinball is a scalable workflow manager
JavaScript Python CSS HTML Thrift Java
Branch: master
Clone or download
Acehaidrey Merge pull request #108 from Oedipus/master
Fix Pinball system image links
Latest commit b8638a0 Mar 18, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
pinball Merge pull request #89 from SinghAsDev/master Jan 31, 2019
pinball_ext
tests also include test Sep 26, 2017
tutorial Replace typographer's quotes with normal ones to avoid Python SyntaxE… Apr 2, 2015
.gitignore update gitignore to exclude folder config files Feb 1, 2016
.travis.yml
ADMIN_GUIDE.rst fixed documentation typo Mar 16, 2015
ARCHITECTURE.rst Fix Pinball system image links Mar 11, 2019
AUTHORS.rst Update AUTHORS.rst Mar 23, 2015
CONTRIBUTING.rst Fixed links Mar 12, 2015
HISTORY.rst pydot2 -> pydot as pydot fixed incompatibility with pyparsing > 1.5.7… Mar 30, 2017
LICENSE Initial checkin Mar 12, 2015
MANIFEST.in
Makefile Initial checkin Mar 12, 2015
README.rst
USER_GUIDE.rst Add documentation regarding USER_GUIDE Jan 31, 2019
instance_view.png Added screenshot Mar 12, 2015
job_states.png Initial checkin Mar 12, 2015
pinball_system.png Initial checkin Mar 12, 2015
requirements.txt Merge pull request #105 from Acehaidrey/BDP-5748 Jan 31, 2019
setup.py
tox.ini Initial checkin Mar 12, 2015
workflow.png

README.rst

Pinball

Pinball is a scalable workflow manager.

Design Principles

  • Simple: based on easy to grasp abstractions
  • Extensible: component-based approach
  • Transparent: state stored in a readable format
  • Reliable: stateless computing components
  • Scalable: scales horizontally
  • Admin-friendly: can be upgraded without aborting workflows
  • Feature-rich: auto-retries, per-job-emails, runtime alternations, priorities, overrun policies, etc.

Installation

If you haven't already installed libmysqlclient-dev, graphviz. Please install them, e.g.,

$ sudo apt-get install libmysqlclient-dev
$ sudo apt-get install graphviz

If you want to install Pinball through pypi package, please do

$ sudo pip install pinball

Pinball uses mysql as persistent storage. Please also make sure mysql is available, and properly configured.

Quick Start

Start Pinball

Once Pinball is installed either through pypi package installation or source code clone, we are ready to run it. There are four important components in Pinball.

  • Master: A frontend to a persistent state repository with an interface supporting atomic job token updates. To start master,

    $ python -m pinball.run_pinball -c path/to/pinball/yaml/configuration/file -m master
    
  • UI: A service reading directly from the storage layer used by the Master. To start UI,

    $ python -m pinball.run_pinball -c path/to/pinball/yaml/configuration/file -m ui
    
  • Scheduler: Scheduler is responsible for running workflows on a schedule. To start scheduler,

    $ python -m pinball.run_pinball -c path/to/pinball/yaml/configuration/file -m scheduler
    
  • Worker: A client of the Master. To start worker,

    $ python -m pinball.run_pinball -c path/to/pinball/yaml/configuration/file -m workers
    

Configure Pinball

In order to start Pinball, user needs to provide a pinball configuration file. A sample pinball configuraiton can be retrived at here.

There are a few parameters to configure. For example:

  • MySQL db configuration

    databases:
         default:
             ENGINE:       django.db.backends.mysql
             NAME:         pinball
             USER:         flipper
             PASSWORD:     flipper123
             HOST:         127.0.0.1
             PORT:         "3306"
         pinball.persistence:
             ENGINE:       django.db.backends.mysql
             NAME:         pinball
             USER:         flipper
             PASSWORD:     flipper123
             HOST:         127.0.0.1
             PORT:         "3306"
    
  • Pinball UI configuration

    ui_host:                  pinball
    ui_port:                  8080
    
  • Application Configuration

    parser:                    pinball_ext.workflows.parser.PyWorkflowParser
    

    parser tells Pinball how to interpret your defined workflow and jobs. The above configuration links to a Python parser provided by Pinball. You can also provide your own parser to intepretate your own definition of workflow and jobs. Please check the tutorial for details.

    parser_params:
      workflows_config:       pinball_ext.examples.workflows.WORKFLOWS
      job_repo_dir:           "~"
      job_import_dirs_config: pinball_ext.examples.jobs.JOB_IMPORT_DIRS
    

    parser_params will be taken by parser. Name of the variable that stores workflows config is workflows_config; root dir of the repo that stores all user defined jobs is stored at job_repo_dir; job_import_dirs_config keeps list of dirs where job class should be imported from.

  • Email configuration

    default_email:              your@email.com
    

    default_email configures default sender of email service of Pinball.

Use Pinball

After starting Pinball with the proper configuration, user can access Pinball at pinball:8080. You may find there is no workflow or jobs listed in Pinball UI when you first start Pinball. To import your workflow into Pinball, do the following command.

python -m pinball.tools.workflow_util -c path/to/pinball/yaml/configuration/file -f reschedule

After this, you should be able to see your workflows in Pinball UI. They will be scheduled and run according to the specified schedules.

Workflow Instance View

Detailed Design

Design details are available in Pinball Architecture Overview

User Guide

Detail user guide is available in Pinball User Guide

Admin Guide

Administrator guide is available in Pinball Administrator Guide

License

Pinball is distributed under Apache License, Version 2.0.

Maintainers

Contributing

Help

If you have any questions or comments, you can reach us at pinball-users@googlegroups.com.

You can’t perform that action at this time.