Queues jobs to be run by the various workers
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
config Update example.yml [ci skip] Feb 15, 2017
lib/travis
script
spec
var/librato scheduler 2.0 com librato space Oct 27, 2016
.gitignore
.librato.yml
.rspec
.rubocop.yml
.rubocop_todo.yml the big little commit Mar 26, 2015
.ruby-version
.simplecov
.travis.yml
CONTRIBUTING.md
Gemfile
Gemfile.lock Bump rack version to pick up security patches Nov 30, 2018
MIT-LICENSE.md add build status badge Oct 9, 2015
Procfile pgbouncer buildpack for more effective connection pooling May 30, 2018
README.md Minor update to README Jan 8, 2019
Rakefile fix specs after rebase Mar 30, 2017
runbook.md Fixing minor typos Dec 4, 2018

README.md

Travis Scheduler Build Status

Keeper of the limits

Scheduler is the application that, in the life-cycle of accepting, evaluating, and executing a build request, sits in the third position.

The first two positions are: Listener accepts a "request" (an incoming event from GitHub). Gatekeeper evaluates, and configures (fetches .travis.yml) the request, and creates a build record, including at least one job per build in the database.

Now, these jobs won't start automatically. They just sit in the database, and wait until they can be "queued" for the workers, so workers can pick them up, and start executing them. So, "queueing" a job, in this context, means: sending the job to the workers.

The app responsible for queueing jobs is travis-scheduler. The current implementation of the scheduler looks at all jobs that are in the created state (i.e. they're "waiting to be queued"), groups them by their owner (organization, user) and evaluates how many of them can be queued. Once done evaluating jobs for all owners it starts over.

E.g. on travis-ci.org the default concurrency limit is 5, so the scheduler will make sure there aren't more than 5 jobs running at a time. If it finds 3 jobs already running for a given owner, and 4 jobs waiting to be queued, it will only queue 2 more jobs in order to respect the limit of 5.

The log output for each of these evaluation rounds looks like this:

user sven public capacity: total=3 running=1 selected=2
user sven boost capacity: total=2 running=0 selected=2
repo sven/repo: queueable=5 running=1 selected=4 waiting=1
user sven: queueable=5 running=1 selected=4 total_waiting=1 waiting_for_concurrency=1

The terminology used here can be confusing. The terms mean:

  • total - number of concurrent jobs, provided by public capacity, plan, boost, etc.
  • running - number of jobs currently running, i.e. in the state queued, received, or started
  • queueable - number of jobs in the state created
  • selected - number of queueable jobs that are being selected to be queued based on concurrency limits
  • total_waiting - total number of queueable jobs that have not been selected to be queued
  • waiting_for_concurrency - number of queueable jobs that have not been selected, and have not been found to be limited by repo settings, queue, or stages

In the example log output above, the owner has a capacity of 3 concurrent jobs provided by public capacity (line 1), and 2 jobs provided by boost capacity (line 2). The job selection finds 5 jobs to be queueable (i.e. in the state created), and 1 job to be running. As the total capacity is 5 jobs it can select 4 jobs to be queued for the workers, leaving 1 job waiting.

Contributing

See CONTRIBUTING for information on how to contribute to travis-scheduler.

License & copyright

See MIT-LICENSE.