where-to-contribute.md

layout	title	category	menu	toc
main	Where to Contribute	About	menu	title url Where to Contribute #where-to-contribute title url Screwdriver API #screwdriver-api title url subitem Models #models true title url subitem Datastores #datastores true title url subitem Source Code Management (SCM) #source-code-management true title url subitem Notifications #notifications true title url subitem Parsers #parsers true title url subitem Templates and Commands #templates-and-commands true title url Launcher #launcher title url Queue #queue title url subitem Executors #executors true title url Artifacts #artifacts title url UI #ui title url Guide and Homepage #guide-and-homepage title url Miscellaneous Tools #miscellaneous-tools title url subitem Bootstrap SD #bootstrap-sd true title url subitem Other #other true title url Adding a New Screwdriver Repo #adding-a-new-screwdriver-repo title url Tests and Examples #tests-and-examples

Where to Contribute

Screwdriver has a modular architecture, and the various responsibilities are split up into separate repos.

Check out the architecture diagram to see the overall workflow of Continuous Delivery using Screwdriver. The next few sections will help lay out where different code repositories fit.

Screwdriver API

The screwdriver repo is the core of screwdriver, providing the API endpoints for everything that screwdriver does. The API is based on the hapijs framework and is implemented in node as a series of plugins.

• Build bookends allows a user to create setup and teardown steps for builds.

• The API can also uploading code coverage reports and/or test results. coverage-bookend defines the relationship between Screwdriver and coverage bookends. coverage-base is the base class for defining the behavior between Screwdriver coverage bookend plugins, like coverage-sonar.

Models

The object models provide the definition of the data that is saved in datastores (analogous to databases). This is done in two parts:

• data-schema: Schema definition with Joi
• models: Specific business logic around the data schema

Datastores

A datastore implementation is used as the interface between the API and a data storage mechanism. There are several implementations written in node around a common interface.

• datastore-base: Base class defining the interface for datastore implementations
• datastore-sequelize: MySQL, PostgreSQL, SQLite3, and MS SQL implementations

Source Code Management

An SCM implementation is used as the interface between the API and an SCM. There are several implementations written in nodejs around a common interface.

• scm-base: Common interface
• scm-bitbucket: Bitbucket.org implementation
• scm-github: GitHub implementation
• scm-gitlab: GitLab implementation
• sd-repo: A Go-based tool for executing the Repo workflow for getCheckoutCommand in scm-github

The scm-router is a generic scm plugin that routes builds to a specified scm.

Notifications

The API can also send notifications to users.

• notifications-base: The base class for defining the behavior between Screwdriver and notifications plugins
• notifications-email: Email implementation
• notifications-slack: Slack implementation

Parsers

Parsers help validate and parse various flows in Screwdriver.

• config-parser: A node module for validating and parsing user's screwdriver.yaml configurations
• workflow-parser: A node module for parsing and converting pipeline configuration into a workflow graph

Templates and Commands

Templates are snippets of predefined code that people can use to replace a job definition in their screwdriver.yaml. A template contains a series of predefined steps along with a selected Docker image.

• templates: A repo for all build templates
• template-main: The CLI for validating and publishing job templates
• template-validator: A tool used by the API to validate a job template
• tmpl-semantic-release: A template that performs a semantic-release for NPM-based modules

Commands are snippets of predefined code that people can use to replace a step definition in their screwdriver.yaml. A command contains a series of predefined commands.

• command-validator: A tool used by the API to validate a command
• cmd-install-node: Shared command to install node.js using nvm
• cmd-docker-trigger: Shared command to trigger Docker build for master and a specified tag
• junit-reports: Shared command to parse Junit reports

Launcher

The launcher performs step execution and housekeeping internal to build containers. This is written in Go and mounted into build containers as a binary.

• sd-cmd: A Go-based CLI for sharing binaries which provides a single interface for executing a versioned command (via remote binary, docker image, or habitat package) during a Screwdriver build
• sd-packages: Builds, packages, and publishes skopeo and zstd and other custom binaries for Launcher
• sd-step: A Shared Step allows people to use the same packages and commands in all build containers, regardless of build environment
• meta-cli: A Go-based CLI for reading/writing information from the metadata

Queue

The queue-service is a highly available REST based queue service for Screwdriver to enqueue builds and process them. It makes use of Resque to add a queueing mechanism.

• buildcluster-queue-worker: An amqp connection manager implementation that consumes jobs from Rabbitmq queue

Executors

An executor is used to manage build containers for any given job. Several implementations of executors have been created. All are designed to follow a common interface. Executor implementations are written in node:

• executor-base: Common interface
• executor-docker: Docker implementation
• executor-j5s: Jenkins implementation
• executor-k8s: Kubernetes implementation
• executor-nomad: Nomad implementation

The executor-queue is a generic executor plugin that routes builds through a Redis queue. The executor-router is a generic executor plugin that routes builds to a specified executor.

Artifacts

The Artifact Store (not to be confused with the datastores mentioned above) is used for saving log outputs, shared steps, templates, test coverage, and any artifacts that are generated during a build.

• artifact-bookend: Bookend for uploading artifacts to the store
• cache-bookend: Bookend for uploading and downloading build cache
• log service: A Go tool for reading logs from the Launcher and uploading them to the store
• store-cli: A Go-based CLI for communicating with the Screwdriver store

UI

The UI is an Ember-based user interface of Screwdriver.

Guide and Homepage

The Guide is documentation! Everything you ever hoped to know about the Screwdriver project. The Homepage is the basic one-pager that powers Screwdriver.cd. The Community is where open source community meeting notes and docs are kept.

Miscellaneous Tools

Bootstrap SD

These repos help you get started running Screwdriver.

• aws-build-cluster: Quickstart tool to provision necessary EKS cluster and cloud infrastructure resources required by Screwdriver build cluster on AWS
• hyperctl-image: Creates a minimal Docker image containing hyperctl and k8s-vm scripts (used in screwdriver-chart)
• in-a-box: Python-based executable for bringing up an entire Screwdriver instance (UI, API, and log store) locally
• screwdriver-chart: Bootstraps the whole Screwdriver ecosystem and also nginx ingress controller
• sd-local: A Go-based tool for running a more feature-complete local Screwdriver
• sonar-pipeline: Pipeline to deploy SonarQube server to Kubernetes

Other

• circuit-fuses: Wrapper to provide a node-circuitbreaker with callback interface
• gitversion: Go-based tool for updating git tags on a repo for a new version number
• keymbinatorial: Generates the unique combinations of key values by taking a single value from each keys array
• logger: Common logging provider for Screwdriver components
• noop-container: Bare minimum Docker container for running builds
• raptor: Load test scripts for Screwdriver API
• sd-housekeeping: Repo containing House Keeping scripts like a bulk pipeline validator
• toolbox: Repository for handy Screwdriver-related scripts and other tools

Adding a New Screwdriver Repo

We have some tools to help start out new repos for Screwdriver:

• generator-screwdriver: Yeoman generator that bootstraps new repos for screwdriver
• eslint-config-screwdriver: Our ESLint rules for node-based code. Included in each new repo as part of the bootstrap process

If you create a new repo, please come back and edit this page so that others can know where your repo fits in.

Tests and Examples

The organization screwdriver-cd-test contains various example repos/screwdriver.yamls and acceptance tests for Screwdriver.cd.