Skip to content
Branch: master
Find file History
ci-automation
ci-automation refresh pdf
Latest commit 7ed4cc2 Aug 12, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
split-pipeline-to-pieces concourse trigger task regularly Nov 3, 2018
README.org update doc Jul 28, 2019
cheatsheet-concourse-A4.pdf refresh pdf Aug 12, 2019
concourse-design.png update image Jan 21, 2019
concourse-sample.png add concourse image Oct 2, 2018
example-git.yml rename concourse example Aug 22, 2018
github-release-specific-tag.md
helloworld.yml concourse render variables Nov 3, 2018
render-for-render.md concourse trigger task regularly Nov 3, 2018
run-test-regularly.yml concourse trigger task regularly Nov 3, 2018
tasks-output-input.yml misc change Nov 4, 2018
url-download.yml concourse render variables Nov 3, 2018

README.org

1 CheatSheet: Concourse

linkedin
github
slack


PRs Welcome

File me Issues or star this repo.

1.1 Concourse Concepts

NameSummary
flyPipelines are configured entirely via fly CLI
TeamsEvery pipeline and one-off build belongs to a team.
TargetsAlias of Concourse endpoints, which composes of server url, credentials, etc
PipelinesA pipeline is the result of configuring Jobs and Resources together.
JobsJobs determine the actions of your pipeline,
ResourceAllow you to define inputs and outputs to jobs in a pipeline
TasksThe smallest configurable unit in a Concourse pipeline is a single task.
ReferenceLink: Concourse Concepts, Link: concoursetutorial.com

1.2 Concourse Resources

NameSummary
file-downloader-resourceenable downloading pivotal product files and stemcells
gcs-resource: google bucket
pool-resourceAtomically manages external environments
github-release
git-releasetracks commits in a branch of a Git repository
docker-image-resource: docker image
Step: tagsUse tag to match workers
bbl (bosh-bootloader)Command line utility for standing up a BOSH director on any IAAS
ReferenceGitHub: pcf-pipelines

1.3 Concourse Examples

NameSummary
Hello world pipelineexample: helloworld.yml
Pass output to input across different tasksexample: tasks-output-input.yml
Trigger pipeline test every 5 minutesexample: run-test-regularly.yml
git-resourceexample: example-git.yml
concourse-curl-resource: http download linkexample: url-download.yml

1.4 Concourse Containers

NameSummary
Login to containerfly -t $target hijack -u $job_url
Run command without sudden exitnohup SOMECOMMAND &; tail -f nohup.out
Hijack to containers with a successful runBy design when task has passed, the container will be deleted

1.5 Concourse Pipeline

NameSummary
List all authenticated end pointsfly targets
Get fly versionfly --version
fly loginfly -t $target login -c $concourse_url -n $team_name
Get pipeline definitionfly -t $target get-pipeline -p $job_name > job.yaml
Update pipelinefly -t $target set-pipeline -p $pipeline_name -c pipeline.yml -l credentials.yml
Update pipelinefly -t $target set-pipeline -p $pipeline_name -c pipeline.yml -v env1=var1
Destroy pipelinefly -t $target destroy-pipeline -p $pipeline_name
Destroy pipeline without confirmationfly -t $target destroy-pipeline -p $pipeline_name -n
Format pipelinefly format-pipeline -c pipeline.yml
Validate pipelinefly validate-pipeline -c pipeline.yml

https://raw.githubusercontent.com/dennyzhang/cheatsheet.dennyzhang.com/master/cheatsheet-concourse-A4/concourse-sample.png

1.6 Concourse Resource

NameSummary
github-release resource get the old tagfly -t $target check-resource -r $pipeline-name/$resource-name -f tag:$tag
bosh-io-stemcell resource get the old versionfly -t $target check-resource -r $pipeline-name/ubuntu-xenial-stemcell -f version:170.25
github release use a specific taggithub-release-specific-tag.md

1.7 Concourse Security

NameSummary
Fly login to the default main teamfly -t $target login -n main -c $concourse_url
Fly login to a different teamfly -t $target login -n $team_name -c $concourse_url
List all teamsfly -t $target teams, fly -t $target teams -d
Fly logout all targetsfly logout --all
Install ansible vault in macbrew install ansible
Create a teamfly -t $target set-team -n $team_name --github-org $my_org
Create a teamfly -t $target set-team -n $team_name --allow-all-users
Destroy a teamfly -t $target destroy-team --team-name $team_name
Login to a teamfly -t $target set-team -n $team_name
Fly conf file~/.flyrc
ReferenceAfter login to the main team, you can update other teams. Link: Concourse Team

1.8 Concourse Workers

NameSummary
List all concourse workersfly -t $target workers
List all concourse containersfly -t $target containers

https://raw.githubusercontent.com/dennyzhang/cheatsheet.dennyzhang.com/master/cheatsheet-concourse-A4/concourse-design.png

1.9 Concourse Opportunities

NameSummary
fly set-pipeline run into “error: forbidden”Need to logout from both fly client (fly logout --all) and Concourse GUI
List team details for all targetsfly -t $target teams -d, ~/.flyrc
Concourse UI runs into a blank pageIf pipeline has recursive dependencies, Concourse UI may have issue to run its javascript code
Split one pipeline yaml into small reusable piecesNot supported. An uncomfortable workaround: generating yaml with other tools. Link: GitHub
Add a readonly member to one teamNot supported. Concourse only support authentication control, but not authorization control
Does Concourse have REST API?Not supported so far. Link: GitHub, Link: Concourse REST endpoints
Referenceconcourse-tutorial, concourse-pipeline-samples

1.10 fly Online Usage

bash-3.2$ fly --help
error: Usage:
  fly [OPTIONS] <command>

Application Options:
  -t, --target=              Concourse target name
  -v, --version              Print the version of Fly and exit
      --verbose              Print API requests and responses
      --print-table-headers  Print table headers even for redirected output

Help Options:
  -h, --help                 Show this help message

Available commands:
  abort-build        Abort a build (aliases: ab)
  builds             List builds data (aliases: bs)
  check-resource     Check a resource (aliases: cr)
  checklist          Print a Checkfile of the given pipeline (aliases: cl)
  containers         Print the active containers (aliases: cs)
  destroy-pipeline   Destroy a pipeline (aliases: dp)
  destroy-team       Destroy a team and delete all of its data (aliases: dt)
  execute            Execute a one-off build using local bits (aliases: e)
  expose-pipeline    Make a pipeline publicly viewable (aliases: ep)
  format-pipeline    Format a pipeline config (aliases: fp)
  get-pipeline       Get a pipeline's current configuration (aliases: gp)
  help               Print this help message
  hide-pipeline      Hide a pipeline from the public (aliases: hp)
  hijack             Execute a command in a container (aliases: intercept, i)
  jobs               List the jobs in the pipelines (aliases: js)
  login              Authenticate with the target (aliases: l)
  logout             Release authentication with the target (aliases: o)
  order-pipelines    Orders pipelines (aliases: op)
  pause-job          Pause a job (aliases: pj)
  pause-pipeline     Pause a pipeline (aliases: pp)
  pause-resource     Pause a resource (aliases: pr)
  pipelines          List the configured pipelines (aliases: ps)
  prune-worker       Prune a stalled, landing, landed, or retiring worker (aliases: pw)
  rename-pipeline    Rename a pipeline (aliases: rp)
  rename-team        Rename a team (aliases: rt)
  set-pipeline       Create or update a pipeline's configuration (aliases: sp)
  set-team           Create or modify a team to have the given credentials (aliases: st)
  status             Login status
  sync               Download and replace the current fly from the target (aliases: s)
  targets            List saved targets (aliases: ts)
  teams              List the configured teams (aliases: t)
  trigger-job        Start a job in a pipeline (aliases: tj)
  unpause-job        Unpause a job (aliases: uj)
  unpause-pipeline   Un-pause a pipeline (aliases: up)
  unpause-resource   Unpause a resource (aliases: ur)
  validate-pipeline  Validate a pipeline config (aliases: vp)
  volumes            List the active volumes (aliases: vs)
  watch              Stream a build's output (aliases: w)
  workers            List the registered workers (aliases: ws)

1.11 fly set-pipeline Online Usage

> fly  set-pipeline --help
error: Usage:
  fly [OPTIONS] set-pipeline [set-pipeline-OPTIONS]

Application Options:
  -t, --target=                     Concourse target name
  -v, --version                     Print the version of Fly and exit
      --verbose                     Print API requests and responses
      --print-table-headers         Print table headers even for redirected output

Help Options:
  -h, --help                        Show this help message

[set-pipeline command options]
      -n, --non-interactive         Skips interactions, uses default values
          --no-color                Disable color output
          --check-creds             Validate credential variables against credential manager
      -p, --pipeline=               Pipeline to configure
      -c, --config=                 Pipeline configuration file
      -v, --var=[NAME=STRING]       Specify a string value to set for a variable in the pipeline
      -y, --yaml-var=[NAME=YAML]    Specify a YAML value to set for a variable in the pipeline
      -l, --load-vars-from=         Variable flag that can be used for filling in template values in configuration from a YAML file

1.12 More Resources

http://www.mikeball.info/blog/concourse-git-resource/

https://concoursetutorial.com/

License: Code is licensed under MIT License.

linkedin github slack
You can’t perform that action at this time.