Skip to content

Commit

Permalink
Reduce the check interval for all resources
Browse files Browse the repository at this point in the history
We are having load issues on our build CI, as all the resources
are checking for new versions across all the pipelines.

But we only want to check for those resources that trigger pipelines,
as the rest of the resources are checked as soon as each job is
triggered.

Additionally, we do not really need the default resolution of 1m
for the git resources in these pipelines. 10m is just fine and
enough, and if needed, the developer can trigger it manually.

To reduce the load we will:

 - Increase `check_every` to a big period for all non triggering
   resources. We pick `24h`, to effectivelly disable the checks
   on these resources.
   This include the docker resources from the resource_types

 - Increase `check_every` to `10m` for all the triggering resources.

More info:

[1] concourse/concourse#1842
[2] concourse/concourse#1445
  • Loading branch information
keymon committed Dec 5, 2018
1 parent e846d33 commit 15217ef
Show file tree
Hide file tree
Showing 9 changed files with 37 additions and 0 deletions.
7 changes: 7 additions & 0 deletions pipelines/build-release.yml
Expand Up @@ -12,24 +12,28 @@ groups:
resource_types:
- name: pull-request
type: docker-image
check_every: 24h
source:
repository: jtarchie/pr

- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

- name: semver-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/semver-resource
tag: ecbdd201e122b44de99a40ac9f24407c1a43b9a2

resources:
- name: bosh-release-pr
type: pull-request
check_every: 10m
source:
repo: ((github_repo))
access_token: ((github_access_token))
Expand All @@ -38,19 +42,22 @@ resources:

- name: bosh-release-repo
type: git
check_every: 10m
source:
uri: ((github_repo_uri))
branch: ((final_release_branch))

- name: bosh-release-tarballs
type: s3-iam
check_every: 24h
source:
bucket: ((releases_bucket_name))
region_name: ((aws_region))
regexp: ([a-z0-9]+).tgz

- name: bosh-release-version
type: semver-iam
check_every: 24h
source:
bucket: ((releases_bucket_name))
region_name: ((aws_region))
Expand Down
5 changes: 5 additions & 0 deletions pipelines/destroy.yml
Expand Up @@ -2,32 +2,37 @@
resource_types:
- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

- name: semver-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/semver-resource
tag: ecbdd201e122b44de99a40ac9f24407c1a43b9a2

resources:
- name: pipeline-trigger
type: semver-iam
check_every: 24h
source:
bucket: ((state_bucket_name))
region_name: ((aws_region))
key: ((pipeline_trigger_file))

- name: paas-release-ci
type: git
check_every: 10m
source:
uri: https://github.com/alphagov/paas-release-ci
branch: ((branch_name))

- name: release-ci-tfstate
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket_name))
region_name: ((aws_region))
Expand Down
6 changes: 6 additions & 0 deletions pipelines/integration-test.yml
Expand Up @@ -2,18 +2,21 @@
resource_types:
- name: pull-request
type: docker-image
check_every: 24h
source:
repository: jtarchie/pr

- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

resources:
- name: pr
type: pull-request
check_every: 10m
source:
repo: ((github_repo))
access_token: ((github_access_token))
Expand All @@ -22,6 +25,7 @@ resources:

- name: release-repository
type: git
check_every: 10m
source:
branch: ((tag_branch))
ignore_paths:
Expand All @@ -31,6 +35,7 @@ resources:

- name: resource-version
type: semver
check_every: 24h
source:
branch: ((tag_branch))
driver: git
Expand All @@ -42,6 +47,7 @@ resources:

- name: secrets
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket))
region_name: ((aws_region))
Expand Down
3 changes: 3 additions & 0 deletions pipelines/plain_pipelines/paas-hackmd.yml
Expand Up @@ -2,19 +2,22 @@
resource_types:
- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

resources:
- name: paas-codimd
type: git
check_every: 10m
source:
uri: https://github.com/alphagov/paas-codimd
branch: gds_master

- name: hackmd-secrets
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket))
region_name: ((aws_region))
Expand Down
3 changes: 3 additions & 0 deletions pipelines/plain_pipelines/paas-product-page.yml
Expand Up @@ -2,19 +2,22 @@
resource_types:
- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

resources:
- name: paas-product-page
type: git
check_every: 10m
source:
uri: https://github.com/alphagov/paas-product-page.git
branch: master

- name: zendesk-secrets
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket))
region_name: ((aws_region))
Expand Down
3 changes: 3 additions & 0 deletions pipelines/plain_pipelines/paas-team-manual.yml
Expand Up @@ -3,19 +3,22 @@
resource_types:
- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

resources:
- name: paas-team-manual
type: git
check_every: 10m
source:
uri: https://github.com/alphagov/paas-team-manual.git
branch: master

- name: ssh-private-key
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket))
versioned_file: ci_build_tag_key
Expand Down
2 changes: 2 additions & 0 deletions pipelines/plain_pipelines/paas-tech-docs.yml
Expand Up @@ -3,13 +3,15 @@
resource_types:
- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

resources:
- name: paas-tech-docs
type: git
check_every: 10m
source:
uri: https://github.com/alphagov/paas-tech-docs.git
branch: master
Expand Down
3 changes: 3 additions & 0 deletions pipelines/plain_pipelines/rubbernecker.yml
Expand Up @@ -2,19 +2,22 @@
resource_types:
- name: s3-iam
type: docker-image
check_every: 24h
source:
repository: governmentpaas/s3-resource
tag: fda60bf4c5f85e96c16f704e128e5ead9e84d30d

resources:
- name: rubbernecker
type: git
check_every: 10m
source:
uri: https://github.com/alphagov/paas-rubbernecker.git
branch: master

- name: rubbernecker-secrets
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket))
region_name: ((aws_region))
Expand Down
5 changes: 5 additions & 0 deletions pipelines/setup.yml
Expand Up @@ -15,19 +15,22 @@ resource_types:
resources:
- name: pipeline-trigger
type: semver-iam
check_every: 24h
source:
bucket: ((state_bucket_name))
region_name: ((aws_region))
key: ((pipeline_trigger_file))

- name: paas-release-ci
type: git
check_every: 10m
source:
uri: https://github.com/alphagov/paas-release-ci
branch: ((branch_name))

- name: release-ci-tfstate
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket_name))
region_name: ((aws_region))
Expand All @@ -50,6 +53,7 @@ resources:
- name: ssh-private-key
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket_name))
versioned_file: ci_build_tag_key
Expand All @@ -58,6 +62,7 @@ resources:

- name: ssh-public-key
type: s3-iam
check_every: 24h
source:
bucket: ((state_bucket_name))
versioned_file: ci_build_tag_key.pub
Expand Down

0 comments on commit 15217ef

Please sign in to comment.