Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Feature: Combined/aggregated triggers #2324
What challenge are you facing?
We currently need a way of aggregating a trigger, for example:
A Modest Proposal
Currently, when using multiple
In the example above, the INTEGRATION_TEST will be triggered whenever package_A
My proposal is to come up with a way of (optionally) changing the default
@vito currently we're limited to: if a single condition is met, the job will be triggered.
As I see it, the limitation is not related to the
One example for this would be an application that requires multiple modules/libraries/whatever to be compiled/tested separately, but still, at some point in the pipeline, if they all pass a series of jobs, they can be put together, tested and packaged.
With this "if a single condition is met, the job will be triggered" logic in place, this "final packaging job" (as described above) will be triggered for each time a single module/library/whatever hits the trigger, generating a bad package.
@romerojunior But don't you need something correlating the packages so that you know they're "for" each other in the first place? Or do you want it to only run when there are new versions of all 3 resources? (...But what if you only need to make a change to one?)
I'd recommend taking a look at the "Fanning In" example here, if you haven't already: https://concourse-ci.org/get-step.html - it shows how
As a real-world example, the Concourse project puts all of our components in one repo as submodules. This way each commit of the toplevel repo is the "source of truth" for the versions that are expected to be compatible with each other. We push a bump commit, which then triggers individual unit tests, each with the same resource as their input. We then "fan in" on that same resource (
You can see the rendered pipeline here: https://ci.concourse-ci.org/teams/main/pipelines/main - and the config here: https://github.com/concourse/pipelines/blob/ae3d57a596adb063b2066d1c3b096b172d32a86c/ci.concourse-ci.org/concourse.yml#L57-L184 (I've highlighted a subset that's relevant, but there are other examples of this workflow in the same config.)
Without having a single resource that you can correlate the jobs and resources to, you would need something like #604 if you want to only trigger on them all changing. But then I'm still not sure how you'd handle the (I'd expect common) case of only changing one of the packages.
Just stumbled across this, we solved our needs for complex fan-in logic using https://github.com/Meshcloud/gate-resource. It doesn't exactly make concourse itself understand these fan-ins on a resource-level but we use the metadata published in gates to do fan-in in task code (i.e. download specific artifact revisions).
This is certainly not ideal and you should rely as much as possible on concourse's preferred way to do things. However, for our use case of building & testing a mono-repo of many services efficiently and collecting the builds for deployment concourse's available resource-flows didn't cut it, but gate-resource has been working great.
To handle such cases I passed original resource (source code ?) across entire pipeline to form a single stream. It doesn't do the job ?
--- resources: - name: source-code - name: package-common - name: package-a - name: package-b jobs: - name: build-common plan: - get: source-code - task: build-common - put: package-common - name: build-a plan: - aggregate: - get: source-code passed: [build-common] - get: package-common passed: [build-common] - task: build-a - put: package-a - name: build-b plan: - aggregate: - get: source-code passed: [build-common] - get: package-common passed: [build-common] - task: build-b - put: package-b - name: check-integration plan: - aggregate: - get: source-code passed: [build-a, build-b] - get: package-a passed: [build-a] - get: package-b passed: [build-b] - task: run-integration-test
This issue has been automatically closed because there has been no response to our request for more information from the original author.