Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Infer `get`-resources in jobs that were defined as `passed` #1593
What challenge are you facing?
There is a common use case where something needs to be "published", "finalized", "tagged" or similar, after several jobs have run and proven that a change didn't break anything.
Similar to the Flightschool, I'm using the following example to build, deploy, integration-test, and publish
jobs: - name: build-rc plan: - get: my-product-develop trigger: true - put: my-product-rc - name: deploy-rc plan: - get: my-product-rc trigger: true - get: ci - put: deployment - name: run-integration-tests plan: - get: my-product-rc passed: [ deploy-rc ] trigger: true - name: publish-release plan: - get: my-product-rc passed: [ run-integration-tests ] trigger: true - get: my-product-develop passed: [run-integration-tests] - put: my-product-master
This is example does not work, because the lines
- get: my-product-develop passed: [run-integration-tests]
All of this can of course be solved by adding the necessary resources to the jobs. While this is a bit annoying, it seems to me there are 2 deeper issues:
A Modest Proposal
In the same way that I manually add these resources to the jobs in a mechanical way, maybe Concourse could infer that automatically. To help it with that, I could add all the jobs in question to the
- name: publish-release plan: - get: my-product-rc passed: [ run-integration-tests ] trigger: true - get: my-product-develop passed: - build-rc - deploy-rc - run-integration-tests - put: my-product-master
That information should be enough for an algorithm to add the resources automatically. Of course this requires that the first job in this "sub-pipeline" (here
No, my example is not valid. It fails with:
As written in the challenges I'm facing, I'm well aware how I can resolve that error. But it's violating some principles that Concourse seemed to sell to me :-). Those I have described in 1. and 2.
In summary, the naive question that I would ask myself after fixing the error above when looking at the resulting yaml is: "Why does the job
It consumes it to maintain the version state. Though it may not do anything with it, there are artifacts that can be generated that you want to follow with that version of a resource.
I don't see a problem with the explicit declaration. It makes it easier to follow in the pipeline YAML. Having some implicitly happen would be difficult to follow when reviewing the YAML.
Concourse sells explicitness over anything. This proposal breaks that. If you believe other principles are being violated please be clear what those are.
@jtarchie Well, it looks like the discussion is over, but I'd still leave some more thoughts behind on this.
Am I really interested in an Integration Test job that tests a release candidate (which has the concept of a version itself) what git commit that rc was built off? It seems not important for the integration test job.
It's a modest proposal, not a perfect one. Would be very interested in better ones if we could have this conversation.
I talked with others about this behavior of concourse and it seems other people stumble across this too. So I feel like it is worth exploring better ideas here.