Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
RFC: Simple pipeline merging and templating. #18
RFC - Simple pipeline merging and templating.
@vito suggested we raise this as an RFC some time before Christmas (on Discord). Now that we've had time to reflect on it, here is our proposal. Looking at the other RFC's, it may be complementary to #7
We love Concourse and use it extensively, but as your pipelines grow they become increasingly more complicated. The cognitive load becomes overwhelming at times, and changes become increasingly more difficult to identify and track down. There is a general need for a way of splitting out parts of a pipeline, and also allowing for some form of templating to accommodate a simpler cognitive mode.
Some of the issues we see with the current handling of pipelines:
For the purposes of our proposal, #1 is out of scope.
To tackle the following:
We propose the following:
The running example is extremely simple, but it's composability and the power of golang templates makes for an extremely powerful and compelling tool.
We originally proposed UAV as a separate tool that did part of this job; pipeline merging with some variable substitution. We have since re-written UAV as a stand-alone app in golang, with all the above functionality, and would propose that the vision outlined here be adopted by Concourse, specifically into
We see the following benefits and problems:
One thing that is not currently covered by
UAV can be found @ https://github.com/finbourne/uav ; the golang version is in the
Hey, haven't dug into this yet but would you mind re-submitting as a PR following the process described here?: https://github.com/concourse/rfcs/blob/master/README.md
It looks pretty thorough already so it'd be good to get the process going properly so it's easier to leave feedback. The other issues are requests-for-RFCs hence they don't have any proposals themselves.
Did you have a look at https://yaml.org/type/merge.html or https://github.com/cloudfoundry-incubator/spiff/blob/master/README.md (don't remember the follow up) before?
I hadn't looked at either, but I also don't deploy bosh. From what I can tell, the templating isn't as powerful, or feature rich as go templates, so not sure it would have covered off my use case. I use