initial template and parameterization proposal #18215

Merged
merged 1 commit into from Feb 9, 2016

Conversation

Projects
None yet
@bparees
Contributor

bparees commented Dec 4, 2015

@bgrant0607 @brendandburns @smarterclayton @derekwaynecarr

This is a proposal to define a syntax for authoring templates that represent k8s object topologies, while allowing those templates to be customized prior to instantiation via a parameterization mechanism.

Related discussions/template proposals:
#14918
#14993
#4210
#10408 (comment)
#11492

@googlebot googlebot added the cla: yes label Dec 4, 2015

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Dec 4, 2015

Collaborator

Labelling this PR as size/L

Collaborator

k8s-merge-robot commented Dec 4, 2015

Labelling this PR as size/L

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Dec 4, 2015

GCE e2e test build/test passed for commit 5da37eb.

k8s-bot commented Dec 4, 2015

GCE e2e test build/test passed for commit 5da37eb.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Dec 5, 2015

Member

Thanks. WTAL

Member

bgrant0607 commented Dec 5, 2015

Thanks. WTAL

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Dec 5, 2015

Member

cc @kubernetes/sig-config

Member

bgrant0607 commented Dec 5, 2015

cc @kubernetes/sig-config

docs/proposals/templates.md
+### Use cases for templates in general
+
+* Providing a full baked application experience in a single portable object that can be repeatably deployed in different environments.
+ * eg Wordpress deployment with separate standalone database pod

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Nit: a standalone pod is an anti-pattern

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Nit: a standalone pod is an anti-pattern

This comment has been minimized.

@bparees

bparees Dec 6, 2015

Contributor

you mean a pod w/o a replication controller? yeah sorry i was a bit sloppy with my language here. I was only trying to distinguish from "single pod containing both a web app an db container", not imply there wasn't also a replication controller managing the DB pod. i'll clarify in the next revision.

@bparees

bparees Dec 6, 2015

Contributor

you mean a pod w/o a replication controller? yeah sorry i was a bit sloppy with my language here. I was only trying to distinguish from "single pod containing both a web app an db container", not imply there wasn't also a replication controller managing the DB pod. i'll clarify in the next revision.

+
+### Use cases for templates in general
+
+* Providing a full baked application experience in a single portable object that can be repeatably deployed in different environments.

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

In general, bulk creation, as with List.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

In general, bulk creation, as with List.

This comment has been minimized.

@bparees

bparees Dec 18, 2015

Contributor

added bulk object creation as another bullet.

@bparees

bparees Dec 18, 2015

Contributor

added bulk object creation as another bullet.

This comment has been minimized.

@jackgr

jackgr Jan 13, 2016

Contributor

@bparees +1 because bulk creation is quite different from application specification, although the 2 use cases share some common requirements, such as parameterization and iterative expansion.

@jackgr

jackgr Jan 13, 2016

Contributor

@bparees +1 because bulk creation is quite different from application specification, although the 2 use cases share some common requirements, such as parameterization and iterative expansion.

docs/proposals/templates.md
+
+* Explicitly constructing a new template that merges the existing templates (tooling can easily be constructed to perform this
+ operation since the templates are first class api objects).
+* Manually instantiating each template and utilizing service linking (as described in the service catalog proposal) to share

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

For others, service catalog is #17543

@bgrant0607

bgrant0607 Dec 5, 2015

Member

For others, service catalog is #17543

docs/proposals/templates.md
+In addition, the template can be repeatedly instantiated for a consistent application deployment experience in different
+namespaces or Kubernetes clusters.
+
+Lastly, we propose the Template API object include a “Labels” section in which the template author can define a set of labels

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Use cases are similar to #17097 and #15390, just with a smaller scope.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Use cases are similar to #17097 and #15390, just with a smaller scope.

+ Parameters []Parameter
+
+ // Required: A list of resources to create
+ Objects []runtime.Object

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

FWIW, List uses Items I think.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

FWIW, List uses Items I think.

This comment has been minimized.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

These are objects specifically so that the objects aren't confused with lists. Objects that have nested "items" are considered lists and decomposed by our client processing. For a template, invoking create -f is more about creating an instance of the template on the server. But creating a List or PodList is not. Other ways to solve that but keeping lists semantically distinct has been helpful so far.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

These are objects specifically so that the objects aren't confused with lists. Objects that have nested "items" are considered lists and decomposed by our client processing. For a template, invoking create -f is more about creating an instance of the template on the server. But creating a List or PodList is not. Other ways to solve that but keeping lists semantically distinct has been helpful so far.

This comment has been minimized.

docs/proposals/templates.md
+// the Template to Config transformation.
+type Parameter struct {
+ // Required: Parameter name must be set and it can be referenced in Template
+ // Items using ${PARAMETER_NAME}

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Do you find it problematic that curly braces break yaml parsing? What do you think about using the same syntax as env var expansion $()? Since both the parameters and the env vars would be explicitly declared we could detect ambiguity.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Do you find it problematic that curly braces break yaml parsing? What do you think about using the same syntax as env var expansion $()? Since both the parameters and the env vars would be explicitly declared we could detect ambiguity.

This comment has been minimized.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

We picked curly before we chose $( for env, so no real preference. It has not been a challenge for yaml, authors figure it out easily enough

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

We picked curly before we chose $( for env, so no real preference. It has not been a challenge for yaml, authors figure it out easily enough

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Ok. I'd prefer to settle on $() as the universal substitution syntax.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Ok. I'd prefer to settle on $() as the universal substitution syntax.

This comment has been minimized.

@bparees

bparees Dec 6, 2015

Contributor

fine with changing to $(). the current solution doesn't break yaml parsing because of the requirement/restriction that we only substitute string fields, which means the ${} value is always quoted.

@bparees

bparees Dec 6, 2015

Contributor

fine with changing to $(). the current solution doesn't break yaml parsing because of the requirement/restriction that we only substitute string fields, which means the ${} value is always quoted.

docs/proposals/templates.md
+
+1. Optionally set an explicit `value` for any parameter values the user wishes to explicitly set (eg non-generated parameters for
+ which the default value is not desired)
+2. Optionally set a generator input (`from`) value to override the default input to a generator for a parameter (eg password

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

How common is that?

@bgrant0607

bgrant0607 Dec 5, 2015

Member

How common is that?

This comment has been minimized.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Pretty common for passwords or fields that have to be unique but benefit when a user picks them (like the name of the user). Presenting a default or generated value and being able to override that feels natural to users - forcing them to mutate the template in order to accomplish the same thing is irritating.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Pretty common for passwords or fields that have to be unique but benefit when a user picks them (like the name of the user). Presenting a default or generated value and being able to override that feels natural to users - forcing them to mutate the template in order to accomplish the same thing is irritating.

docs/proposals/templates.md
+
+Given a well-formed template, a client will
+
+1. Optionally set an explicit `value` for any parameter values the user wishes to explicitly set (eg non-generated parameters for

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

I'd like to be able to specify value overrides in a separate map that is passed to the processing endpoint, so that users wouldn't even necessarily have to look at a template in order to use it. They just need a simple parameter form to fill out.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

I'd like to be able to specify value overrides in a separate map that is passed to the processing endpoint, so that users wouldn't even necessarily have to look at a template in order to use it. They just need a simple parameter form to fill out.

This comment has been minimized.

@bparees

bparees Dec 6, 2015

Contributor

this could handled via a separate endpoint which took a new object consisting of a parameter map and a template name/template reference.

it would retrieve the template, set the parameter values, and submit to the existing endpoint for substitution.

(or the existing endpoint could be extended to take this new object type and do the above work).

thoughts?

@bparees

bparees Dec 6, 2015

Contributor

this could handled via a separate endpoint which took a new object consisting of a parameter map and a template name/template reference.

it would retrieve the template, set the parameter values, and submit to the existing endpoint for substitution.

(or the existing endpoint could be extended to take this new object type and do the above work).

thoughts?

This comment has been minimized.

@bgrant0607

bgrant0607 Jan 13, 2016

Member

One possibility would be to leverage my proposal for PetSet: #18016 (comment)

If we thought users might not want a persistent object capturing their parameter values (though that usually seems desirable to me), I'd leverage my existence-dependency proposal (#1535 (comment)) to make the ParameterizedSet get deleted automatically when the resources generated by it were deleted.

Otherwise, yes, a separate API endpoint would work. The workflow would be:

  • GET the template
  • substitute the values in the parameter list
  • perform template substitution
  • create the fully expanded resources
@bgrant0607

bgrant0607 Jan 13, 2016

Member

One possibility would be to leverage my proposal for PetSet: #18016 (comment)

If we thought users might not want a persistent object capturing their parameter values (though that usually seems desirable to me), I'd leverage my existence-dependency proposal (#1535 (comment)) to make the ParameterizedSet get deleted automatically when the resources generated by it were deleted.

Otherwise, yes, a separate API endpoint would work. The workflow would be:

  • GET the template
  • substitute the values in the parameter list
  • perform template substitution
  • create the fully expanded resources
+
+*It should be possible to validate templates and template parameters, both values and the schema.*
+
+* Template objects are subject to standard api validation.

This comment has been minimized.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Except when the value does not validate - such as replacement in names ($( is not valid)

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Except when the value does not validate - such as replacement in names ($( is not valid)

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

We should be able to validate the template field names.

As for values, if we had jsonschema-like validation specifications for parameters and/or for the resource fields, we could validate the values to some degree.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

We should be able to validate the template field names.

As for values, if we had jsonschema-like validation specifications for parameters and/or for the resource fields, we could validate the values to some degree.

This comment has been minimized.

@bparees

bparees Dec 6, 2015

Contributor

Except when the value does not validate - such as replacement in names ($( is not valid)

we don't validate the objects within the template definition, they are just runtime.Object, right? i'm just saying the template api object is validated as such, not that the things it contains are validated as whatever type they are. that does not happen (and as you note, really can't happen until after substitution happens. This situation will be even worse once we add support for substitution into integer fields)

@bparees

bparees Dec 6, 2015

Contributor

Except when the value does not validate - such as replacement in names ($( is not valid)

we don't validate the objects within the template definition, they are just runtime.Object, right? i'm just saying the template api object is validated as such, not that the things it contains are validated as whatever type they are. that does not happen (and as you note, really can't happen until after substitution happens. This situation will be even worse once we add support for substitution into integer fields)

docs/proposals/templates.md
+* Customize cross-component references (eg user provides the name of a secret that already exists in their namespace, to use in
+ a pod as a TLS cert).
+* Provide guidance to users for parameters such as default values, descriptions, and whether or not a particular parameter value
+ is required or can be left blank.

This comment has been minimized.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Parameterize the replica count of a deployment or PetSet
Parameterize at least part of the labels and selector for a DaemonSet.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Parameterize the replica count of a deployment or PetSet
Parameterize at least part of the labels and selector for a DaemonSet.

This comment has been minimized.

@bparees

bparees Dec 6, 2015

Contributor

keeping in mind that parameterizing replica counts requires addressing the current "string only substitution" limitation. I was trying to restrict the use cases to ones solved by the proposal. if we agree on how we want to add generic type substitution to this proposal, we can certainly add other use cases that are solved with it.

@bparees

bparees Dec 6, 2015

Contributor

keeping in mind that parameterizing replica counts requires addressing the current "string only substitution" limitation. I was trying to restrict the use cases to ones solved by the proposal. if we agree on how we want to add generic type substitution to this proposal, we can certainly add other use cases that are solved with it.

This comment has been minimized.

@sparkprime

sparkprime Dec 16, 2015

It's definitely a valid use case and one that can be achieved with several of the options on the table so I think it should be included.

@sparkprime

sparkprime Dec 16, 2015

It's definitely a valid use case and one that can be achieved with several of the options on the table so I think it should be included.

This comment has been minimized.

@bparees

bparees Dec 18, 2015

Contributor

parameter types other than string are now supported by this proposal.

@bparees

bparees Dec 18, 2015

Contributor

parameter types other than string are now supported by this proposal.

docs/proposals/templates.md
+* Currently only string fields can be parameterized. See discussion [here](https://github.com/openshift/origin/issues/3946) for some proposed
+ solutions. (tldr: a syntax like ${FOO | int} could instruct the template processor to convert the json field to an integer when doing the
+ substitution. This could also support secret generation by using a syntax like `${FOO | base64}` which would take the value of parameter
+ `$FOO` and base64 encode it before setting the json field to the resulting value)

This comment has been minimized.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Definitely need a solution for this. Functor pipelines start to increase the complexity of the proposal and the line between template and language gets blurry.

@smarterclayton

smarterclayton Dec 5, 2015

Contributor

Definitely need a solution for this. Functor pipelines start to increase the complexity of the proposal and the line between template and language gets blurry.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Dec 5, 2015

Member

I think we need something like this.

Kubernetes is about automating operations:
https://github.com/kubernetes/kubernetes/blob/master/docs/whatisk8s.md

Templates could also be used for a new generation of controllers #14961, like PetSet #18016, replacing the original idea of just pod templates #170.

We also have at least half a dozen cases in flight where we need parameter substitution.

Member

bgrant0607 commented Dec 5, 2015

I think we need something like this.

Kubernetes is about automating operations:
https://github.com/kubernetes/kubernetes/blob/master/docs/whatisk8s.md

Templates could also be used for a new generation of controllers #14961, like PetSet #18016, replacing the original idea of just pod templates #170.

We also have at least half a dozen cases in flight where we need parameter substitution.

docs/proposals/templates.md
+ solutions. (tldr: a syntax like ${FOO | int} could instruct the template processor to convert the json field to an integer when doing the
+ substitution. This could also support secret generation by using a syntax like `${FOO | base64}` which would take the value of parameter
+ `$FOO` and base64 encode it before setting the json field to the resulting value)
+ * Another option would be to add type information to the parameter definitions, making it slightly easier to reference a parameter within a

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Can't we use the swagger to get the type?

@bgrant0607

bgrant0607 Dec 5, 2015

Member

Can't we use the swagger to get the type?

This comment has been minimized.

@bparees

bparees Dec 8, 2015

Contributor

only if template processing becomes schema aware which it currently is not. will add it as an option.

@bparees

bparees Dec 8, 2015

Contributor

only if template processing becomes schema aware which it currently is not. will add it as an option.

docs/proposals/templates.md
+*It shouldn't be inordinately difficult to evolve templates. Thus, strategies such as versioning and encapsulation should be
+encouraged, at least by convention.*
+
+* Templates are an api object and as such can be evolved and versioned along with the rest of the Kubernetes api.

This comment has been minimized.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

I meant that individual templates should be versioned. But that's possible with this proposal, by convention.

@bgrant0607

bgrant0607 Dec 5, 2015

Member

I meant that individual templates should be versioned. But that's possible with this proposal, by convention.

This comment has been minimized.

@bparees

bparees Dec 6, 2015

Contributor

ah. yes, annotations on the template object could be used for that purpose.

@bparees

bparees Dec 6, 2015

Contributor

ah. yes, annotations on the template object could be used for that purpose.

@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Dec 8, 2015

Contributor

@bgrant0607 @smarterclayton addressed some of your comments, in particular changed to the $() syntax and added making template processing schema aware to the list of ways to address typed parameter substitution.

I have not addressed @bgrant0607's request to support a map of parameter values + template for instantiation, i'd like to see a little more discussion on the use case for that given that we've so far not seen a need for it in openshift. blind submission of parameter key/value pairs for a template seems likely to lead to failure for a user.

Contributor

bparees commented Dec 8, 2015

@bgrant0607 @smarterclayton addressed some of your comments, in particular changed to the $() syntax and added making template processing schema aware to the list of ways to address typed parameter substitution.

I have not addressed @bgrant0607's request to support a map of parameter values + template for instantiation, i'd like to see a little more discussion on the use case for that given that we've so far not seen a need for it in openshift. blind submission of parameter key/value pairs for a template seems likely to lead to failure for a user.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Dec 8, 2015

GCE e2e test build/test passed for commit 2d0df64.

k8s-bot commented Dec 8, 2015

GCE e2e test build/test passed for commit 2d0df64.

docs/proposals/templates.md
+* Utilizes multiple files with different syntax to define an application topology.
+* Does not define a first class API object.
+* Does not appear to natively offer generation logic for producing parameter values.
+

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

The preceding statements about DM are inaccurate. Please read the DM documentation at https://github.com/kubernetes/deployment-manager, including the contents of the docs folder, and correct these statements.

@jackgr

jackgr Dec 8, 2015

Contributor

The preceding statements about DM are inaccurate. Please read the DM documentation at https://github.com/kubernetes/deployment-manager, including the contents of the docs folder, and correct these statements.

This comment has been minimized.

@bparees

bparees Dec 8, 2015

Contributor

Utilizes multiple files with different syntax to define an application topology.

this example uses 4 files including both yaml and jinja formats: https://github.com/kubernetes/deployment-manager/tree/master/examples/wordpress

the docs also indicate templates can be written in python. none of the examples/formats match k8s api objects/syntax.

Does not define a first class API object.

see above, what DM defines/consumes is not a first class k8s api object. Perhaps there's some confusion in what i meant by api object, I did not mean that DM doesn't define an api for itself, I meant it does not define api objects that are consumed by k8s. I can clarify this statement.

Does not appear to natively offer generation logic for producing parameter values.

I understand DM has parameters, i don't see native logic for defining how a parameter value should be generated (eg generate a random password). Presumably it's possible to make that happen via jinja or python, but I wouldn't consider that native support within DM, hence my statement. Again, I can clarify this to indicate value generation is deferred to jinja/python.

If I am still not understanding the design of DM, I'm afraid I'm going to need you to be more explicit about what is inaccurate/what I have misunderstood.

@bparees

bparees Dec 8, 2015

Contributor

Utilizes multiple files with different syntax to define an application topology.

this example uses 4 files including both yaml and jinja formats: https://github.com/kubernetes/deployment-manager/tree/master/examples/wordpress

the docs also indicate templates can be written in python. none of the examples/formats match k8s api objects/syntax.

Does not define a first class API object.

see above, what DM defines/consumes is not a first class k8s api object. Perhaps there's some confusion in what i meant by api object, I did not mean that DM doesn't define an api for itself, I meant it does not define api objects that are consumed by k8s. I can clarify this statement.

Does not appear to natively offer generation logic for producing parameter values.

I understand DM has parameters, i don't see native logic for defining how a parameter value should be generated (eg generate a random password). Presumably it's possible to make that happen via jinja or python, but I wouldn't consider that native support within DM, hence my statement. Again, I can clarify this to indicate value generation is deferred to jinja/python.

If I am still not understanding the design of DM, I'm afraid I'm going to need you to be more explicit about what is inaccurate/what I have misunderstood.

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

DM defines a single template usage format. It's a YAML based list of resource, each of which contains a name, type and parameters. True, the template definition can be written either in Jinja, which is simply marked up YAML, or Python, which produces YAML. A Jinja template for a k8s primitive is easily recognizable as a k8s configuration file with mustache markup.

Also, there's no requirement to use both languages in a single example, or to use multiple files, so both assertions are inaccurate.

DM can also take vanilla k8s configuration files as input. And, when a template is expanded, the result is always a flat list of vanilla k8s configuration files, which is readily accessible via the API.

DM doesn't define a k8s API object specifically because a native implementation baked into k8s has been discouraged by the preceding discussion. If that has somehow changed, we could easily make a DM template a native API object. However, we believe there are good reasons not to do that, not least of which is the well established precedent of publishing content in registries, rather than in runtimes. That said, when a configuration is deployed using DM, it is capture and stored in the cluster, providing a record of the deployment, including the parameters and expansion results.

If by native support for generation logic, you mean hard wired, as opposed to programmable, then no, DM doesn't have hard wired generators. We consider hard wired implementations to be an anti-pattern, in general.

@jackgr

jackgr Dec 8, 2015

Contributor

DM defines a single template usage format. It's a YAML based list of resource, each of which contains a name, type and parameters. True, the template definition can be written either in Jinja, which is simply marked up YAML, or Python, which produces YAML. A Jinja template for a k8s primitive is easily recognizable as a k8s configuration file with mustache markup.

Also, there's no requirement to use both languages in a single example, or to use multiple files, so both assertions are inaccurate.

DM can also take vanilla k8s configuration files as input. And, when a template is expanded, the result is always a flat list of vanilla k8s configuration files, which is readily accessible via the API.

DM doesn't define a k8s API object specifically because a native implementation baked into k8s has been discouraged by the preceding discussion. If that has somehow changed, we could easily make a DM template a native API object. However, we believe there are good reasons not to do that, not least of which is the well established precedent of publishing content in registries, rather than in runtimes. That said, when a configuration is deployed using DM, it is capture and stored in the cluster, providing a record of the deployment, including the parameters and expansion results.

If by native support for generation logic, you mean hard wired, as opposed to programmable, then no, DM doesn't have hard wired generators. We consider hard wired implementations to be an anti-pattern, in general.

This comment has been minimized.

@bparees

bparees Dec 10, 2015

Contributor

Also, there's no requirement to use both languages in a single example, or to use multiple files, so both assertions are inaccurate.

is it possible to write a single file, in k8s-compatible json syntax, which provides parameterization? That is what is achieved by the openshift template schema, and what I am trying to distinguish here as a desirable trait. Whether an example can be created which doesn't need additional syntax or files isn't really my point, the question is whether the full power of DM can be realized without branching out into multiple files and multiple syntaxes (above and beyond the k8s syntax).

My impression is that the answer is no. Is it a desirable goal (single file, k8s syntax, api object)? in my opinion, based on the user experience we have been able to deliver in openshift by integrating templates as a first class concept, is that it is, but I think we'll be hashing that out in the syntax/schema discussion once I update this PR per our discussion yesterday.

Regarding parameter generation, I agree with the concerns about it being compiled into the runtime and it would be nice to make it more flexible. On the other hand, by being a little opinionated here, we provided a generation function that covers probably 70% of the use cases for parameter generation (no one has come to us so far asking for additional generation functionality) in an extremely consumable format that minimizes complexity or requirements to learn new languages.

@bparees

bparees Dec 10, 2015

Contributor

Also, there's no requirement to use both languages in a single example, or to use multiple files, so both assertions are inaccurate.

is it possible to write a single file, in k8s-compatible json syntax, which provides parameterization? That is what is achieved by the openshift template schema, and what I am trying to distinguish here as a desirable trait. Whether an example can be created which doesn't need additional syntax or files isn't really my point, the question is whether the full power of DM can be realized without branching out into multiple files and multiple syntaxes (above and beyond the k8s syntax).

My impression is that the answer is no. Is it a desirable goal (single file, k8s syntax, api object)? in my opinion, based on the user experience we have been able to deliver in openshift by integrating templates as a first class concept, is that it is, but I think we'll be hashing that out in the syntax/schema discussion once I update this PR per our discussion yesterday.

Regarding parameter generation, I agree with the concerns about it being compiled into the runtime and it would be nice to make it more flexible. On the other hand, by being a little opinionated here, we provided a generation function that covers probably 70% of the use cases for parameter generation (no one has come to us so far asking for additional generation functionality) in an extremely consumable format that minimizes complexity or requirements to learn new languages.

docs/proposals/templates.md
+* Provide a management mechanism for deleting/uninstalling an entire set of components related to a single deployed application
+* As a developer I want to select the template to use from a list of available templates
+* Enabling the creation of user interfaces that can guide an application deployer through the deployment process with descriptive help about the configuration value decisions they are making, and useful default values where appropriate
+

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

Additional use case:

  • Capturing the topology of an application from template expansion as metadata within the cluster, so that
    • visualizations can be provided that expose application structure,
    • developers can inspect a cluster to determine what's running in it,
    • tools can use the metadata to aggregate and analyze signals.
@jackgr

jackgr Dec 8, 2015

Contributor

Additional use case:

  • Capturing the topology of an application from template expansion as metadata within the cluster, so that
    • visualizations can be provided that expose application structure,
    • developers can inspect a cluster to determine what's running in it,
    • tools can use the metadata to aggregate and analyze signals.

This comment has been minimized.

@bparees

bparees Dec 10, 2015

Contributor

is
"* Exporting a set of objects in a namespace as a template so the topology can be inspected/visualized or recreated in another environment"

a fair summary of that?

@bparees

bparees Dec 10, 2015

Contributor

is
"* Exporting a set of objects in a namespace as a template so the topology can be inspected/visualized or recreated in another environment"

a fair summary of that?

This comment has been minimized.

@bgrant0607

bgrant0607 Jan 4, 2016

Member

Note that application topology could be captured by labels and/or annotations, as well, without templates.

@bgrant0607

bgrant0607 Jan 4, 2016

Member

Note that application topology could be captured by labels and/or annotations, as well, without templates.

docs/proposals/templates.md
+descriptions of the purpose of each knob
+* Allow application deployers to easily customize exposed values like object names while maintaining referential integrity
+ between dependent pieces
+* Support maintaining a library of templates within Kubernetes that can be accessed and instantiated by end users

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

Template libraries are likely to evolve at a different rates and through different means than the Kubernetes code base. In addition, there are well established mechanisms and processes for evolving shared development artifacts, most notably Github. We should therefore not require the template libraries to reside within Kubernetes itself.

We can and should require templates to be addressable using a simple URL, and/or a shortened URL format similar to go lang imports (e.g., github.com/kubernetes/application-dm-templates/storage/redis:v1).

@jackgr

jackgr Dec 8, 2015

Contributor

Template libraries are likely to evolve at a different rates and through different means than the Kubernetes code base. In addition, there are well established mechanisms and processes for evolving shared development artifacts, most notably Github. We should therefore not require the template libraries to reside within Kubernetes itself.

We can and should require templates to be addressable using a simple URL, and/or a shortened URL format similar to go lang imports (e.g., github.com/kubernetes/application-dm-templates/storage/redis:v1).

This comment has been minimized.

@bparees

bparees Dec 10, 2015

Contributor

There's definitely no requirement they live there, we support instantiating templates from github as well (oc process -f https://raw.github.com/foo/bar/mytemplate.json), however it is a very convenient way for an administrator to make a DB service available to users, for example, or provide a common jenkins service. You provide a template within the cluster and users can instantiate that template without leaving the confines of the cluster.

@bparees

bparees Dec 10, 2015

Contributor

There's definitely no requirement they live there, we support instantiating templates from github as well (oc process -f https://raw.github.com/foo/bar/mytemplate.json), however it is a very convenient way for an administrator to make a DB service available to users, for example, or provide a common jenkins service. You provide a template within the cluster and users can instantiate that template without leaving the confines of the cluster.

This comment has been minimized.

@sparkprime

sparkprime Dec 16, 2015

Terraform has a centralized template library on github: https://github.com/terraform-community-modules

@sparkprime

sparkprime Dec 16, 2015

Terraform has a centralized template library on github: https://github.com/terraform-community-modules

docs/proposals/templates.md
+arbitrarily complex templates with parameters, and tooling can be built around such schemes.
+
+This desire for simplicity also intentionally excludes template composability/embedding as a supported use case.
+

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

Excluding template composition/embedding doesn't necessarily make things simpler. In fact, the DRY (don't repeat yourself) principle codifies the widely shared industry experience that reuse by copy/paste rapidly leads to much greater complexity and much more difficult maintenance challenges than inclusion by reference.

Unless we have compelling evidence to the contrary, we should strike this requirement from this document.

@jackgr

jackgr Dec 8, 2015

Contributor

Excluding template composition/embedding doesn't necessarily make things simpler. In fact, the DRY (don't repeat yourself) principle codifies the widely shared industry experience that reuse by copy/paste rapidly leads to much greater complexity and much more difficult maintenance challenges than inclusion by reference.

Unless we have compelling evidence to the contrary, we should strike this requirement from this document.

This comment has been minimized.

@bparees

bparees Dec 10, 2015

Contributor

including things by reference leads to fragility as the things you reference bit rot or disappear or get changed underneath you. Furthermore a consumer of something that includes 3 other things by reference, if they are diligent, is going to need to go track down those other 3 things to understand what they are going to do before blindly instantiating it.

obviously i'm not saying there's no place for DRY, but in my experience writing templates so far, i think embedding them would have only made my life harder in trying to keep everything in sync when i changed something that other templates depended on, rather than making it simpler by allowing to me write something one time. And that's in a scenario where i had total control over the entire hierarchy of template content.

lastly the biggest challenge we've had with templates so far is getting people to understand how to write them (what's the valid syntax, what's the schema for the objects, etc), and i'm worried that adding additional semantics like includes is only going to raise the bar on understanding them.

Regardless i'd like to see some other folks weigh in before we remove the restriction, it's a pretty clear decision point so hopefully we can get some consensus one way or another as we attempt to define the proposed syntax.

@bparees

bparees Dec 10, 2015

Contributor

including things by reference leads to fragility as the things you reference bit rot or disappear or get changed underneath you. Furthermore a consumer of something that includes 3 other things by reference, if they are diligent, is going to need to go track down those other 3 things to understand what they are going to do before blindly instantiating it.

obviously i'm not saying there's no place for DRY, but in my experience writing templates so far, i think embedding them would have only made my life harder in trying to keep everything in sync when i changed something that other templates depended on, rather than making it simpler by allowing to me write something one time. And that's in a scenario where i had total control over the entire hierarchy of template content.

lastly the biggest challenge we've had with templates so far is getting people to understand how to write them (what's the valid syntax, what's the schema for the objects, etc), and i'm worried that adding additional semantics like includes is only going to raise the bar on understanding them.

Regardless i'd like to see some other folks weigh in before we remove the restriction, it's a pretty clear decision point so hopefully we can get some consensus one way or another as we attempt to define the proposed syntax.

This comment has been minimized.

@thockin

thockin Dec 12, 2015

Member

We need to fix the referential integrity problem in the API anyway. I'd say that is a more urgent problem than templating, in general.

@thockin

thockin Dec 12, 2015

Member

We need to fix the referential integrity problem in the API anyway. I'd say that is a more urgent problem than templating, in general.

This comment has been minimized.

@sparkprime

sparkprime Dec 16, 2015

All templates need to be versioned. Git allows this already, so it's not a big deal. That avoids the problem of changing dependencies / bit rot.

The syntax for invoking a sub template has to be the same as invoking the first template, assuming multiple languages are supported (see DM). So there is not a big cognitive burden.

@sparkprime

sparkprime Dec 16, 2015

All templates need to be versioned. Git allows this already, so it's not a big deal. That avoids the problem of changing dependencies / bit rot.

The syntax for invoking a sub template has to be the same as invoking the first template, assuming multiple languages are supported (see DM). So there is not a big cognitive burden.

This comment has been minimized.

@bparees

bparees Dec 18, 2015

Contributor

i've removed this explicit exclusion for now. Whether the template api object itself allows the definition of compositions, or if we punt it into the template body and let the expansion plugin do what it wants, still needs to be decided.

@bparees

bparees Dec 18, 2015

Contributor

i've removed this explicit exclusion for now. Whether the template api object itself allows the definition of compositions, or if we punt it into the template body and let the expansion plugin do what it wants, still needs to be decided.

docs/proposals/templates.md
+
+Allowing templates to reference other templates presents versioning+consistency challenges along with making the template
+no longer a self-contained portable object. Scenarios necessitating multiple templates can be handled in one of several
+alternate ways:

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

It's quite straightforward to version templates, and to include template version information in references. See this document for an example of a simple but effective versioning scheme.

Beyond including template version information in references, a standardized reference format is essential, so that references can be unambiguously resolved against public registries. See this document for an example of Github based template references.

We should revise this document to require that templates be versioned and that templates residing in public registries be accessible through a standardized reference format.

@jackgr

jackgr Dec 8, 2015

Contributor

It's quite straightforward to version templates, and to include template version information in references. See this document for an example of a simple but effective versioning scheme.

Beyond including template version information in references, a standardized reference format is essential, so that references can be unambiguously resolved against public registries. See this document for an example of Github based template references.

We should revise this document to require that templates be versioned and that templates residing in public registries be accessible through a standardized reference format.

This comment has been minimized.

@bparees

bparees Dec 10, 2015

Contributor

in the next iteration of the document i'm striking any concerns with tooling or storage of templates and focusing only on the syntax of the template definition, so the public registry issue will go away.

We can add a version field to the proposed template schema.

@bparees

bparees Dec 10, 2015

Contributor

in the next iteration of the document i'm striking any concerns with tooling or storage of templates and focusing only on the syntax of the template definition, so the public registry issue will go away.

We can add a version field to the proposed template schema.

This comment has been minimized.

@thockin

thockin Dec 12, 2015

Member

Will it be possible, in general, to convert templates between API version? If I have a template of a v1 Pod, and I want to upgrade to V2, it's possible the fields that were templated no longer exist in v2, or have been decompsed or semantically changed. I think we need to call this out as a general problem of server-side templating, unless there's a great answer I don't see.

@thockin

thockin Dec 12, 2015

Member

Will it be possible, in general, to convert templates between API version? If I have a template of a v1 Pod, and I want to upgrade to V2, it's possible the fields that were templated no longer exist in v2, or have been decompsed or semantically changed. I think we need to call this out as a general problem of server-side templating, unless there's a great answer I don't see.

This comment has been minimized.

@bparees

bparees Dec 12, 2015

Contributor

Today we take the approach that the content within the template (eg the v1 pod) is opaque, so migrating a template from v1 to v2 would not also migrate the content within the template, just the fields of the template itself (parameter definitions, etc).

Not sure how the problem is specific to server-side templating. A client processing a v1 template to submit to a v2 api endpoint would face the same problem.

@bparees

bparees Dec 12, 2015

Contributor

Today we take the approach that the content within the template (eg the v1 pod) is opaque, so migrating a template from v1 to v2 would not also migrate the content within the template, just the fields of the template itself (parameter definitions, etc).

Not sure how the problem is specific to server-side templating. A client processing a v1 template to submit to a v2 api endpoint would face the same problem.

This comment has been minimized.

@bgrant0607

bgrant0607 Jan 4, 2016

Member

With respect to template versioning:

I agree that templates need to be versioned, but I don't know that we need an explicit field for that.

The version could be included in the resource name, and labels could be used to find a collection of similar templates.

@bgrant0607

bgrant0607 Jan 4, 2016

Member

With respect to template versioning:

I agree that templates need to be versioned, but I don't know that we need an explicit field for that.

The version could be included in the resource name, and labels could be used to find a collection of similar templates.

This comment has been minimized.

@jackgr

jackgr Jan 13, 2016

Contributor

Placing the version in the resource name makes it easy to create versioned graphs. That's how template versioning works in DM. See this document section.

@jackgr

jackgr Jan 13, 2016

Contributor

Placing the version in the resource name makes it easy to create versioned graphs. That's how template versioning works in DM. See this document section.

docs/proposals/templates.md
+We begin by looking at the List object which allows a user to easily group a set of objects together for easy creation via a
+single CLI invocation. It also provides a portable format which requires only a single file to represent an application.
+
+From that starting point, we propose a Template API object which can encapsulate the definition of all components of an

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

One of the requirements governing the design and development of templating technology for Kubernetes is that a templating mechanism should not be baked into the API server. It seems reasonable that templating services might want to store state in etcd using Third Party Resource support, but a strongly typed one size fits all native templating API has been considered an anti pattern up to this point.

@jackgr

jackgr Dec 8, 2015

Contributor

One of the requirements governing the design and development of templating technology for Kubernetes is that a templating mechanism should not be baked into the API server. It seems reasonable that templating services might want to store state in etcd using Third Party Resource support, but a strongly typed one size fits all native templating API has been considered an anti pattern up to this point.

docs/proposals/templates.md
+
+The primary capability provided is that parameter values can easily be shared between components, such as a database password
+that is provided by the user once, but then attached as an environment variable to both a database pod and a web frontend pod.
+

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

This benefit can be achieved through other means, such as storing template parameter values in the cluster using service specific storage. This is what DM does today.

@jackgr

jackgr Dec 8, 2015

Contributor

This benefit can be achieved through other means, such as storing template parameter values in the cluster using service specific storage. This is what DM does today.

This comment has been minimized.

@bgrant0607

bgrant0607 Jan 13, 2016

Member

I think the point is to simplify the creation of a related set of objects that are linked together in some way, taking into account that resource names, label selectors, and other values (e.g., an application cluster instance id, secrets) need to be substituted consistently across the set of objects.

@bgrant0607

bgrant0607 Jan 13, 2016

Member

I think the point is to simplify the creation of a related set of objects that are linked together in some way, taking into account that resource names, label selectors, and other values (e.g., an application cluster instance id, secrets) need to be substituted consistently across the set of objects.

docs/proposals/templates.md
+
+## Known Limitations
+
+* Generators must be statically compiled into the template api endpoint

This comment has been minimized.

@jackgr

jackgr Dec 8, 2015

Contributor

This is a significant limitation. By contrast, DM allows an open ended set of templates to be stored in registries, and referenced on demand. Templates can be written in Python, allowing for open ended generation capability. No changes to the Kubernetes master are required to introduce new capabilities.

@jackgr

jackgr Dec 8, 2015

Contributor

This is a significant limitation. By contrast, DM allows an open ended set of templates to be stored in registries, and referenced on demand. Templates can be written in Python, allowing for open ended generation capability. No changes to the Kubernetes master are required to introduce new capabilities.

docs/proposals/templates.md
+ any necessary configuration data.
+
+
+## Proposed Design

This comment has been minimized.

@jackgr

jackgr Dec 9, 2015

Contributor

Up to this point, the document can read as a general proposal for templating in Kubernetes. I'd like to see it broken into two documents:

  • A document that establishes the general requirements and design assumptions, consisting of the content up to this point. Since this content deals with templating in general, the current document and PR names are appropriate.
  • A document proposing to use the OpenShift templating mechanism as the solution adopting to address those requirements. The name of this second document and the name of the PR for it should both mention OpenShift explicitly, so that it's clear they're referring to a specific implementation technology.

The motivation for this request is to create a level playing field for the several technologies available to satisfy the general requirements. By splitting this document into two, all technology providers can collaborate on the requirements document, and each technology provider can submit a technology proposal.

I would further suggest that this effort be clearly advertised via the Config SIG mailing list and chat room, so that all technology providers can be apprised of its existence, and can choose to participate, if interested.

@jackgr

jackgr Dec 9, 2015

Contributor

Up to this point, the document can read as a general proposal for templating in Kubernetes. I'd like to see it broken into two documents:

  • A document that establishes the general requirements and design assumptions, consisting of the content up to this point. Since this content deals with templating in general, the current document and PR names are appropriate.
  • A document proposing to use the OpenShift templating mechanism as the solution adopting to address those requirements. The name of this second document and the name of the PR for it should both mention OpenShift explicitly, so that it's clear they're referring to a specific implementation technology.

The motivation for this request is to create a level playing field for the several technologies available to satisfy the general requirements. By splitting this document into two, all technology providers can collaborate on the requirements document, and each technology provider can submit a technology proposal.

I would further suggest that this effort be clearly advertised via the Config SIG mailing list and chat room, so that all technology providers can be apprised of its existence, and can choose to participate, if interested.

@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Dec 9, 2015

Contributor

@brendandburns this is the template proposal mentioned on the call.

Contributor

bparees commented Dec 9, 2015

@brendandburns this is the template proposal mentioned on the call.

+// the Template to Config transformation.
+type Parameter struct {
+ // Required: Parameter name must be set and it can be referenced in Template
+ // Items using $(PARAMETER_NAME)

This comment has been minimized.

@thockin

thockin Dec 10, 2015

Member

Where can this expansion be done? Any string field of any object in the Objects[] list? How do we do that - reflection/code-gen (e.g. func ExpandTemplates(obj, expansions)) ?

@thockin

thockin Dec 10, 2015

Member

Where can this expansion be done? Any string field of any object in the Objects[] list? How do we do that - reflection/code-gen (e.g. func ExpandTemplates(obj, expansions)) ?

This comment has been minimized.

@bparees

bparees Dec 10, 2015

Contributor

Today yes, any string field gets checked for parameter substitution:
https://github.com/openshift/origin/blob/master/pkg/template/template.go#L120-L139

however as discussed below, i'd like to eliminate the string-only restriction as part of this proposal. a few approaches to that are discussed toward the end of the document.

@bparees

bparees Dec 10, 2015

Contributor

Today yes, any string field gets checked for parameter substitution:
https://github.com/openshift/origin/blob/master/pkg/template/template.go#L120-L139

however as discussed below, i'd like to eliminate the string-only restriction as part of this proposal. a few approaches to that are discussed toward the end of the document.

docs/proposals/templates.md
+ // Optional: Value holds the Parameter data. If specified, the generator
+ // will be ignored. The value replaces all occurrences of the Parameter
+ // $(Name) expression during the Template to Config transformation.
+ Value string

This comment has been minimized.

@thockin

thockin Dec 10, 2015

Member

What about numeric or bool values? Those don't seem unreasonable. Struct values I can see being OOB. Should this at least carry a 'Type' field for which "String" is the only value?

@thockin

thockin Dec 10, 2015

Member

What about numeric or bool values? Those don't seem unreasonable. Struct values I can see being OOB. Should this at least carry a 'Type' field for which "String" is the only value?

This comment has been minimized.

@bparees

bparees Dec 10, 2015

Contributor

Agree, again this restriction is discussed at the end of the doc and i'd love to eliminate it. Adding a "type" field to the Parameter struct is one of the possible ways to address it, and it's the one i'm currently leaning towards.

@bparees

bparees Dec 10, 2015

Contributor

Agree, again this restriction is discussed at the end of the doc and i'd love to eliminate it. Adding a "type" field to the Parameter struct is one of the possible ways to address it, and it's the one i'm currently leaning towards.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Feb 5, 2016

GCE e2e test build/test passed for commit 7ad5ea6.

k8s-bot commented Feb 5, 2016

GCE e2e test build/test passed for commit 7ad5ea6.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Feb 5, 2016

GCE e2e test build/test passed for commit 882cf8a.

k8s-bot commented Feb 5, 2016

GCE e2e test build/test passed for commit 882cf8a.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Feb 6, 2016

Member

I agree with @liggitt

Member

bgrant0607 commented Feb 6, 2016

I agree with @liggitt

@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Feb 6, 2016

Contributor

I've added the Type back in as optional:
bparees@882cf8a#diff-fcdb9e0b56ea1ae85513bbea4972949eR233

remaining Todo is how to use this w/ secret generation. Is there a proposal for secret generation I can refer to?

Contributor

bparees commented Feb 6, 2016

I've added the Type back in as optional:
bparees@882cf8a#diff-fcdb9e0b56ea1ae85513bbea4972949eR233

remaining Todo is how to use this w/ secret generation. Is there a proposal for secret generation I can refer to?

@smarterclayton

This comment has been minimized.

Show comment
Hide comment
@smarterclayton

smarterclayton Feb 6, 2016

Contributor
Contributor

smarterclayton commented Feb 6, 2016

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Feb 7, 2016

Member

I'm using #12732 for secret generation. There's not a proposal yet AFAIK.

Member

bgrant0607 commented Feb 7, 2016

I'm using #12732 for secret generation. There's not a proposal yet AFAIK.

docs/proposals/templates.md
+* Follow established Kubernetes API patterns by defining new template related APIs which consume+return first class Kubernetes
+ API (and therefore json conformant) objects.
+
+We do not wish to invent a new turing complete templating language. There are good options available

This comment has been minimized.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

Nit: Turing-complete

@bgrant0607

bgrant0607 Feb 7, 2016

Member

Nit: Turing-complete

docs/proposals/templates.md
+ // Optional: Parameter can have description
+ Description string
+
+ // Optional: Value holds the Parameter data. If specified, the generator

This comment has been minimized.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

We removed generators, so this comment needs to be updated.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

We removed generators, so this comment needs to be updated.

docs/proposals/templates.md
+
+### API Endpoints
+
+* **/processedTemplates** - when a template is POSTed to this endpoint, all parameters in the template are processed, values

This comment has been minimized.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

Please remove text regarding generation.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

Please remove text regarding generation.

This comment has been minimized.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

Also, endpoints are all lowercase: /processedtemplates

In the future I'll want a subresource endpoint that injects parameter values, but this is an acceptable initial step.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

Also, endpoints are all lowercase: /processedtemplates

In the future I'll want a subresource endpoint that injects parameter values, but this is an acceptable initial step.

This comment has been minimized.

@bparees

bparees Feb 8, 2016

Contributor

the subresource is described under the /templates endpoint. (/templates/templatename/processed)

@bparees

bparees Feb 8, 2016

Contributor

the subresource is described under the /templates endpoint. (/templates/templatename/processed)

docs/proposals/templates.md
+generated as necessary, and substituted into appropriate locations in the object definitions. Validation is performed to ensure
+required parameters have a value (either supplied in the original template or generated by a defined generator). In addition
+labels defined in the template are applied to the object definitions. Finally the customized template (still a `Template` object)
+is returned to the caller. (Would we prefer to return an ObjectList instead of a Template object, to make it easier for clients

This comment has been minimized.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

I guess returning a Template is more consistent with other endpoints, but is a bit more work for clients. I'm fine with the former for now.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

I guess returning a Template is more consistent with other endpoints, but is a bit more work for clients. I'm fine with the former for now.

This comment has been minimized.

@smarterclayton

smarterclayton Feb 7, 2016

Contributor

In the absence of generation, the only value from the server returning a
template is the possibility of returning other side channel information in
status field.

On Sun, Feb 7, 2016 at 5:31 PM, Brian Grant notifications@github.com
wrote:

In docs/proposals/templates.md
#18215 (comment)
:

  •  "name": "REPLICA_COUNT",
    
  •  "description": "Number of mongo replicas to run",
    
  •  "value": "1",
    
  •  "required": true
    
  • }
  • ]
    +}
    +```

+### API Endpoints
+
+* /processedTemplates - when a template is POSTed to this endpoint, all parameters in the template are processed, values
+generated as necessary, and substituted into appropriate locations in the object definitions. Validation is performed to ensure
+required parameters have a value (either supplied in the original template or generated by a defined generator). In addition
+labels defined in the template are applied to the object definitions. Finally the customized template (still aTemplate object)
+is returned to the caller. (Would we prefer to return an ObjectList instead of a Template object, to make it easier for clients

I guess returning a Template is more consistent with other endpoints, but
is a bit more work for clients. I'm fine with the former for now.


Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/18215/files#r52126263.

@smarterclayton

smarterclayton Feb 7, 2016

Contributor

In the absence of generation, the only value from the server returning a
template is the possibility of returning other side channel information in
status field.

On Sun, Feb 7, 2016 at 5:31 PM, Brian Grant notifications@github.com
wrote:

In docs/proposals/templates.md
#18215 (comment)
:

  •  "name": "REPLICA_COUNT",
    
  •  "description": "Number of mongo replicas to run",
    
  •  "value": "1",
    
  •  "required": true
    
  • }
  • ]
    +}
    +```

+### API Endpoints
+
+* /processedTemplates - when a template is POSTed to this endpoint, all parameters in the template are processed, values
+generated as necessary, and substituted into appropriate locations in the object definitions. Validation is performed to ensure
+required parameters have a value (either supplied in the original template or generated by a defined generator). In addition
+labels defined in the template are applied to the object definitions. Finally the customized template (still aTemplate object)
+is returned to the caller. (Would we prefer to return an ObjectList instead of a Template object, to make it easier for clients

I guess returning a Template is more consistent with other endpoints, but
is a bit more work for clients. I'm fine with the former for now.


Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/18215/files#r52126263.

docs/proposals/templates.md
+* Editing a template (if/when first class editing tools are created)
+* Storing/retrieving template objects with a central store
+
+Note that the `/processedTemplates/sometemplate` subresource would accept a standalone set of parameters to be applied to `sometemplate`.

This comment has been minimized.

@bgrant0607

bgrant0607 Feb 7, 2016

Member

The subresource I imagined was something like:

/apis/extensions/v1beta1/namespaces/myns/templates/mytemplate/substituted

@bgrant0607

bgrant0607 Feb 7, 2016

Member

The subresource I imagined was something like:

/apis/extensions/v1beta1/namespaces/myns/templates/mytemplate/substituted

This comment has been minimized.

@smarterclayton

smarterclayton Feb 7, 2016

Contributor

We still need a POST /processedTemplate in order to allow clients to
transform templates not stored on the server, but agree subresource vs. top
level transform for templates stored on the server probably leans a bit
towards subresource for consistency.

On Sun, Feb 7, 2016 at 5:42 PM, Brian Grant notifications@github.com
wrote:

In docs/proposals/templates.md
#18215 (comment)
:

+### Where to define parameters
+
+There has been some discussion around where to define parameters that are being injected into a Template
+
+1. In a separate standalone file
+2. Within the Template itself
+
+This proposal suggests including the parameter definitions within the Template, which provides a self-contained structure that
+can be easily versioned, transported, and instantiated without risk of mismatching content. In addition, a Template can easily
+be validated to confirm that all parameter references are resolveable.
+
+Separating the parameter definitions makes for a more complex process with respect to
+* Editing a template (if/when first class editing tools are created)
+* Storing/retrieving template objects with a central store
+
+Note that the /processedTemplates/sometemplate subresource would accept a standalone set of parameters to be applied to sometemplate.

The subresource I imagined was something like:

/apis/extensions/v1beta1/namespaces/myns/templates/mytemplate/substituted


Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/18215/files#r52126446.

@smarterclayton

smarterclayton Feb 7, 2016

Contributor

We still need a POST /processedTemplate in order to allow clients to
transform templates not stored on the server, but agree subresource vs. top
level transform for templates stored on the server probably leans a bit
towards subresource for consistency.

On Sun, Feb 7, 2016 at 5:42 PM, Brian Grant notifications@github.com
wrote:

In docs/proposals/templates.md
#18215 (comment)
:

+### Where to define parameters
+
+There has been some discussion around where to define parameters that are being injected into a Template
+
+1. In a separate standalone file
+2. Within the Template itself
+
+This proposal suggests including the parameter definitions within the Template, which provides a self-contained structure that
+can be easily versioned, transported, and instantiated without risk of mismatching content. In addition, a Template can easily
+be validated to confirm that all parameter references are resolveable.
+
+Separating the parameter definitions makes for a more complex process with respect to
+* Editing a template (if/when first class editing tools are created)
+* Storing/retrieving template objects with a central store
+
+Note that the /processedTemplates/sometemplate subresource would accept a standalone set of parameters to be applied to sometemplate.

The subresource I imagined was something like:

/apis/extensions/v1beta1/namespaces/myns/templates/mytemplate/substituted


Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/18215/files#r52126446.

This comment has been minimized.

@bparees

bparees Feb 8, 2016

Contributor

@smarterclayton I think you're referring to something different.
@bgrant0607 yup, i had updated it under the /templates api endpoint but missed it here.

For clarity in this comment:

  • /processedtemplates - accepts a template json, performs substitution, returns a template json.
  • /templates - CRUD for template api objects
  • /templates/sometemplate/processed - accepts a set of parameters, returns a parameter-substituted template json
@bparees

bparees Feb 8, 2016

Contributor

@smarterclayton I think you're referring to something different.
@bgrant0607 yup, i had updated it under the /templates api endpoint but missed it here.

For clarity in this comment:

  • /processedtemplates - accepts a template json, performs substitution, returns a template json.
  • /templates - CRUD for template api objects
  • /templates/sometemplate/processed - accepts a set of parameters, returns a parameter-substituted template json
@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Feb 7, 2016

Member

Other than minor cleanup, LGTM.

Member

bgrant0607 commented Feb 7, 2016

Other than minor cleanup, LGTM.

@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Feb 8, 2016

Contributor

@bgrant0607: comments addressed in latest commit.

Contributor

bparees commented Feb 8, 2016

@bgrant0607: comments addressed in latest commit.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Feb 8, 2016

GCE e2e test build/test passed for commit 246fd71.

k8s-bot commented Feb 8, 2016

GCE e2e test build/test passed for commit 246fd71.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Feb 9, 2016

Member

LGTM. Please rebase, run hack/update-generated-docs.sh, and squash, and then I'll apply the label for merging.

Member

bgrant0607 commented Feb 9, 2016

LGTM. Please rebase, run hack/update-generated-docs.sh, and squash, and then I'll apply the label for merging.

@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Feb 9, 2016

Contributor

@bgrant0607 done.

Contributor

bparees commented Feb 9, 2016

@bgrant0607 done.

@k8s-teamcity-mesosphere

This comment has been minimized.

Show comment
Hide comment
@k8s-teamcity-mesosphere

k8s-teamcity-mesosphere Feb 9, 2016

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 15061 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 225 Build time: 00:03:59

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 15061 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 225 Build time: 00:03:59

This comment has been minimized.

Show comment
Hide comment
@k8s-teamcity-mesosphere

k8s-teamcity-mesosphere Feb 9, 2016

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 15062 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 225 Build time: 00:03:56

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 15062 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 225 Build time: 00:03:56

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Feb 9, 2016

GCE e2e test build/test passed for commit a2b5112.

k8s-bot commented Feb 9, 2016

GCE e2e test build/test passed for commit a2b5112.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Feb 9, 2016

GCE e2e test build/test passed for commit 89a6561.

k8s-bot commented Feb 9, 2016

GCE e2e test build/test passed for commit 89a6561.

@bgrant0607 bgrant0607 added the lgtm label Feb 9, 2016

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Feb 9, 2016

Member

LGTM!

Member

bgrant0607 commented Feb 9, 2016

LGTM!

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Feb 9, 2016

Collaborator

Automatic merge from submit-queue

Collaborator

k8s-merge-robot commented Feb 9, 2016

Automatic merge from submit-queue

k8s-merge-robot added a commit that referenced this pull request Feb 9, 2016

Merge pull request #18215 from bparees/templates
Auto commit by PR queue bot

@k8s-merge-robot k8s-merge-robot merged commit 33636a3 into kubernetes:master Feb 9, 2016

5 checks passed

Jenkins GCE e2e 226 tests run, 106 skipped, 0 failed.
Details
Jenkins unit/integration 2724 tests run, 9 skipped, 0 failed.
Details
Submit Queue Queued to run github e2e tests a second time.
Details
cla/google All necessary CLAs are signed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Feb 9, 2016

Contributor

thanks @bgrant0607. Hopefully we can find some people to start the implementation come early march.

Contributor

bparees commented Feb 9, 2016

thanks @bgrant0607. Hopefully we can find some people to start the implementation come early march.

@jimmycuadra

This comment has been minimized.

Show comment
Hide comment
@jimmycuadra

jimmycuadra Feb 25, 2016

Member

Not sure if this is the best place to discuss this now that the proposal has been merged or whether or I should start a new issue, but:

I've started on a CLI tool that attempts to implement client-side template processing using the format described in this proposal. You can find the program at https://github.com/InQuicker/ktmpl. It's written in Rust, but the details of the source code are not really important for discussion.

There are a few issues I've coming across while working on this implementation (some minor, some more significant):

  • The example shown in the proposal is not valid JSON. (Extra comma, I think?)
  • The example shown in the proposal uses the default value "password" for the parameter "MONGODB_USER" which is confusing and initially made me think my program was putting values in the wrong place until I went back and looked at the original text.
  • The example shown in the proposal does not include interpolations that cover all the cases described elsewhere in the proposal. I know it's just an example and not a comprehensive test suite, but without a more detailed specification, there wasn't much else to go on. Specifically, there are no examples of the two types of interpolation combined in one value, and there aren't examples of each of the types of interpolation surrounded by literal text.
  • ktmpl only supports YAML for now, since it's for my company's use specifically and our manifests will only be in YAML. When using the non-quoted interpolation, the type of the resulting value may be ambiguous without making ParameterType mandatory. If the template processor is implemented using simple text substitution, it may not be a problem to just output the value as a string without quotes and let the Kubernetes API figure it out, but in my case I'm actually walking the YAML document and transforming string values that include the interpolation syntax into their final type. Since the decision of whether or value will have quotes around it or not comes from the data type, there are cases (e.g. true vs. "true") where I'd need to know during substitution what the final type should be.
  • The comment above the ParameterType field says "one of string, int, bool, or base64" but that list doesn't quite make sense. Neither YAML nor JSON have a native base64 type (it'd just be a string) and a simple "int" does not cover how numbers are represented in either YAML or JSON. In YAML, there are many more number types, and it's unclear if or why a non-integer number should be rejected. In JSON, all numbers are the same type, so specifying "int" either seems like a mistake or implies some additional validation beyond what the the serialization format can represent.
  • The meaning of the Required field for a parameter is not well explained. It seems like it could be inferred from whether or not there is a default Value specified in the manifest. Or is Required supposed to indicate whether or not a literal null is acceptable?
Member

jimmycuadra commented Feb 25, 2016

Not sure if this is the best place to discuss this now that the proposal has been merged or whether or I should start a new issue, but:

I've started on a CLI tool that attempts to implement client-side template processing using the format described in this proposal. You can find the program at https://github.com/InQuicker/ktmpl. It's written in Rust, but the details of the source code are not really important for discussion.

There are a few issues I've coming across while working on this implementation (some minor, some more significant):

  • The example shown in the proposal is not valid JSON. (Extra comma, I think?)
  • The example shown in the proposal uses the default value "password" for the parameter "MONGODB_USER" which is confusing and initially made me think my program was putting values in the wrong place until I went back and looked at the original text.
  • The example shown in the proposal does not include interpolations that cover all the cases described elsewhere in the proposal. I know it's just an example and not a comprehensive test suite, but without a more detailed specification, there wasn't much else to go on. Specifically, there are no examples of the two types of interpolation combined in one value, and there aren't examples of each of the types of interpolation surrounded by literal text.
  • ktmpl only supports YAML for now, since it's for my company's use specifically and our manifests will only be in YAML. When using the non-quoted interpolation, the type of the resulting value may be ambiguous without making ParameterType mandatory. If the template processor is implemented using simple text substitution, it may not be a problem to just output the value as a string without quotes and let the Kubernetes API figure it out, but in my case I'm actually walking the YAML document and transforming string values that include the interpolation syntax into their final type. Since the decision of whether or value will have quotes around it or not comes from the data type, there are cases (e.g. true vs. "true") where I'd need to know during substitution what the final type should be.
  • The comment above the ParameterType field says "one of string, int, bool, or base64" but that list doesn't quite make sense. Neither YAML nor JSON have a native base64 type (it'd just be a string) and a simple "int" does not cover how numbers are represented in either YAML or JSON. In YAML, there are many more number types, and it's unclear if or why a non-integer number should be rejected. In JSON, all numbers are the same type, so specifying "int" either seems like a mistake or implies some additional validation beyond what the the serialization format can represent.
  • The meaning of the Required field for a parameter is not well explained. It seems like it could be inferred from whether or not there is a default Value specified in the manifest. Or is Required supposed to indicate whether or not a literal null is acceptable?
@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Feb 25, 2016

Contributor

thanks for the feedback, i've pushed fixes to the syntactical issues here:
#21981

Specifically, there are no examples of the two types of interpolation combined in one value

I believe there is:
// if both types of substitution exist, quoting is performed:

somefield: "prefix_$((FOO))_$(FOO)_suffix"  ->  somefield: "prefix_BAR_BAR_suffix"

and there aren't examples of each of the types of interpolation surrounded by literal text.

there can't be such an example because the format precludes using literal text (by which I assume you mean unquoted text). That's the purpose of the $(()) syntax, to strip the quotes where appropriate.

When using the non-quoted interpolation, the type of the resulting value may be ambiguous without making ParameterType mandatory. If the template processor is implemented using simple text substitution, it may not be a problem to just output the value as a string without quotes and let the Kubernetes API figure it out, but in my case I'm actually walking the YAML document and transforming string values that include the interpolation syntax into their final type.

There is an inherent assumption that the user who put the $() or $(()) reference into the field understood the type of the field and used the correct syntax/type. The purpose of the type field on the parameter definition is to allow clients to guide users with providing a value, not to aid in the actual substitution logic.

The comment above the ParameterType field says "one of string, int, bool, or base64" but that list doesn't quite make sense. Neither YAML nor JSON have a native base64 type (it'd just be a string) and a simple "int" does not cover how numbers are represented in either YAML or JSON.

"int" was probably overly shorthand(ie we should include other types), but the point of "base64" is again that this is intended to hint to the user of the template how the value is going to be used. Telling them the type is base64 tells them they need to provide a base64 encoded value for this parameter. The fact that ultimately that base64 value gets plugged in as a json string isn't relevant.

The meaning of the Required field for a parameter is not well explained. It seems like it could be inferred from whether or not there is a default Value specified in the manifest. Or is Required supposed to indicate whether or not a literal null is acceptable?

Required is intended to indicate that null/empty string is not an acceptable value for this parameter. The parameter must have a non-empty value either provided via the default, or provided by the user. I will update this in the above PR as well.

Contributor

bparees commented Feb 25, 2016

thanks for the feedback, i've pushed fixes to the syntactical issues here:
#21981

Specifically, there are no examples of the two types of interpolation combined in one value

I believe there is:
// if both types of substitution exist, quoting is performed:

somefield: "prefix_$((FOO))_$(FOO)_suffix"  ->  somefield: "prefix_BAR_BAR_suffix"

and there aren't examples of each of the types of interpolation surrounded by literal text.

there can't be such an example because the format precludes using literal text (by which I assume you mean unquoted text). That's the purpose of the $(()) syntax, to strip the quotes where appropriate.

When using the non-quoted interpolation, the type of the resulting value may be ambiguous without making ParameterType mandatory. If the template processor is implemented using simple text substitution, it may not be a problem to just output the value as a string without quotes and let the Kubernetes API figure it out, but in my case I'm actually walking the YAML document and transforming string values that include the interpolation syntax into their final type.

There is an inherent assumption that the user who put the $() or $(()) reference into the field understood the type of the field and used the correct syntax/type. The purpose of the type field on the parameter definition is to allow clients to guide users with providing a value, not to aid in the actual substitution logic.

The comment above the ParameterType field says "one of string, int, bool, or base64" but that list doesn't quite make sense. Neither YAML nor JSON have a native base64 type (it'd just be a string) and a simple "int" does not cover how numbers are represented in either YAML or JSON.

"int" was probably overly shorthand(ie we should include other types), but the point of "base64" is again that this is intended to hint to the user of the template how the value is going to be used. Telling them the type is base64 tells them they need to provide a base64 encoded value for this parameter. The fact that ultimately that base64 value gets plugged in as a json string isn't relevant.

The meaning of the Required field for a parameter is not well explained. It seems like it could be inferred from whether or not there is a default Value specified in the manifest. Or is Required supposed to indicate whether or not a literal null is acceptable?

Required is intended to indicate that null/empty string is not an acceptable value for this parameter. The parameter must have a non-empty value either provided via the default, or provided by the user. I will update this in the above PR as well.

@jimmycuadra

This comment has been minimized.

Show comment
Hide comment
@jimmycuadra

jimmycuadra Feb 25, 2016

Member

Thank you, @bparees, that was very helpful! Your changes in that PR do well to clarify the things that tripped me up.

When I said "there are no examples of the two types of interpolation combined in one value and there aren't examples of each of the types of interpolation surrounded by literal text," I meant that the example template doesn't include those forms, so you don't see them in a realistic context and you can't use that example as a very complete test case. That's a minor issue, though, and I can adjust for my own needs.

Thanks again!

Member

jimmycuadra commented Feb 25, 2016

Thank you, @bparees, that was very helpful! Your changes in that PR do well to clarify the things that tripped me up.

When I said "there are no examples of the two types of interpolation combined in one value and there aren't examples of each of the types of interpolation surrounded by literal text," I meant that the example template doesn't include those forms, so you don't see them in a realistic context and you can't use that example as a very complete test case. That's a minor issue, though, and I can adjust for my own needs.

Thanks again!

@bparees bparees deleted the bparees:templates branch Dec 5, 2016

@kargakis kargakis referenced this pull request in kubernetes/test-infra Mar 8, 2017

Merged

prow: make GKE-specific variables optional #2185

@jolestar

This comment has been minimized.

Show comment
Hide comment
@jolestar

jolestar Sep 5, 2017

Contributor

Not sure if this is the best place to ask that the proposal has been merged, but I can not find the progress of this proposal's implement anywhere. How about this proposal?

Contributor

jolestar commented Sep 5, 2017

Not sure if this is the best place to ask that the proposal has been merged, but I can not find the progress of this proposal's implement anywhere. How about this proposal?

@bparees

This comment has been minimized.

Show comment
Hide comment
@bparees

bparees Sep 5, 2017

Contributor

@jolestar it has not been implemented and at this point seems unlikely to be.

Contributor

bparees commented Sep 5, 2017

@jolestar it has not been implemented and at this point seems unlikely to be.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Sep 5, 2017

Member

@jolestar A client-based implementation is mentioned above: https://github.com/InQuicker/ktmpl

Openshift also has an implementation.

I agree with @bparees that it is unlikely to be implemented at this point.

My current thinking on this and related topics is here:
https://goo.gl/T66ZcD

Member

bgrant0607 commented Sep 5, 2017

@jolestar A client-based implementation is mentioned above: https://github.com/InQuicker/ktmpl

Openshift also has an implementation.

I agree with @bparees that it is unlikely to be implemented at this point.

My current thinking on this and related topics is here:
https://goo.gl/T66ZcD

@jolestar

This comment has been minimized.

Show comment
Hide comment
@jolestar

jolestar Sep 6, 2017

Contributor

Thank you @bparees @bgrant0607 for reply

Contributor

jolestar commented Sep 6, 2017

Thank you @bparees @bgrant0607 for reply

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment