Template Api Support #25293

pwittrock · 2016-05-07T00:29:19Z

* Add extensions/v1beta1 'Template' Api object to support parameter substitution into configs

This change is

pwittrock · 2016-05-07T00:32:42Z

cc @bparees Since you worked on the original design proposal it might be helpful for you to take a look at the api

cc @kubernetes/kube-api

pwittrock · 2016-05-07T00:35:35Z

All commits including "fixup" and earlier are from another PR that this depends on. No need to review them.

bparees · 2016-05-09T00:58:35Z

pkg/apis/extensions/types.go

+
+	// Optional: Value holds the Parameter data. If specified, the generator
+	// will be ignored. The value replaces all occurrences of the Parameter
+	// ${Name} expression during the Template to Config transformation.


there will no generators in the k8s implementation, and the substitution string/syntax will be $(Name) instead of ${Name} (actually the syntax will be $(Name) and $((Name)) depending whether you want the substituted result to be quoted or not)

bparees · 2016-05-09T01:11:43Z

cc @bparees Since you worked on the original design proposal it might be helpful for you to take a look at the api

the api objects look good, i'm a little fuzzier on the rest api, particularly around processing templates. In the spec we talked about 2 key aspects:

whether you post parameters, or a full template
whether you get back a full template(with substitution complete), or a list of objects

it looks like what's defined here is: post parameters, get back a list of objects.

I think we may still want the "post a template" input option so users can process a template without having to first "create" it. (if that's there and i missed it, sorry)

for the response object, i think just returning the list of objects (instead of the full template) is right.

0xmichalis · 2016-05-09T10:14:16Z

cc: @mfojtik

pwittrock · 2016-05-09T18:12:39Z

I think we may still want the "post a template" input option so users can process a template without having to first "create" it. (if that's there and i missed it, sorry)

Sounds good. I will talk with @lavalamp about how to best expose this in api. Do you have a preference (e.g. different process endpoints for different types, a single request that wraps both input types, etc)?

bparees · 2016-05-09T18:27:06Z

Sounds good. I will talk with @lavalamp about how to best expose this in api. Do you have a preference (e.g. different process endpoints for different types, a single request that wraps both input types, etc)?

a separate endpoint makes sense to me... logically you'd then have:

an api//templates//process api that you can post parameters to, which will be combined with that specific template
and
an api/<namespace(?)>/processtemplate api that takes a template input for processing. (having a namespace in the path doesn't make much sense here since you're not actually interacting with the namespace)

pwittrock · 2016-05-09T18:47:28Z

@bparees Daniel agreed the second endpoint it sensible way of doing this. Since this is the second to last week of coding for 1.3 I'd like to minimize the scope to an MVP before addressing anything else. Would you be ok with me adding a TODO for adding the second api endpoint once the first endpoint is complete?

bparees · 2016-05-09T18:50:03Z

Would you be ok with me adding a TODO for adding the second api endpoint once the first endpoint is complete?

sounds reasonable

lavalamp · 2016-05-09T20:50:15Z

Making the processtemplate endpoint cluster-scoped (no namespace) would make sense as long as there's no fan-out (calls to other resources).

Maybe too late to change now, but "config" might be a good group name for both template resources.

pwittrock · 2016-05-09T21:05:53Z

@lavalamp How hard would it be to create a config/v1/beta1 api group? Do you think it is worth it / Does it put this feature at risk for 1.3?

lavalamp · 2016-05-09T21:12:10Z

It requires a bunch of counter-intuitive boilerplate, basically, although
you can mostly copy-pasta. Maybe not worth it this late in the process.

On Mon, May 9, 2016 at 2:06 PM, Phillip Wittrock notifications@github.com
wrote:

@lavalamp https://github.com/lavalamp How hard would it be to create a
config/v1/beta1 api group? Do you think it is worth it / Does it put this
feature at risk for 1.3?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#25293 (comment)

pwittrock · 2016-05-09T21:15:08Z

Leaving in extensions SGTM then. Just getting the api hooked up was mainly copy paste, but it took me longer than expected to figure out exactly how it needed to be done.

pwittrock · 2016-05-09T21:56:05Z

PTAL

pwittrock · 2016-05-17T06:56:10Z

Squashed

pwittrock · 2016-05-17T06:56:30Z

The validation and processing logic will be in #25622

pwittrock · 2016-05-17T14:04:00Z

@k8s-bot node e2e test this issue: #25735

pwittrock · 2016-05-17T14:13:58Z

PTAL.

lavalamp · 2016-05-17T18:52:58Z

pkg/apis/extensions/types.go

+	unversioned.TypeMeta `json:",inline"`
+
+	// Name is the name of the Template to be processed.
+	Name string `json:"name"`


Looks like Brian is correct. Consider just using object meta instead, to be consistent with Binding.

ObjectMeta has a bunch of stuff like namespace and nameGenerator that I don't think we want. This is consistent with extensions/v1beta.DeploymentRollback and I think I prefer it. WDYT?

We use ObjectMeta in many places for virtual resources (like
SubjectAccessReview).

On Tue, May 17, 2016 at 2:53 PM, Daniel Smith notifications@github.com
wrote:

In pkg/apis/extensions/types.go
#25293 (comment)
:

Required bool json:"required,omitempty"

// Type is the type that the parameter value must be parsed to. Type may be one of

// 'string', 'integer', 'boolean', or 'base64'.

// Type is used by clients to provide validation of user input and direction to users.

// Parameters used to define integer or boolean fields (e.g. replicaCount) should have the

// Type set to integer or boolean accordingly.

Type string json:"type,omitempty"
+}

+// TemplateParameters contains the substitution parameter overrides when processing a Template
+type TemplateParameters struct {

unversioned.TypeMeta json:",inline"

// Name is the name of the Template to be processed.

Name string json:"name"

Looks like Brian is correct. Consider just using object meta instead, to
be consistent with Binding.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/25293/files/58c7e4126d33836cce7082e9aa04f6818be06129..c7a56db1935dabe7aea0399ebfbed3cf5661c927#r63581936

Never mind, that was a lie.

On Tue, May 17, 2016 at 3:43 PM, Clayton Coleman ccoleman@redhat.com
wrote:

We use ObjectMeta in many places for virtual resources (like
SubjectAccessReview).

On Tue, May 17, 2016 at 2:53 PM, Daniel Smith notifications@github.com
wrote:

In pkg/apis/extensions/types.go
#25293 (comment)
:

Required bool json:"required,omitempty"

// Type is the type that the parameter value must be parsed to. Type may be one of

// 'string', 'integer', 'boolean', or 'base64'.

// Type is used by clients to provide validation of user input and direction to users.

// Parameters used to define integer or boolean fields (e.g. replicaCount) should have the

// Type set to integer or boolean accordingly.

Type string json:"type,omitempty"
+}

+// TemplateParameters contains the substitution parameter overrides when processing a Template
+type TemplateParameters struct {

unversioned.TypeMeta json:",inline"

// Name is the name of the Template to be processed.

Name string json:"name"

Looks like Brian is correct. Consider just using object meta instead, to
be consistent with Binding.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/25293/files/58c7e4126d33836cce7082e9aa04f6818be06129..c7a56db1935dabe7aea0399ebfbed3cf5661c927#r63581936

I don't want any fields here that aren't actually needed/supported.

In retrospect, ObjectMeta was confusing, since most of the fields aren't usable, and it refers to the parent resource rather than the subresource. A subset of ObjectReference probably would have been more appropriate.

I'm open to anything that would make programmatic clients easier to write (e.g., used with the discovery API), but I don't envision users writing declarative config files for subresources.

@bgrant0607 @smarterclayton @lavalamp bump

@bparees @pwittrock I don't understand how the kubectl proposal relates to this API discussion.

AIUI, the problem we have is that, unlike admission control, REST handlers aren't passed all of the information available from the path in the request context. If we fix that problem, then the information doesn't need to be explicitly represented in the body schema for imperative subresources that aren't serialized in manifests nor in etcd.

@bgrant0607 You mean if we post the parameters to an api endpoint like:
api/templates/<templatename>/process

the rest handler can't determine "templatename"?

I would tend to agree that it makes more sense to fix that problem than to require the parameters struct include the templatename. Especially since i might want to use the same parameters resource definition with multiple templates.

@bparees Yes, that's exactly what I mean. I haven't looked at the code recently, but I recall it being a problem for /binding, /scale, and /rollback.

@bgrant0607 definitely sounds like something we want to fix if this is the fourth endpoint to have a use for that info.

pwittrock · 2016-05-17T23:28:21Z

This should be pushed to 1.4. Will still update with your comments @lavalamp.

k8s-github-robot · 2016-05-18T07:56:16Z

@pwittrock PR needs rebase

ncdc · 2016-07-12T15:01:01Z

@pwittrock do you still intend to try to get this in 1.4?

pwittrock · 2016-07-14T15:21:36Z

@ncdc I don't think so. Is this something that you are waiting for or were planning on?

smarterclayton · 2016-07-14T16:08:00Z

I think we need to discuss in the apps sig about the timing and implementation. It's important to us, but there's a lot of important stuff going on right now. I think the clearest overarching theme for Kube right now is "getting started" and there are things in templates that do address that, but it'll be a balancing on other timing.

pwittrock · 2016-07-18T16:46:33Z

@smarterclayton Sounds good to me. I am hoping that Helm will be able to address some of the same things as Templates for the getting started experience.

k8s-bot · 2016-08-18T00:52:16Z

GCE e2e build/test passed for commit 5124918.

k8s-bot · 2016-08-30T00:34:16Z

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message will repeat several times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

k8s-github-robot · 2016-10-09T15:24:13Z

This PR hasn't been active in 60 days. It will be closed in 29 days (Nov 8, 2016).

cc @janetkuo @pwittrock

You can add 'keep-open' label to prevent this from happening, or add a comment to keep it open another 90 days

pwittrock assigned janetkuo May 7, 2016

pwittrock added the team/ux label May 7, 2016

pwittrock added this to the v1.3 milestone May 7, 2016

googlebot added the cla: yes label May 7, 2016

k8s-github-robot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/new-api size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels May 7, 2016

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 7, 2016

bparees reviewed May 9, 2016
View reviewed changes

janetkuo added release-note Denotes a PR that will be considered when it comes time to generate release notes. release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. and removed release-note-label-needed labels May 9, 2016

pwittrock force-pushed the templates2 branch from 04f9316 to ab8364c Compare May 10, 2016 03:10

k8s-github-robot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/new-api and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels May 10, 2016

pwittrock force-pushed the templates2 branch from f82aff4 to 5124918 Compare May 17, 2016 06:51

lavalamp reviewed May 17, 2016
View reviewed changes

pwittrock removed this from the v1.3 milestone May 17, 2016

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 18, 2016

jcantrill mentioned this pull request Jul 12, 2016

Provide a way to categorize or group common parameters in a template #28847

Closed

This was referenced Jul 19, 2016

Config template parameterization #11492

Closed

Implement templates #23896

Closed

Preprocessing config files for kubectl #13241

Closed

zakiva mentioned this pull request Aug 16, 2016

Add support for processing templates ManageIQ/kubeclient#183

Merged

simon3z mentioned this pull request Sep 14, 2016

Support k8s api discovery and group api ManageIQ/kubeclient#185

Merged

k8s-github-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Oct 4, 2016

pwittrock assigned pwittrock and unassigned janetkuo Oct 14, 2016

pwittrock closed this Oct 14, 2016

Template Api Support #25293

Template Api Support #25293

Conversation

pwittrock commented May 7, 2016 • edited by k8s-oncall

pwittrock commented May 7, 2016

pwittrock commented May 7, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bparees commented May 9, 2016

0xmichalis commented May 9, 2016

pwittrock commented May 9, 2016

bparees commented May 9, 2016

pwittrock commented May 9, 2016

bparees commented May 9, 2016

lavalamp commented May 9, 2016

pwittrock commented May 9, 2016

lavalamp commented May 9, 2016

pwittrock commented May 9, 2016

pwittrock commented May 9, 2016

pwittrock commented May 17, 2016

pwittrock commented May 17, 2016

pwittrock commented May 17, 2016

pwittrock commented May 17, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bparees Aug 10, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pwittrock commented May 17, 2016

k8s-github-robot commented May 18, 2016

ncdc commented Jul 12, 2016

pwittrock commented Jul 14, 2016

smarterclayton commented Jul 14, 2016

pwittrock commented Jul 18, 2016

k8s-bot commented Aug 18, 2016

k8s-bot commented Aug 30, 2016

k8s-github-robot commented Oct 9, 2016

pwittrock commented May 7, 2016 •

edited by k8s-oncall

bparees Aug 10, 2016 •

edited