TPRs and Custom Resource Definitions (CRDs)

[arschles]

CustomResourceDefinitions (CRDs) will eventually replace Third Party Resources (TPRs). This issue is for tracking that transition and the work we need to do to follow it.

Related issues:

• CustomResourceDefinitions kubernetes/enhancements#95

[kibbles-n-bytes]

Blog post about the change: https://coreos.com/blog/custom-resource-kubernetes-v17

[MHBauer]

kubernetes/kubernetes#48152 plan to remove in 1.8

[mengqiy]

Anyone working on this already? If not I will pick it up.

[nilebox]

kubernetes/kubernetes#48152 plan to remove in 1.8

So we have ~2-3 months to migrate to CRDs (or drop TPR support completely).
We also should consider what to do for kube 1.6 (as @arschles was working on keeping Service Catalog instructions for 1.6), where CRDs are not available.

[ash2k]

@mengqiy I think migration to CRDs is blocked by #944 because client is obviously is not available in old client-go.

[MHBauer]

It's the dynamic client, we don't require support from client go.

If it's so different, I would think we fork TPR and have support for etcd, tpr, and crd at the same time.

[nilebox]

@MHBauer I don't see the benefit of continuing the support of TPR once 1.8 is out. It will die very soon anyway, and Service Catalog is not even beta yet, why bother.

And there will still be support for etcd on Kubernetes 1.6, which seems fine to me. Let's just pretend that TPR was never supported and now we present the support of CRD 😉

[pmorie]

I think we should ditch TPR for CRD as soon as possible, and not support TPR at all.

…

On Thu, Jul 13, 2017 at 5:54 PM, Nail Islamov ***@***.***> wrote: @MHBauer <https://github.com/mhbauer> I don't see the benefit of continuing the support of TPR once 1.8 is out. It will die very soon anyway, and Service Catalog is not even beta yet, why bother. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#987 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWXmAduGfDgEj9aA-MRiUzVlXirzj2Cks5sNpIGgaJpZM4OF8R5> .

[nilebox]

I can take this task after #944, and I would prefer using a typed CRD client for this instead of dynamic one.

[MHBauer]

I'm fine with that. I wanted to make the argument for backwards compatibility.

[mengqiy]

@nilebox Can you list the tasks required for the transition from TPR to CRD.

[nilebox]

@mengqiy it's probably easier for @arschles to do, as he has added support for TPRs in Service Catalog.

[mengqiy]

Yes, the initial support for TPR seems to be in PR #338.

[arschles]

Here is some background on why we wanted a TPR storage backend in the first place. We (Deis) had some customers, and anticipated more, that did not want to connect the service-catalog software to the etcd servers they used with the core API server, and we didn't believe it to be acceptable to ask users to install etcd (i.e. the operator) on their cluster just to use service-catalog (we had experience doing this ourselves in the past, we had some problems, and we didn't want to support customers having the same problems).

So, we looked for a different solution with as few setup and maintenance requirements as possible. TPRs were available and had major issues when we started, but we managed to work around almost all of them and release service-catalog that used them successfully. This was worth it because there were no setup or maintenance requirements for TPRs as a datastore.

Now, there are actually a few reasons we need to re-evaluate:

1. They are being deprecated (what this issue is about)
2. Setting up RBAC rules to support the TPR backend requires additional work

(1) is, of course, enough to trigger this discussion, but (2) points to a larger problem: MSFT seems to be the only stakeholder in this group that wants a CRD/TPR based implementation. While we don't need to have CRD be a storage backend, I still believe that we should have an alternative to etcd.

I have scheduled us to talk about this in our design meeting on 7/24/2017.

[ash2k]

@arschles I think we also want CRD-based storage backed. It should be much easier to maintain vs an etcd.

p.s. I understand that CRDs are not meant to be used that way, but it is hopefully acceptable while there is no alternative.

[mengqiy]

@arschles @ash2k We (Google) are also interested in a CRD-based storage. If you guys are not working on that, I can pick it up.

[arschles]

@mengqiy we are going to talk about this on today's design meeting. I will update here

[arschles]

Here is what I proposed in the design meeting:

• Implement CRD support next to TPRs
• Document the fact that TPR is perpetually in alpha and won't be supported at all after we rebase on 1.8's API machinery
• After we rebase on 1.8, we'll tear out the TPR code

@nilebox @ash2k I will meet with you today. @mengqiy please ping me on the slack channel (arschles) so we can talk about this implementation.

[arschles]

After talking with @nilebox and @mengqiy about the work involved with migrating in this fashion to a CRD storage backend, we have settled on the following work items:

• Create Custom Resource Definitions on Startup #1087
• Implement Custom Resource Definitions storage backend #1088
• Remove TPR storage backend after Custom Resource Definitions are widely supported #1089

[MHBauer]

With the above work items, do we still need this issue?

[MHBauer]

TPR is out, so no transition to do, wholly replaced by #1088, #1087 can be a subtask of #1088

[mszostok]

done by: #2633

[MHBauer]

comment probably better placed on #1088 & #1087

[mszostok]

linked with mentioned issues :)