New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Federation: Support deploying federation control plane on non-cloud environment #40536

Closed
shashidharatd opened this Issue Jan 26, 2017 · 19 comments

Comments

@shashidharatd
Member

shashidharatd commented Jan 26, 2017

There may be scenarios where there is a need to federate k8s clusters on private-cloud (on-premise) environments.
Listed below are some of the tasks to achieve the scenario above

  • Federation control plane deployment itself should be able to be deployed in non-cloud environments. #40516
  • Should be possible to disable persistent storage for etcd. #40862
  • Ability to configure desired DNS provider. Public DNS servers (AWS Route53/ Google CloudDNS) or Private DNS servers (e.g. CoreDNS) #40528
  • Document Federation CoreDNS support. kubernetes/website#2810
  • coredns federation provider should support etcd with TLS #47049
  • Document using hosted DNS providers (AWS Route53/ Google CloudDNS) in non-cloud environments.
@shashidharatd

This comment has been minimized.

Show comment
Hide comment
@shashidharatd

shashidharatd Jan 26, 2017

Member

@marun, Please feel free to correct or add any additional points to this issue.

Member

shashidharatd commented Jan 26, 2017

@marun, Please feel free to correct or add any additional points to this issue.

@shashidharatd

This comment has been minimized.

Show comment
Hide comment
@shashidharatd

shashidharatd Jan 26, 2017

Member

This issue will partly address the deployment part of federation on non-google clouds as described in #39870

Member

shashidharatd commented Jan 26, 2017

This issue will partly address the deployment part of federation on non-google clouds as described in #39870

@marun

This comment has been minimized.

Show comment
Hide comment
@marun

marun Jan 26, 2017

Member

I think this issue can serve as an umbrella to track issues targeting improved support for non-cloud deployments with kubefed. Rather than conflating the work involved, work items can be tracked and discussed separately:

Member

marun commented Jan 26, 2017

I think this issue can serve as an umbrella to track issues targeting improved support for non-cloud deployments with kubefed. Rather than conflating the work involved, work items can be tracked and discussed separately:

@marun

This comment has been minimized.

Show comment
Hide comment
@marun

marun Jan 26, 2017

Member

cc: @madhusudancs @kubernetes/sig-federation-misc

Member

marun commented Jan 26, 2017

cc: @madhusudancs @kubernetes/sig-federation-misc

@mattymo

This comment has been minimized.

Show comment
Hide comment
@mattymo

mattymo Mar 17, 2017

#42995 should be considered for the list

mattymo commented Mar 17, 2017

#42995 should be considered for the list

@shashidharatd

This comment has been minimized.

Show comment
Hide comment
@shashidharatd

shashidharatd May 3, 2017

Member

/assign

Member

shashidharatd commented May 3, 2017

/assign

@liqlin2015

This comment has been minimized.

Show comment
Hide comment
@liqlin2015

liqlin2015 Aug 7, 2017

Is there any way we can use a local service load balancer for federation service?

And is there any plan to support services only with external IP? Currently in the dns.go:wantsDNSRecords() function, we hard coded to only support v1.ServiceTypeLoadBalancer type of services. Can we use external IP as another option for on-prem clusters?

liqlin2015 commented Aug 7, 2017

Is there any way we can use a local service load balancer for federation service?

And is there any plan to support services only with external IP? Currently in the dns.go:wantsDNSRecords() function, we hard coded to only support v1.ServiceTypeLoadBalancer type of services. Can we use external IP as another option for on-prem clusters?

@shashidharatd

This comment has been minimized.

Show comment
Hide comment
@shashidharatd

shashidharatd Aug 7, 2017

Member

what is the meaning of local service load balancer.

Currently only loadbalancer type services are supported for federated services. Currently there is no other option available as we know, which works.

Member

shashidharatd commented Aug 7, 2017

what is the meaning of local service load balancer.

Currently only loadbalancer type services are supported for federated services. Currently there is no other option available as we know, which works.

@luckyfengyong

This comment has been minimized.

Show comment
Hide comment
@luckyfengyong

luckyfengyong Aug 7, 2017

@shashidharatd In the doc of federation service discovery, it is mentioned as follows

image

After checking the codes we found only when a service is created with the type of Loadbalancer, the DNS record of the service is created in federation dns service. Just wondering what if the service is created with exnternalip. For that case, can we also ask federation dns service to create the DNS record?

Basically our use case is just like what this ticket is described to run cluster federation within on-prem environment. However we find there is no out-of-box external loadbalancer service available.

Therefore we will need either ask dns server of cluster federation supporting the service with externalip, or have a out-of-box loadbalancer service (just like coredns as out-of-box dns server of cluster federation)

Any suggestion?

luckyfengyong commented Aug 7, 2017

@shashidharatd In the doc of federation service discovery, it is mentioned as follows

image

After checking the codes we found only when a service is created with the type of Loadbalancer, the DNS record of the service is created in federation dns service. Just wondering what if the service is created with exnternalip. For that case, can we also ask federation dns service to create the DNS record?

Basically our use case is just like what this ticket is described to run cluster federation within on-prem environment. However we find there is no out-of-box external loadbalancer service available.

Therefore we will need either ask dns server of cluster federation supporting the service with externalip, or have a out-of-box loadbalancer service (just like coredns as out-of-box dns server of cluster federation)

Any suggestion?

@quinton-hoole

This comment has been minimized.

Show comment
Hide comment
@quinton-hoole

quinton-hoole Sep 8, 2017

Member

Required for GA of federation.

Member

quinton-hoole commented Sep 8, 2017

Required for GA of federation.

@luxas

This comment has been minimized.

Show comment
Hide comment
@luxas

luxas Sep 8, 2017

Member

@quinton-hoole Any relevant design doc/proposal for this issue/feature request?

Member

luxas commented Sep 8, 2017

@quinton-hoole Any relevant design doc/proposal for this issue/feature request?

@quinton-hoole

This comment has been minimized.

Show comment
Hide comment
@quinton-hoole

quinton-hoole Sep 14, 2017

Member

@luxas Nothing beyond what it linked in this issue, that I am aware of.

Member

quinton-hoole commented Sep 14, 2017

@luxas Nothing beyond what it linked in this issue, that I am aware of.

@shashidharatd

This comment has been minimized.

Show comment
Hide comment
@shashidharatd

shashidharatd Sep 27, 2017

Member

All the tasks we envisioned are complete and hence closing this issue. If anyone feels this issue needs to be kept open for additional tasks, please feel free to do so.

Member

shashidharatd commented Sep 27, 2017

All the tasks we envisioned are complete and hence closing this issue. If anyone feels this issue needs to be kept open for additional tasks, please feel free to do so.

@luxas

This comment has been minimized.

Show comment
Hide comment
@luxas

luxas Sep 27, 2017

Member

@shashidharatd Where is the documentation for this feature?

Member

luxas commented Sep 27, 2017

@shashidharatd Where is the documentation for this feature?

@shashidharatd

This comment has been minimized.

Show comment
Hide comment
@shashidharatd

shashidharatd Sep 27, 2017

Member

@luxas, here is the documentation for running federation with CoreDNS as dns provider. https://kubernetes.io/docs/tasks/federation/set-up-coredns-provider-federation.
In non-cloud environments, we could also use hosted DNS providers such as AWS Route53 or Google Cloud-DNS by specifying the dns-provider configuration as secret to kubefed init.

I agree there is no detailed document for user explaining how to use the hosted DNS providers (AWS Route53/Google Cloud-DNS). We shall add this document in the upcoming release.

Member

shashidharatd commented Sep 27, 2017

@luxas, here is the documentation for running federation with CoreDNS as dns provider. https://kubernetes.io/docs/tasks/federation/set-up-coredns-provider-federation.
In non-cloud environments, we could also use hosted DNS providers such as AWS Route53 or Google Cloud-DNS by specifying the dns-provider configuration as secret to kubefed init.

I agree there is no detailed document for user explaining how to use the hosted DNS providers (AWS Route53/Google Cloud-DNS). We shall add this document in the upcoming release.

@shashidharatd shashidharatd reopened this Sep 27, 2017

@shashidharatd

This comment has been minimized.

Show comment
Hide comment
@shashidharatd

shashidharatd Sep 27, 2017

Member

Added a new item to document explaining how to use the hosted DNS providers in non-cloud environments and so reopened this issue.

Member

shashidharatd commented Sep 27, 2017

Added a new item to document explaining how to use the hosted DNS providers in non-cloud environments and so reopened this issue.

@luxas

This comment has been minimized.

Show comment
Hide comment
@luxas

luxas Sep 27, 2017

Member

Thanks!

Member

luxas commented Sep 27, 2017

Thanks!

@pilsy

This comment has been minimized.

Show comment
Hide comment
@pilsy

pilsy commented Oct 15, 2017

Any answer to #40536 (comment) ?

@irfanurrehman

This comment has been minimized.

Show comment
Hide comment
@irfanurrehman

irfanurrehman Oct 31, 2017

Member

This issue was labelled only for sig/multicluster and is thus moved over to kubernetes/federation#101.
If this does not seem to be right, please reopen this and notify us @kubernetes/sig-multicluster-misc.
/close

Member

irfanurrehman commented Oct 31, 2017

This issue was labelled only for sig/multicluster and is thus moved over to kubernetes/federation#101.
If this does not seem to be right, please reopen this and notify us @kubernetes/sig-multicluster-misc.
/close

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment