New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubernetes: Config files + setup script for secure multiregion clusters #27092

Merged
merged 1 commit into from Aug 14, 2018
Commits on Aug 14, 2018
  1. kubernetes: Config files + setup script for secure multiregion clusters

    a-robinson committed Jul 1, 2018
    tl;dr fill in a few constants at the top of secure.py and then run it,
    and you'll have a working secure multiregion cluster. Works on GKE. Does
    not work on AWS.
    
    This relies on linking together each cluster's DNS servers so that
    they're able to defer requests to each other for a special zone-scoped
    namespace. This is a little hacky, but very maintainable and survivable
    even if all the nodes in a cluster are taken down and brought back up
    again later with different IP addresses.
    
    The big caveat that I learned after I thought I was done, though, is
    that GCE's internal load balancers only work within a region.
    Unfortunately I had done my prototyping all within one region, with each
    cluster just in a different zone. To get around this, I've had to switch
    from exposing each DNS server on an internal IP to exposing them on
    external IPs, which some users may not like, and which might tip the
    scales in favor of a different solution. I'd be happy to discuss the
    alternatives (hooking up a CoreDNS instance in each cluster to every
    cluster's apiserver, or using istio multicluster) with anyone
    interested.
    
    I've only handled the secure cluster case in the script, but it can be
    easily modified to also handle insecure clusters as an option. Insecure
    clusters can actually run in more environments than secure clusters,
    though, and should probably be handled differently for that reason.
    
    Release note (general change): Configuration files and a setup script
    for secure multiregion deployments on GKE are now provided.
    
    Release note: None