Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support GlobalAccess for gce Internal Loadbalancers #81549

Merged
merged 2 commits into from Aug 30, 2019

Conversation

@prameshj
Copy link
Contributor

commented Aug 17, 2019

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:
This PR adds Support for GlobalAccess for gce Internal Loadbalancers.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Added support to specify a global-access annotation for gce ILB.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


Copy link
Contributor

left a comment

Thanks for your pr :)

Is there an issue or KEP discussing adding support for GlobalAccess for gce internal load balancers? This seems like a fairly large, user-facing change, and to be honest, I don't have a ton of context on what global access is or the ramifications of adding it.

@prameshj

This comment has been minimized.

Copy link
Contributor Author

commented Aug 17, 2019

Thanks for your pr :)

Is there an issue or KEP discussing adding support for GlobalAccess for gce internal load balancers? This seems like a fairly large, user-facing change, and to be honest, I don't have a ton of context on what global access is or the ramifications of adding it.

Thanks Matt. I just opened issue - #81555 and added some context.

GlobalAccess is an option GCE k8s users can pass in by adding the appropriate annotation. The code changes for this are in commit:
5c1dde7

The rest are all autogenerated vendor/go.mod changes.

@prameshj prameshj force-pushed the prameshj:ilb-globalaccess branch 3 times, most recently from 0ea98c1 to 3e905c3 Aug 18, 2019
@mattjmcnaughton

This comment has been minimized.

Copy link
Contributor

commented Aug 18, 2019

Thanks for your pr :)
Is there an issue or KEP discussing adding support for GlobalAccess for gce internal load balancers? This seems like a fairly large, user-facing change, and to be honest, I don't have a ton of context on what global access is or the ramifications of adding it.

Thanks Matt. I just opened issue - #81555 and added some context.

GlobalAccess is an option GCE k8s users can pass in by adding the appropriate annotation. The code changes for this are in commit:
5c1dde7

The rest are all autogenerated vendor/go.mod changes.

I wonder, would it be possible to make the vendor changes in a separate pr? I think it could make it a lot easier for reviewers to focus on the application logic changes.

@cblecker

This comment has been minimized.

Copy link
Member

commented Aug 18, 2019

@mattjmcnaughton Separate commit, sure. Not a separate PR. Vendor changes that are related to a feature should be in the same PR, so if it's reverted, all of it gets reverted.

@prameshj

This comment has been minimized.

Copy link
Contributor Author

commented Aug 19, 2019

Matt McNaughton Separate commit, sure. Not a separate PR. Vendor changes that are related to a feature should be in the same PR, so if it's reverted, all of it gets reverted.

Yes, the changes are in a separate commit currently. I put the vendor changes in the same PR so that the build/tests will succeed too.

@prameshj prameshj force-pushed the prameshj:ilb-globalaccess branch from 3e905c3 to e9b4cb5 Aug 19, 2019
@mattjmcnaughton

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2019

@prameshj prameshj force-pushed the prameshj:ilb-globalaccess branch from 91df213 to 0290e91 Aug 29, 2019
@k8s-ci-robot k8s-ci-robot added size/XL and removed size/L labels Aug 29, 2019
@prameshj

This comment has been minimized.

Copy link
Contributor Author

commented Aug 29, 2019

/assign @bowei

@prameshj prameshj force-pushed the prameshj:ilb-globalaccess branch 2 times, most recently from 640abb8 to 78239bc Aug 29, 2019
@@ -152,7 +152,11 @@ func (g *Cloud) EnsureLoadBalancer(ctx context.Context, clusterName string, svc
default:
status, err = g.ensureExternalLoadBalancer(clusterName, clusterID, svc, existingFwdRule, nodes)
}
klog.V(4).Infof("EnsureLoadBalancer(%v, %v, %v, %v, %v): done ensuring loadbalancer. err: %v", clusterName, svc.Namespace, svc.Name, loadBalancerName, g.region, err)
if err != nil {
klog.Errorf("Failed to EnsureLoadBalancer(%v, %v, %v, %v, %v), err: %v", clusterName, svc.Namespace, svc.Name, loadBalancerName, g.region, err)

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

Use the right type in the log statement
e.g. %s for string
%d for integers etc

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

done

klog.Errorf("Failed to EnsureLoadBalancer(%v, %v, %v, %v, %v), err: %v", clusterName, svc.Namespace, svc.Name, loadBalancerName, g.region, err)
return status, err
}
klog.V(4).Infof("EnsureLoadBalancer(%v, %v, %v, %v, %v): done ensuring loadbalancer.", clusterName, svc.Namespace, svc.Name, loadBalancerName, g.region)

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

same here

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

done

}

func (f *forwardingRuleComposite) Equal(other *forwardingRuleComposite) bool {
// IPAddress being an output-only field, equality check ignored if one is empty.

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

if it is output only, shouldn't we just omit from equality?

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

Actually the comment is not correct. We do hold the ip address based on the subnet/user-input. So user could specify a different IP and then it is an update. Removed the comment.

}
return frc, err
default:
return nil, fmt.Errorf("Invalid object type %v to compute ForwardingRuleComposite from", v)

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

type is %T

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

done

// ToForwardingRuleComposite converts a compute beta or GA ForwardingRule into the composite type
func ToForwardingRuleComposite(rule interface{}) (frc *forwardingRuleComposite, err error) {
switch v := rule.(type) {
case *compute.ForwardingRule:

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

use a type switch https://tour.golang.org/methods/16:

switch fr := rule.(type) {
  case *compute.ForwardingRule:
    ...
  case *computebeta.ForwardingRule:
    ...

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

This is already using type switch. Removed unnecessary typecast.

oldFrc, err = ToForwardingRuleComposite(existingFwdRule)
}
if err != nil {
return fmt.Errorf("Failed to fetch ForwardingRuleComposite from rule %s, err: %v", existingFwdRule.Name, err)

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

don't wrap the error

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

done

if err != nil {
return fmt.Errorf("Failed to fetch ForwardingRuleComposite from rule %s, err: %v", existingFwdRule.Name, err)
}
if oldFrc.Equal(newFrc) {

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

klog.V(4).Infof("oldFRC == newFRC, no updates needed (oldFRC == %+v)", oldFRC)

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

done

if oldFrc.Equal(newFrc) {
return nil
}
// delete forwarding rule

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

remove this comment

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

done

err = g.CreateRegionForwardingRule(gaRule, g.region)
}
klog.V(2).Infof("ensureInternalLoadBalancer(%v): created forwarding rule, err : %s", newFrc.name, err)
return nil

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

doesn't this have to be return err

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

good catch, fixed.

AllowGlobalAccess: true,
}
err = gce.CreateRegionForwardingRule(gaRule, gce.region)
assert.NoError(t, err)

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

don't use aserrtion, use

if err != nil {
  t.Errorf(...)
}

This comment has been minimized.

Copy link
@prameshj

prameshj Aug 29, 2019

Author Contributor

done

@prameshj prameshj force-pushed the prameshj:ilb-globalaccess branch 3 times, most recently from 75e44f3 to f0dc597 Aug 29, 2019
@bowei

This comment has been minimized.

Copy link
Member

commented Aug 29, 2019

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 29, 2019
@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bowei, prameshj

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

func (f *forwardingRuleComposite) ToGA() (*compute.ForwardingRule, error) {
descStr, err := f.description.marshal()
if err != nil {
return nil, fmt.Errorf("Failed to compute description for beta forwarding rule %s, err: %v", f.name, err)

This comment has been minimized.

Copy link
@bowei

bowei Aug 29, 2019

Member

beta => GA

Uses beta api when global access is enabled and ga api otherwise
Deprecate the older load-balancer-type annotation

Unit test to verify global access
Also added tests to verify new and old load-balancer-type annotations

Addressed review comments, staticcheck fixes

fixed typo
@prameshj prameshj force-pushed the prameshj:ilb-globalaccess branch from f0dc597 to c09c8ab Aug 29, 2019
@k8s-ci-robot k8s-ci-robot removed the lgtm label Aug 29, 2019
@freehan

This comment has been minimized.

Copy link
Member

commented Aug 29, 2019

/lgtm

@bowei

This comment has been minimized.

Copy link
Member

commented Aug 29, 2019

/lgtm

exported func ToForwardingRuleComposite returns unexported type *gce.forwardingRuleComposite, which can be annoying to use
@k8s-ci-robot k8s-ci-robot removed the lgtm label Aug 30, 2019
@MrHohn
MrHohn approved these changes Aug 30, 2019
Copy link
Member

left a comment

LGTM for the golint fix.
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 30, 2019
@k8s-ci-robot k8s-ci-robot merged commit a7e9704 into kubernetes:master Aug 30, 2019
24 checks passed
24 checks passed
cla/linuxfoundation prameshj authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-image-test Skipped.
pull-kubernetes-conformance-kind-ipv6 Skipped.
pull-kubernetes-cross Skipped.
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-csi-serial Skipped.
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gce-iscsi Skipped.
pull-kubernetes-e2e-gce-iscsi-serial Skipped.
pull-kubernetes-e2e-gce-storage-slow Skipped.
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-node-e2e-containerd Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details
@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Aug 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.