Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS: More ELB attributes via service annotations #30695

Merged
merged 1 commit into from Aug 22, 2016

Conversation

@krancour
Copy link
Member

@krancour krancour commented Aug 16, 2016

Replaces #25015 and addresses all of @justinsb's feedback therein. This is a new PR because I was unable to reopen #25015 to amend it.

I noticed recently that there is existing (but undocumented) precedent for the AWS cloud provider to manage ELB-specifc load balancer configuration based on service annotations. In particular, one can already designate an ELB as "internal" or enable PROXY protocol.

This PR extends this capability to the management of ELB attributes, which includes the following items:

  • Access logs:
    • Enabled / disabled
    • Emit interval
    • S3 bucket name
    • S3 bucket prefix
  • Connection draining:
    • Enabled / disabled
    • Timeout
  • Connection:
    • Idle timeout
  • Cross-zone load balancing:
    • Enabled / disabled

Some of these are possibly more useful than others. Use cases that immediately come to mind:

  • Enabling cross-zone load balancing is potentially useful for "Ubernetes Light," or anyone otherwise attempting to spread worker nodes around multiple AZs.
  • Increasing idle timeout is useful for the benefit of anyone dealing with long-running requests. An example I personally care about would be git pushes to Deis' builder component.

This change is Reviewable

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 16, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

Loading

6 similar comments
@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 16, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

Loading

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 16, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

Loading

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 16, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

Loading

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 16, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

Loading

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 16, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

Loading

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 16, 2016

Can one of the admins verify that this patch is reasonable to test? If so, please reply "ok to test".
(Note: "add to whitelist" is no longer supported. Please update configurations in kubernetes/test-infra/jenkins/job-configs/kubernetes-jenkins-pull instead.)

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

Loading

@krancour
Copy link
Member Author

@krancour krancour commented Aug 19, 2016

@justinsb, could you possibly mark this as "ok to test?"

Loading

foundAttributes := &describeAttributesOutput.LoadBalancerAttributes

// Update attributes if they're dirty
if !reflect.DeepEqual(loadBalancerAttributes, foundAttributes) {
Copy link
Member

@justinsb justinsb Aug 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nice to log here if they are unequal, just in case reflect.DeepEqual has false positives

Loading

@justinsb
Copy link
Member

@justinsb justinsb commented Aug 19, 2016

ok to test

Loading

@justinsb
Copy link
Member

@justinsb justinsb commented Aug 19, 2016

lgtm

Loading

@justinsb
Copy link
Member

@justinsb justinsb commented Aug 19, 2016

Thanks @krancour this looks great. If you have time it would be great to add a glog when updating attributes, but not a blocker for merge.

Loading

@justinsb justinsb changed the title Add support for managing ELB attributes with service annotations AWS: More ELB attributes via service annotations Aug 19, 2016
@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 19, 2016

GCE e2e build/test passed for commit 96dad1f.

Loading

@krancour
Copy link
Member Author

@krancour krancour commented Aug 20, 2016

@justinsb... I will first thing Monday.

Loading

@justinsb
Copy link
Member

@justinsb justinsb commented Aug 22, 2016

@krancour thanks - ping me when you've done it and I'll LGTM. Also if you don't have time I would like to still get this into 1.4, so just LMK. Adding the logging can be done post feature-freeze (today!).

Loading

@krancour
Copy link
Member Author

@krancour krancour commented Aug 22, 2016

@justinsb if feature freeze is today, I'd say we're better off putting that last log-line in ex post facto-- just given what I have on my plate this morning. Happy to hear this can make it into 1.4!

Loading

@justinsb
Copy link
Member

@justinsb justinsb commented Aug 22, 2016

Awesome @krancour . Opened #31127 to track the logging. This LGTM

Loading

@justinsb justinsb added the lgtm label Aug 22, 2016
@justinsb justinsb added this to the v1.4 milestone Aug 22, 2016
@k8s-github-robot
Copy link
Contributor

@k8s-github-robot k8s-github-robot commented Aug 22, 2016

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

Loading

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Aug 22, 2016

GCE e2e build/test passed for commit 96dad1f.

Loading

@k8s-github-robot
Copy link
Contributor

@k8s-github-robot k8s-github-robot commented Aug 22, 2016

Automatic merge from submit-queue

Loading

@k8s-github-robot k8s-github-robot merged commit bfafb6f into kubernetes:master Aug 22, 2016
6 of 7 checks passed
Loading
@therc
Copy link
Member

@therc therc commented Sep 24, 2016

Are these documented anywhere? I'll be the first to confess that the only user-visible documentation for my annotations are buried deep in user-guide/services/ ("SSL support on AWS"). We should find a home for them all.

Loading

@krancour krancour deleted the manage-elb-attributes branch Sep 27, 2016
felixbuenemann pushed a commit to felixbuenemann/workflow that referenced this issue Feb 25, 2017
This updates the docs to describe how to persist the AWS ELB idle timeout by using the proper k8s service annotation instead of following the manual instructions, which get reset if k8s re-configures the ELB.

The annotation was added in kubernetes/kubernetes#30695 and merged targeting k8s v1.4 in August 2016. I have verified that it works as expected on k8s v1.4.6.
bacongobbler pushed a commit to bacongobbler/workflow that referenced this issue Feb 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants