Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Azure] Cross resource group nodes #604

Closed
feiskyer opened this issue Jul 31, 2018 · 43 comments
Closed

[Azure] Cross resource group nodes #604

feiskyer opened this issue Jul 31, 2018 · 43 comments

Comments

@feiskyer
Copy link
Member

@feiskyer feiskyer commented Jul 31, 2018

Feature Description

  • One-line feature description (can be used as a release note): Add support of cross resource group nodes.
  • Primary contact (assignee): @feiskyer
  • Responsible SIGs: sig-azure
  • Design proposal link (community repo): kubernetes/community#2479
  • KEP: 20180809-cross-resource-group-nodes
  • Link to e2e and/or unit tests:
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @khenidak
  • Approver (likely from SIG/area to which feature belongs): @brendandburns
  • Feature target (which target equals to which milestone):
    • Alpha release target (1.12)
    • Beta release target (1.15)
    • Stable release target (x.y)
@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Jul 31, 2018

/sig azure
/milestone v1.12

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Jul 31, 2018

/kind feature
/stage alpha

@feiskyer feiskyer self-assigned this Jul 31, 2018
@feiskyer feiskyer added this to the v1.12 milestone Jul 31, 2018
@kacole2

This comment has been minimized.

Copy link
Member

@kacole2 kacole2 commented Jul 31, 2018

@feiskyer thank you. this has been added to the 1.12 Feature Tracking sheet. Please keep the post updated with links to PRs and design docs proposals.

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Aug 6, 2018

Community PR created kubernetes/community#2479

@zparnold

This comment has been minimized.

Copy link
Member

@zparnold zparnold commented Aug 20, 2018

Hey there! @feiskyer I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Aug 21, 2018

@zparnold We will update docs in repo https://github.com/kubernetes/cloud-provider-azure. Should it also be placeholded?

k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Aug 23, 2018
Kubernetes Submit Queue
Automatic merge from submit-queue (batch tested with PRs 66980, 67604, 67741, 67715). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add support of Azure cross resource group nodes

**What this PR does / why we need it**:

Part of feature [Cross resource group nodes](kubernetes/enhancements#604).

This PR adds support of Azure cross resource group nodes that are labeled with `kubernetes.azure.com/resource-group=<rg-name>` and `alpha.service-controller.kubernetes.io/exclude-balancer=true`

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

See designs [here](kubernetes/community#2479).

**Release note**:

```release-note
Azure cloud provider now supports cross resource group nodes that are labeled with `kubernetes.azure.com/resource-group=<rg-name>` and `alpha.service-controller.kubernetes.io/exclude-balancer=true`
```

/sig azure
/kind feature
@zparnold

This comment has been minimized.

Copy link
Member

@zparnold zparnold commented Aug 25, 2018

@justaugustus

This comment has been minimized.

Copy link
Member

@justaugustus justaugustus commented Aug 29, 2018

@feiskyer -- it would be nice to have a blurb about configuring this on kubernetes.io

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Aug 29, 2018

@justaugustus Which folder do you suggest?

k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Aug 30, 2018
Kubernetes Submit Queue
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Add support for unmanaged nodes for Azure cloud provider

**What this PR does / why we need it**:

Continue of [Azure cross resource groups feature](kubernetes/enhancements#604).

This PR adds support for unmanaged nodes (such as on-prem or on other clouds) that are labeled with `alpha.service-controller.kubernetes.io/exclude-balancer=true` and `kubernetes.azure.com/managed=false`. Azure cloud provider would exclude such nodes from LoadBalancer backends and always assumes they are existing.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

See KEP [here](https://github.com/kubernetes/community/blob/master/keps/sig-azure/0025-20180809-cross-resource-group-nodes.md).

**Special notes for your reviewer**:

Azure cloud provider won't provision network routes for on-prem nodes, so cluster admins should ensure the network (including pod-to-pod, pod-to-node and node-to-node connectivity) has been set up properly.

**Release note**:

```release-note
Azure cloud provider now supports unmanaged nodes (such as on-prem) that are labeled with `kubernetes.azure.com/managed=false` and `alpha.service-controller.kubernetes.io/exclude-balancer=true`
```

/assign @khenidak @andyzhangx
/sig azure
/kind feature
/milestone v1.12
@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Sep 6, 2018

@justaugustus Yep, codes have been merged now and docs will be added to https://github.com/kubernetes/cloud-provider-azure

@justaugustus

This comment has been minimized.

Copy link
Member

@justaugustus justaugustus commented Sep 6, 2018

Thanks for the update!

@ameukam

This comment has been minimized.

Copy link
Contributor

@ameukam ameukam commented Oct 5, 2018

Hi folks,
Kubernetes 1.13 is going to be a 'stable' release since the cycle is only 10 weeks. We encourage no big alpha features and only consider adding this feature if you have a high level of confidence it will make code slush by 11/09. Are there plans for this enhancement to graduate to alpha/beta/stable within the 1.13 release cycle? If not, can you please remove it from the 1.12 milestone or add it to 1.13?

We are also now encouraging that every new enhancement aligns with a KEP. If a KEP has been created, please link to it in the original post. Please take the opportunity to develop a KEP.

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Oct 5, 2018

It's already part of KEP, kubernetes/community#2479. Let's keep it in v1.13.

/milestone v1.13

@k8s-ci-robot k8s-ci-robot modified the milestones: v1.12, v1.13 Oct 5, 2018
@kacole2

This comment has been minimized.

Copy link
Member

@kacole2 kacole2 commented Oct 5, 2018

@feiskyer can you please update the original post with milestone of beta for 1.13? Thanks!
/stage beta

@kacole2

This comment has been minimized.

Copy link
Member

@kacole2 kacole2 commented Apr 29, 2019

@feiskyer, we're doing a KEP review for enhancements to be included in the Kubernetes v1.15 milestone. After reviewing your KEP, it's currently missing test plans and graduation criteria which is required information per the KEP Template. Please update the KEP to include the required information before the Kubernetes 1.15 Enhancement Freeze date of 4/30/2019. Thank you

@kacole2

This comment has been minimized.

Copy link
Member

@kacole2 kacole2 commented May 1, 2019

@feiskyer, Enhancement Freeze for Kubernetes 1.15 has passed and this did not meet the deadline. Still missing Test Plans and Graduation Criteria from the KEP. This is now being removed from the 1.15 milestone and the tracking sheet. If there is a need for this to be in 1.15, please file an Enhancement Exception. Thank you. @claurence

/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.15 milestone May 1, 2019
@kacole2 kacole2 added tracked/no and removed tracked/yes labels May 1, 2019
@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Jul 5, 2019

/milestone v1.16

@rbitia

This comment has been minimized.

Copy link

@rbitia rbitia commented Jul 11, 2019

Hi @feiskyer , I'm a 1.16 Enhancement Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet. If it's not graduating, I will remove it from the milestone and change the tracked label.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thanks!

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Jul 11, 2019

@rbitia Yep, already added to 1.16 milestone

@rbitia

This comment has been minimized.

Copy link

@rbitia rbitia commented Jul 15, 2019

Are you planning to graduate to stable or beta?

@justaugustus

This comment has been minimized.

Copy link
Member

@justaugustus justaugustus commented Jul 15, 2019

Hey @rbitia! We're going GA/Stable with this in 1.16. :)

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Jul 30, 2019

Yep, we're working on the e2e tests for this.

@VineethReddy02

This comment has been minimized.

Copy link

@VineethReddy02 VineethReddy02 commented Aug 1, 2019

Hi @feiskyer, I'm the v1.16 docs release shadow.

Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.16) due by Friday,August 23rd. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

Thanks!

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Aug 2, 2019

@VineethReddy02 Thanks, no new docs are required.

@kacole2

This comment has been minimized.

Copy link
Member

@kacole2 kacole2 commented Aug 5, 2019

@feiskyer urgent request can you please update the KEP with Graduation Criteria. This is a requirement for being included in the current 1.16 milestone. Please respond back with the PR ASAP. Thank you.

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Aug 6, 2019

@kacole2 thanks, would do

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Aug 6, 2019

Update: PR sent at #1202

@feiskyer

This comment has been minimized.

Copy link
Member Author

@feiskyer feiskyer commented Aug 9, 2019

Documents have been added at https://github.com/kubernetes/cloud-provider-azure/blob/master/docs/using-cross-resource-group-nodes.md.

E2e tests have been added at kubernetes-sigs/cloud-provider-azure#205.

Because aks-engine doesn't support to provision cross-resource-group nodes, the e2e tests couldn't be run periodically yet. But I think aks-engine should not be the blocker for this feature.

/close

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

@k8s-ci-robot k8s-ci-robot commented Aug 9, 2019

@feiskyer: Closing this issue.

In response to this:

Documents have been added at https://github.com/kubernetes/cloud-provider-azure/blob/master/docs/using-cross-resource-group-nodes.md.

E2e tests have been added at kubernetes-sigs/cloud-provider-azure#205.

Because aks-engine doesn't support to provision cross-resource-group nodes, the e2e tests couldn't be run periodically yet. But I think aks-engine should not be the blocker for this feature.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Provider Azure automation moved this from KEPs to Done Aug 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.