AWS: remove volume IOPS limit

[jacobmarble]

What type of PR is this?

/kind bug

What this PR does / why we need it:

Max IOPS for SSD (io1) volumes was increased from 20,000 to 32,000:
https://aws.amazon.com/about-aws/whats-new/2017/12/amazon-ebs-provisioned-iops-ssd--io1--volumes-now-support-32-000-iops-and-500-mbs-per-volume/

and later to 64,000:
https://aws.amazon.com/about-aws/whats-new/2018/11/amazon-elastic-block-store-announces-double-the-performance-of-provisioned-iops-volumes/

Rather than chasing this limit, allow the AWS SDK to surface errors for IOPS value out of range.

Which issue(s) this PR fixes:

(none found)

Special notes for your reviewer:

I'm proposing the same for CSI:
kubernetes-sigs/aws-ebs-csi-driver#483

Does this PR introduce a user-facing change?:

Increased maximum IOPS of AWS EBS io1 volumes to 64,000 (current AWS maximum).

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
• Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-ci-robot]

Welcome @jacobmarble!

It looks like this is your first PR to kubernetes/kubernetes 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/kubernetes has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @jacobmarble. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jacobmarble]

I have signed the CLA. Let's see if this message triggers a recheck.

[jacobmarble]

I have manually tested this change. A PV/PVC based on io1 with <64,000 IOPS created successfully. A PV/PVC based on io1 with >64,000 IOPS did not. The errors exposed in logs and kubectl described are short, clear, useful.

Resource config:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-io1-1000g-50000iops
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1000G
  storageClassName: io1-50
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-io1-1500g-75000iops
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1500G
  storageClassName: io1-50
---
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: io1-50
parameters:
  iopsPerGB: "50"
  type: io1
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Delete
volumeBindingMode: Immediate

Results of kubectl describe on failing PVC:

Name:          data-io1-1500g-75000iops
Namespace:     default
StorageClass:  io1-50
Status:        Pending
Volume:
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Mounted By:    <none>
Events:
  Type     Reason              Age   From                         Message
  ----     ------              ----  ----                         -------
  Warning  ProvisioningFailed  89s   persistentvolume-controller  Failed to provision volume with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
           status code: 400, request id: 20c19f61-7ceb-4275-baf3-a51dd9dd067d
  Warning  ProvisioningFailed  86s  persistentvolume-controller  Failed to provision volume with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
           status code: 400, request id: 2abbb3e3-ffa3-4815-bfb1-cd4c410455c2
  Warning  ProvisioningFailed  71s  persistentvolume-controller  Failed to provision volume with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
           status code: 400, request id: 666f9642-e454-4c3d-b9a9-f347223ce033
  Warning  ProvisioningFailed  56s  persistentvolume-controller  Failed to provision volume with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
           status code: 400, request id: c7bcee49-d5c4-4015-b32a-c5fd409fcfba
  Warning  ProvisioningFailed  41s  persistentvolume-controller  Failed to provision volume with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
           status code: 400, request id: e0e5d299-014c-4c60-ab50-524283b24679
  Warning  ProvisioningFailed  26s  persistentvolume-controller  Failed to provision volume with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
           status code: 400, request id: b1bf92e1-76b1-4a7e-a589-418d470f1b97

Kubernetes log:

I0430 05:58:31.041343   15843 aws_util.go:110] Error creating EBS Disk volume: InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
	status code: 400, request id: 6ccfc493-5141-454d-a578-04e659708067
E0430 05:58:31.041377   15843 aws_ebs.go:494] Provision failed: InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
	status code: 400, request id: 6ccfc493-5141-454d-a578-04e659708067
I0430 05:58:31.041415   15843 pv_controller.go:1499] failed to provision volume for claim "default/data-io1-1500g-75000iops" with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
	status code: 400, request id: 6ccfc493-5141-454d-a578-04e659708067
E0430 05:58:31.041467   15843 goroutinemap.go:150] Operation for "provision-default/data-io1-1500g-75000iops[5bab79c7-7517-4a15-a6af-485d2ceb55e7]" failed. No retries permitted until 2020-04-30 05:59:35.041448037 +0000 UTC m=+3039.192762601 (durationBeforeRetry 1m4s). Error: "InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.\n\tstatus code: 400, request id: 6ccfc493-5141-454d-a578-04e659708067"
I0430 05:58:31.041528   15843 event.go:278] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"data-io1-1500g-75000iops", UID:"5bab79c7-7517-4a15-a6af-485d2ceb55e7", APIVersion:"v1", ResourceVersion:"934", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' Failed to provision volume with StorageClass "io1-50": InvalidParameterValue: Volume iops of 69850 is too high; maximum is 64000.
	status code: 400, request id: 6ccfc493-5141-454d-a578-04e659708067
I0430 05:58:31.232585   15843 pvc_protection_controller.go:263] PVC default/data-io1-1500g-75000iops is unused
I0430 05:58:31.235708   15843 pvc_protection_controller.go:209] Removed protection finalizer from PVC default/data-io1-1500g-75000iops

@gnufied @justinsb please share your thoughts.

[leakingtapan]

/ok-to-test

[leakingtapan]

/lgtm

[jacobmarble]

Matching PR for AWS/EBS CSI driver has also been tested.

Please advise next steps.

[jacobmarble]

Friendly note here. I believe that this and kubernetes-sigs/aws-ebs-csi-driver#483 are ready for consideration to be merged.

[leakingtapan]

/lgtm
/approve

@gnufied mind approve as well?

[nikhita]

/assign @jsafrane
for approval

[philjb]

/retest

[leakingtapan]

/lgtm

[jsafrane]

Thanks a lot!
/approve

[jsafrane]

/retest

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jacobmarble, jsafrane, leakingtapan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/volume/awsebs/OWNERS [jsafrane]
• staging/src/k8s.io/legacy-cloud-providers/aws/OWNERS [jsafrane]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

[philjb]

@jsafrane - could you add the milestone?

[jsafrane]

No. @saad-ali, can you please add 1.19 milestone? This one is harmless.

[philjb]

@saad-ali - just following up here on adding the milestone. thanks for your help.

[philjb]

@saad-ali - friendly weekly reminder here for the milestone.

[philjb]

@jsafrane - could you help find @saad-ali ?

[saad-ali]

/assign

[saad-ali]

/milestone v1.19

[justaugustus]

/test pull-kubernetes-e2e-kind-ipv6

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.