Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests are failing because we are running out of PD quota in us-east1 #618

Closed
jlewi opened this issue Apr 9, 2018 · 7 comments
Closed

Tests are failing because we are running out of PD quota in us-east1 #618

jlewi opened this issue Apr 9, 2018 · 7 comments
Assignees
@jlewi
Labels
area/testing priority/p1

Comments

@jlewi
Copy link
Contributor

jlewi commented Apr 9, 2018

Post submits show failures deploying Kubeflow on GKE
https://k8s-testgrid.appspot.com/sig-big-data#kubeflow-postsubmit

Presubmits having failures with minikube
#613

Documentation PR in which minikube failed.
@jlewi
Copy link
Contributor Author

jlewi commented Apr 9, 2018

/assign jlewi

@jlewi
Copy link
Contributor Author

jlewi commented Apr 9, 2018 

gcloud --project=kubeflow-ci compute regions list
NAME                     CPUS    DISKS_GB     ADDRESSES  RESERVED_ADDRESSES  STATUS  TURNDOWN_DATE
asia-east1               0/24    0/4096       0/8        0/8                 UP
asia-northeast1          0/24    0/4096       0/8        0/8                 UP
asia-south1              0/24    0/4096       0/8        0/8                 UP
asia-southeast1          0/24    0/4096       0/8        0/8                 UP
australia-southeast1     0/24    0/4096       0/8        0/8                 UP
europe-west1             0/24    0/4096       0/8        0/8                 UP
europe-west2             0/24    0/4096       0/8        0/8                 UP
europe-west3             0/24    0/4096       0/8        0/8                 UP
europe-west4             0/24    0/4096       0/8        0/8                 UP
northamerica-northeast1  0/24    0/4096       0/8        0/8                 UP
southamerica-east1       0/24    0/4096       0/8        0/8                 UP
us-central1              0/320   0/4096       0/32       0/8                 UP
us-east1                 72/320  16300/16384  9/32       0/8                 UP
us-east4                 0/24    0/4096       0/8        0/8                 UP
us-west1                 0/24    0/4096       0/8        0/8                 UP

@jlewi
Copy link
Contributor Author

jlewi commented Apr 9, 2018

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/kubeflow_kubeflow/613/kubeflow-presubmit/1072/
pod_id = "kubeflow-presubmit-kubeflow-e2e-minikube-613-82d0ce4-1072-42f9-745541553"
The test logs indicate we timed out waiting for the VM to be sshable

@jlewi
Copy link
Contributor Author

jlewi commented Apr 9, 2018

I looked at one of the gcloud insert operations

gcloud --project=kubeflow-ci compute operations describe --zone=us-east1-d operation-1523277349726-56969a5532731-18601d17-4611c4b6  
endTime: '2018-04-09T05:36:02.873-07:00'
error:
  errors:
  - code: QUOTA_EXCEEDED
    message: "Quota 'DISKS_TOTAL_GB' exceeded.  Limit: 16384.0 in region us-east1."
httpErrorMessage: FORBIDDEN
httpErrorStatusCode: 403
id: '7785415339107868873'
insertTime: '2018-04-09T05:35:50.996-07:00'
kind: compute#operation
name: operation-1523277349726-56969a5532731-18601d17-4611c4b6
operationType: insert
progress: 100
selfLink: https://www.googleapis.com/compute/v1/projects/kubeflow-ci/zones/us-east1-d/operations/operation-1523277349726-56969a5532731-18601d17-4611c4b6
startTime: '2018-04-09T05:35:51.939-07:00'
status: DONE
targetId: '4819522288420839626'
targetLink: https://www.googleapis.com/compute/v1/projects/kubeflow-ci/zones/us-east1-d/instances/zbeflow-presubmit-kubeflow-e2e-minikube-595-3da4f9f-1078-7581
user: kubeflow-testing@kubeflow-ci.iam.gserviceaccount.com
zone: https://www.googleapis.com/compute/v1/projects/kubeflow-ci/zones/us-east1-d

The insert is failing because we are out of PD.

It also looks like our minikube test isn't detect that the VM insert failed.

@jlewi jlewi changed the title Tests are failing Tests are failing because we are running out of PD quota in us-east1 Apr 9, 2018
@jlewi
Copy link
Contributor Author

jlewi commented Apr 9, 2018

It looks like we may be leaking the PDs for the minikube test.

gcloud --project=kubeflow-ci compute disks list
NAME                                                           ZONE        SIZE_GB  TYPE         STATUS
gke-kubeflow-testing-default-pool-90dc0402-bj6f                us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-default-pool-90dc0402-czdg                us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-default-pool-90dc0402-dzvl                us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-default-pool-90dc0402-x2pr                us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-gpu-pool-117658d4-40sn                    us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-gpu-pool-117658d4-qhsh                    us-east1-d  100      pd-standard  READY
gke-zor-postsubmit-tfjob-default-pool-74847b36-fprn            us-east1-d  100      pd-standard  READY
gke-zresubmit-tfjob-e2e--default-pool-67d961c7-gf3k            us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-0823a40-199-5cd0     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-087a3ae-208-b4e7     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-0bdb6eb-216-beaf     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-1615d75-204-73ee     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-1f854c4-214-2c9a     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-20dec55-201-71e3     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-36ce624-224-e754     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-3d5e099-225-cf9c     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-61240be-215-69da     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-7b20214-205-09e4     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-7e8e046-212-512b     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-87763e3-209-4991     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-8f8e57d-200-e4c6     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-95346d5-210-dee3     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-9765993-203-345c     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-af4a777-219-31a0     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-b679242-217-9aa1     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-be3729d-218-bb1a     us-east1-d  100      pd-standard  READY
kubeflow-postsubmit-kubeflow-e2e-minikube-d604f5a-226-0548     us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-105bec7-843-d3d9       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-299ac86-841-1b64       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-6c7ad78-847-6581       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-804f9d5-846-9e52       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-cd96b38-849-0e31       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-d050692-850-e030       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-d4112e8-851-906a       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-fb1bcf8-852-e804       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-00b1555-871-1481  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-105bec7-843-81cc  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-299ac86-841-169b  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-36b09cd-900-6a0d  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-3d05109-897-80f5  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-406c470-906-28f4  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-43fa1a5-858-33f1  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-52b0fd0-869-651a  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-565be88-898-20cc  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-6247f2a-904-8277  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-6c7ad78-847-97a2  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-7dcaf96-861-b623  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-7dcaf96-863-4d54  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-804f9d5-846-fb5c  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-8706f3f-888-4deb  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-892c6cb-909-2b8f  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-8f9458b-899-e9e2  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-9191c4c-896-897a  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-950c4b5-889-71a4  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-9b35e67-903-9925  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-9e98e2b-892-1be3  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-a4154e0-886-1170  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-aaf5eb8-870-f88c  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-b4d7831-908-7951  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-bd6305b-891-65c3  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-c627c05-881-4d39  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-cac42bb-901-93de  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-cac42bb-902-a0a0  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-cd96b38-849-11aa  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-d050692-850-260f  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-d4112e8-851-e88a  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-dd576ad-884-ebd4  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-defc9d9-905-0379  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-e2b4bb4-879-d1bd  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-e66307e-887-aab4  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-e67cdbd-859-afd4  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-f1e1c6d-868-2b4d  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-faf54af-910-1825  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-473-fb1bcf8-852-341f  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-16f5910-963-f484  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-3afa439-951-3a21  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-56335d5-911-83e8  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-652e5f9-950-5662  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-6d1f8d6-966-a393  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-6d1f8d6-972-777c  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-6d1f8d6-976-0870  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-523-d262a4a-965-8cca  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-532-0177fc4-980-b667  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-532-0177fc4-996-5ce2  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-542-546ad9b-989-84c3  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-558-86aca5f-982-8f41  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-558-86aca5f-993-3a80  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-558-86aca5f-995-0128  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-565-299f163-984-ffa9  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-565-ad0ba42-998-e634  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-567-14931a9-986-50cf  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-567-14931a9-987-431c  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-570-155b462-991-4d47  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-570-7c3d627-990-9e43  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-572-d8bec86-992-465e  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-572-d8bec86-994-e305  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-minikube-572-d8bec86-997-08d4  us-east1-d  100      pd-standard  READY
kubeflow-test-nfs-vm                                           us-east1-d  10       pd-ssd       READY
kubeflow-test-nfs-vm-data                                      us-east1-d  1000     pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-470-c1cac93-1034-c6c7  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-519-eaf5440-1050-23ef  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-519-eaf5440-1058-ca94  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-529-0feb3db-1021-489c  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-529-2c1959f-1041-f16a  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-529-42e14f6-1022-1489  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-529-5c91328-1040-debd  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-532-0177fc4-1019-8b43  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-552-e15744e-1028-2e26  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-558-cf8d89f-1001-1cf7  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-558-cf8d89f-1017-dcc2  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-558-fc89688-1030-0351  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-558-fc89688-1035-8958  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-558-fd51ba2-1036-f83a  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-565-ad0ba42-1013-cbf7  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-577-1978706-1005-9213  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-577-79a26bc-1003-64ff  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-577-a0a114a-1002-5abe  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-579-1d688ff-1004-01bd  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-579-8ffbf4e-1054-0f04  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-579-c95e242-1014-d353  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-580-8d4b6b2-1007-b885  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-580-8d4b6b2-1008-e430  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-581-4c69643-1009-0d0d  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-583-4c69643-1011-7a1f  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-583-4c69643-1016-2f3d  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-587-c1a1242-1023-e6c7  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-587-c1a1242-1026-43ab  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-588-3f1b165-1024-4651  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-588-3f1b165-1025-39d0  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-589-af7d319-1027-29d0  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-589-af7d319-1029-bccb  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-591-de68577-1032-5f46  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-593-c5ec7fd-1038-899d  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-595-1a165e8-1039-1b0a  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-595-2cac99d-1049-0060  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-595-a626fbd-1059-db22  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-595-c4ad9ca-1066-f544  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-597-01702fa-1045-5cfd  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-597-01702fa-1046-4058  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-597-15c8726-1053-540c  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-597-61a3ec0-1052-7d7d  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-599-feddf56-1047-3762  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-599-feddf56-1055-fa83  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-600-173be20-1048-876b  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-600-f55198c-1061-b163  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-601-b00ab07-1051-0385  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-602-a897a2d-1056-b67b  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-604-79fb79c-1057-ce88  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-605-30921b9-1067-1c8a  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-605-dc9d8b9-1060-a58e  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-608-70314c0-1063-fb23  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-608-727644d-1062-1812  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-609-30ddebb-1064-c008  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-610-4da3427-1065-d3ec  us-east1-d  100      pd-standard  READY
zbeflow-presubmit-kubeflow-e2e-minikube-610-b59f2bb-1068-efb0  us-east1-d  100      pd-standard  READY

@jlewi
Copy link
Contributor Author

jlewi commented Apr 9, 2018

The test should be configured to autodelete the PD
https://github.com/kubeflow/kubeflow/pull/523/files#diff-2552a6a0cb5a74e3184790464405bedcR404

@jlewi
Copy link
Contributor Author

jlewi commented Apr 9, 2018

Deleted all the old minikube disks

gcloud --project=kubeflow-ci compute disks list
NAME                                                      ZONE        SIZE_GB  TYPE         STATUS
gke-kubeflow-testing-default-pool-90dc0402-bj6f           us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-default-pool-90dc0402-czdg           us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-default-pool-90dc0402-dzvl           us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-default-pool-90dc0402-x2pr           us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-gpu-pool-117658d4-40sn               us-east1-d  100      pd-standard  READY
gke-kubeflow-testing-gpu-pool-117658d4-qhsh               us-east1-d  100      pd-standard  READY
gke-zor-postsubmit-tfjob-default-pool-74847b36-fprn       us-east1-d  100      pd-standard  READY
gke-zresubmit-tfjob-e2e--default-pool-67d961c7-gf3k       us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-105bec7-843-d3d9  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-299ac86-841-1b64  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-6c7ad78-847-6581  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-804f9d5-846-9e52  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-cd96b38-849-0e31  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-d050692-850-e030  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-d4112e8-851-906a  us-east1-d  100      pd-standard  READY
kubeflow-presubmit-kubeflow-e2e-gke-473-fb1bcf8-852-e804  us-east1-d  100      pd-standard  READY
kubeflow-test-nfs-vm                                      us-east1-d  10       pd-ssd       READY
kubeflow-test-nfs-vm-data                                 us-east1-d  1000     pd-standard  READY

@jlewi jlewi mentioned this issue Apr 9, 2018
Merged
@jlewi jlewi closed this as completed May 10, 2018
yanniszark pushed a commit to arrikto/kubeflow that referenced this issue Feb 15, 2021
@johnugeorge @k8s-ci-robot
go unit tests from presubmits (kubeflow#618)
8056907 
* Remove go unit tests from presubmits

* Minor fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jlewi jlewi

Labels
area/testing priority/p1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jlewi