E2E kfctl go test should verify that Kubeflow is deployed. #2795
Conversation
…oyed. * Related to kubeflow#2610 E2E test for go binary. * Create kf_is_ready_test.py to verify that Kubeflow is deployed successfully. * We don't reuse wait_for_kubeflow.py because that is configured to wait on JupyterHub. * We could have modified wait_for_kubeflow.py but I took this as an opportunity to convert the test to use pytest. * We don't run the component tests because those are currently configured to depend on wait_for_kubeflow.py. So we will modify them in a follow on PR.
jlewi
changed the title
|
Test passed before rebasing So this should be ready to go. |
|
/assign @gabrielwen |
jlewi
changed the title
|
Most recent failure was due to running out of CPU quota. |
|
/test all |
| // Increase the log level so that info level log statements show up. | ||
| "--log-cli-level=info", | ||
| // Test timeout in seconds. | ||
| "--timeout=500", |
There was a problem hiding this comment.
if it's waiting for the deployment to be ready, 500 seconds seems to be too constraint?
There was a problem hiding this comment.
Intersting. It doesn't seem to work like that so I guess I can remove it.
If we look at the runs the tests are definitely running longer than 5 minutes.
http://testing-argo.kubeflow.org/workflows/kubeflow-test-infra/kubeflow-postsubmit-kfctl-go-4485a65-1297-efc5?tab=workflow
| # Verify that components are actually deployed. | ||
| # TODO(jlewi): We need to parameterize this list based on whether | ||
| # we are using IAP or basic auth. | ||
| deployment_names = [ |
There was a problem hiding this comment.
maybe read config files from bootstrap/config/... ?
There was a problem hiding this comment.
I think it might be better to be explicit. This way its clear what the expectations are (e.g. which servicecs could be running).
If we parse the config and get a list of components dynamically then its not clear if we are setting expectations correctly. Also, if we do it that way and someone accidentally deletes some component then our E2E test wouldn't catch that.
|
/test all |
|
/lgtm |
|
/approve Fingers crossed tests pass. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jlewi The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/test all Latest failure was on delete; looks like a transitory issue calling Deployment manager delete. |
|
/test all |
|
/lgtm |
|
/test all |
…2795) * Extend the E2E test for the go binary to verify that Kubeflow is deployed. * Related to kubeflow#2610 E2E test for go binary. * Create kf_is_ready_test.py to verify that Kubeflow is deployed successfully. * We don't reuse wait_for_kubeflow.py because that is configured to wait on JupyterHub. * We could have modified wait_for_kubeflow.py but I took this as an opportunity to convert the test to use pytest. * We don't run the component tests because those are currently configured to depend on wait_for_kubeflow.py. So we will modify them in a follow on PR. * kfctl_delete is now passing so enable it. * Address comments. * Disable katib stduy job test because it is flaky (Related to kubeflow#2825).
…2795) * Extend the E2E test for the go binary to verify that Kubeflow is deployed. * Related to kubeflow#2610 E2E test for go binary. * Create kf_is_ready_test.py to verify that Kubeflow is deployed successfully. * We don't reuse wait_for_kubeflow.py because that is configured to wait on JupyterHub. * We could have modified wait_for_kubeflow.py but I took this as an opportunity to convert the test to use pytest. * We don't run the component tests because those are currently configured to depend on wait_for_kubeflow.py. So we will modify them in a follow on PR. * kfctl_delete is now passing so enable it. * Address comments. * Disable katib stduy job test because it is flaky (Related to kubeflow#2825).
…2795) * Extend the E2E test for the go binary to verify that Kubeflow is deployed. * Related to kubeflow#2610 E2E test for go binary. * Create kf_is_ready_test.py to verify that Kubeflow is deployed successfully. * We don't reuse wait_for_kubeflow.py because that is configured to wait on JupyterHub. * We could have modified wait_for_kubeflow.py but I took this as an opportunity to convert the test to use pytest. * We don't run the component tests because those are currently configured to depend on wait_for_kubeflow.py. So we will modify them in a follow on PR. * kfctl_delete is now passing so enable it. * Address comments. * Disable katib stduy job test because it is flaky (Related to kubeflow#2825).
Adds the component tests but not the spark operator tests.
Related to [kfctl] Need E2E test for go binary #2610 E2E test for go binary.
This change is