Add more tests to the subgraph we created to run the tests.

[jlewi]

• Update wait_for_deployment.py to use KUBECONFIG so its not limited to GKE.

• I don't think we actually need wait_for_deployment; all it does is wait
  for the CRD to be created and that's not very useful. It would
  be better to wait for the actual controller deployments to start.

• Create a script wait_for_kubeflow based on deploy_kubeflow.py that
  waits for Kubeflow to be deployed and performs basic checks like
  ensuring everything started correctly.

• Fix a typo in wait_for_kubeflow

• Add steps to copy artifacts to prow bucket.

• Create a workflow for unittests."

• Don't run workflows.jsonnet on presubmit; we use kfctl_test now.

• Add an option to kfctl_test to not delete the cluster; useful
  for leaving it up to debug tests.

Related to: #1325 Make our E2E tests more reusable and composable.

---

This change is 

[jlewi]

/test all

[jlewi]

Test failures look like SSL errors on checkout.

[jlewi]

/test all

[jlewi]

/test all

[jlewi]

Tests passed:
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/kubeflow_kubeflow/1342/kubeflow-presubmit/2922/

[jlewi]

/test all

[jlewi]

/assign @lluunn

[lluunn]

will review it tomorrow morning

[lluunn]

Reviewed 7 of 9 files at r1.
Reviewable status: 7 of 9 files reviewed, 4 unresolved discussions (waiting on @jlewi, @pdmack, and @gaocegege)

---

testing/wait_for_deployment.py, line 17 at r1 (raw file):

# limitations under the License.

"""Wait for kubeflow deployment.

[optional] might be better to change the file name to wait_for_kf_deployment since deployment can be confused with k8s resource

---

testing/workflows/components/kfctl_test.jsonnet, line 138 at r1 (raw file):

  },  // checkout
  {
    template: buildTemplate("create-pr-symlink", [

pr-symlink should be common and can move to workflow.lib?

---

testing/workflows/components/unit_tests.jsonnet, line 15 at r1 (raw file):

local bucket = params.bucket;

// mountPath is the directory where the volume to store the test data

[optional] The code block (setting up path and dir variables) is also dup in all workflows, might be good to find a way to dedup.

---

testing/workflows/components/unit_tests.jsonnet, line 54 at r1 (raw file):

    image: image,
    workingDir: working_dir,
    // TODO(jlewi): Change to IfNotPresent.

when should this happen?

[jlewi]

Reviewable status: 7 of 9 files reviewed, 4 unresolved discussions (waiting on @jlewi, @pdmack, and @gaocegege)

---

testing/wait_for_deployment.py, line 17 at r1 (raw file):

Previously, lluunn (Lun-Kai Hsu) wrote…

[optional] might be better to change the file name to wait_for_kf_deployment since deployment can be confused with k8s resource

I think we can eventually get rid of this script but this PR is already quite large and I don't want to cause unrelated breakages.

---

testing/workflows/components/kfctl_test.jsonnet, line 138 at r1 (raw file):

Previously, lluunn (Lun-Kai Hsu) wrote…

pr-symlink should be common and can move to workflow.lib?

Probably but lets leave that for a follow on PR. It depends on the build template which means we'd have to define a stanza like kfTests in workflows.libsonnet; so its a bit of work and I'd like to leave that to the future.

---

testing/workflows/components/unit_tests.jsonnet, line 15 at r1 (raw file):

Previously, lluunn (Lun-Kai Hsu) wrote…

[optional] The code block (setting up path and dir variables) is also dup in all workflows, might be good to find a way to dedup.

Its part of the step template; we could try to define a common step template but I think doing that in a way that
makes it easy to override the parts we care about without creating too many layers of obfuscation takes some work.

---

testing/workflows/components/unit_tests.jsonnet, line 54 at r1 (raw file):

Previously, lluunn (Lun-Kai Hsu) wrote…

when should this happen?

This isn't blocked on anything but I'm hesitant to make more changes in this PR.

[lluunn]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lluunn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [lluunn]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment