Add E2E test setup and first test

[alculquicondor]

We can use a kind cluster.

The test should create basic Capacity, Queue and a batchv1/Job, and wait for it to complete.

[alculquicondor]

/kind feature
/priority important-longterm
/size M

[utkarsh-singh1]

Hi everyone , Is there anyone working on this issue, if there is none then I would like to contribute.

[alculquicondor]

For a change like this, maybe first give us an overview on how you plan to solve it. Otherwise, you can take it.

[kerthcet]

I'll take a look of this when I have time.
/assign

[kannon92]

Hello @kerthcet, I'm interested in helping out. Could I take this issue if you haven't made much progress on it?

I was going to follow a similar pattern as what is done with https://github.com/kubeflow/mpi-operator/tree/master/v2/test/integration.

[kerthcet]

Thanks Kevin, I made some progress months ago, see here kerthcet@34c78bd, but then moved on to other things. You can work on this if you like. If not, I'll work on the features required in v0.3 firstly and then turn to this feature.

I hope we can have e2e tests in v0.3.

[kannon92]

Yea I'll take this then.

/assign

[kannon92]

So where do we want the e2e tests to run? I see that we use Prow for running our CI. Do I follow similar behavior for running the CI tests? Or can I add a github action to run the e2e tests?

[alculquicondor]

Prow please.

[kannon92]

@alculquicondor Any notes on how to add these e2e tests to Prow? I don't see anything in the repo and I'm unfamiliar with Prow.

[alculquicondor]

I'm not fully familiar, but here is where our presubmits are defined https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes-sigs/kueue/kueue-presubmits.yaml

[alculquicondor]

Maybe you can take the descheduler as reference https://github.com/kubernetes/test-infra/blob/654386a21f6f378024dcedea7e6b167da0b25de7/config/jobs/kubernetes-sigs/descheduler/descheduler-presubmits-master.yaml#L56

[kannon92]

Also, I notice that this project lacks examples (other than samples in config).

I was thinking that I could approach the e2e tests in two ways:

1. Define inline CQ and Workload in the e2e test and run them.
   • I don't love this because users glancing at a project won't see these as examples
   • It is pretty easy for developers to add though.
2. Point our e2e test to an example directory and have the e2e test suite run the examples files.

• We can create an examples directory and add new features as examples.

I like 2 because I find a project that has examples to be welcoming. Having them integrated in the e2e test at least verifies that functionally they are correct.

I'm open to either but I just wanted your thoughts.

[alculquicondor]

There is already a samples directory https://github.com/kubernetes-sigs/kueue/tree/main/config/samples (maybe it's worth moving it up to the root)

As long as it's easy to setup kubectl in Prow, I'm ok with option 2.

[kannon92]

Hello. I have #421 up but I need a little guidance on it. I am having trouble with getting the webhooks working with Kind and the controller runtime. Unsure if this is the right path or if I should try something different.