merge tests repo into this one

[chavafg]

As discussed in the arch meeting, we want to merge the tests repo into this one.

[ariel-adam]

@chavafg was this done?

[jodh-intel]

The benefit of doing this (I think) is to simplify the CI setup and further consolidate the Kata repos.

Some further information:

• All stand-alone code repos now exist in this repo, which means that...
• All the unit tests exist in this repo.

The problem is that since we dropped a number of features in Kata 2.0 to simplify the codebase and concentrate on the core features. One outcome of this is that we no longer have a OCI CLI for Kata 2.x (it's not needed any more 😄)

However, since a lot of tests in https://github.com/kata-containers/tests assume an OCI CLI (for docker-based tests for example), we cannot run all of the 1.x tests against a 2.x system. That implies we need to:

• Review all the 1.x tests to see if they:
  • can be ported to a 2.x environment.
  • are no longer required in 2.x.
• Find another way to re-enable those 1.x tests. Examples being:
  • Resolve moby/docker shimv2 support #191
  • Make podman work with shimv2.

In summary, whatever we do will entail a large amount of effort.

Other considerations:

• We may end up breaking the CI temporarily (that's not a reason not to do this, but just something to point out ;)
• It would be easier to handle porting by retaining the 1.x and 2.x non-unit tests in a single repo (https://github.com/kata-containers/tests ).
• Moving the tests repo into this one will add a lot of extra code. That's not necessarily a problem but if a user want to "just build the runtime", they probably won't want to run all the big integration tests (the unit tests alone might be sufficient for their needs).

We also have this related but potentially conflicting issue which is still open: kata-containers/ci#260.

We might be able to use a git submodule to accomplish this work. But if we only want a sub-set of the 1.x tests in 2.y, that may not buy us much benefit as we either can't use a submodule, or we have to modify the code in the stand-alone tests repo to detect they are running in the 2.y repo and disable certain tests.

Given all the issues around this, I wonder if we want to discuss this further at the Architecture Committee meeting?

[jcvenegas]

For migration, it can be done in parallel to the current CI jobs.

1. Create a Job that will test the CI using the merged code.
2. Mark the new job as not required with a different trigger keyword /test-merge
3. Incrementally fix the job that uses the merged test repo.
4. Once we are happy lets migrate the other jobs in jenkins.

[fidencio]

/cc @c3d

[fidencio]

@kata-containers/architecture-committee, we didn't have a final decision about merging, but we were leaning towards it. One point missing was consulting @amshinde.

@amshinde, is there something that concerns you about moving forward with this?

Then, we also discussed there's enough tasks to be done before this merge happens, as mentioned by @jcvenegas. Who'll take the tasks over and work on them?

[fidencio]

The @kata-containers/architecture-committee decided this is the way to go as part of Today's meeting.

[jodh-intel]

I'm not sure how this got closed (even though GH shows I closed it), so re-opening! 😄

[fidencio]

The @kata-containers/architecture-committee decided this is the way to go as part of Today's meeting.

Okay, let me summarise what happened here. We had a meeting at the beginning of the year where we discussed this and we agreed that we should slowly do the move. However, we didn't get anyone to officially work on that.

This discussion showed up later in the year, and we had this topic brought up again, see: http://lists.katacontainers.io/pipermail/kata-dev/2021-August/002091.html and Gabriela mentioned she's not that much in favour of the change (see: http://lists.katacontainers.io/pipermail/kata-dev/2021-August/002095.html) and I can only back her up on this.

Unless someone volunteers to actually do a PoC, raise PRs, and ensure they'll commit to maintain the changes, this won't happen. So, with this in mind, I'm closing this one for now and if someone from the community decides to step in we can re-open this one and re-discuss the issue, but it has to come with actions. :-)