Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: make it easier to migrate off of gopkg.in/check.v1 #25484

Merged
merged 4 commits into from May 24, 2023

Conversation

lmb
Copy link
Contributor

@lmb lmb commented May 16, 2023

Merge after the community meeting on 2023-05-24. Since this PR touches a lot of files it won't require all CODEOWNERs to sign off before merging. Feel free to provide feedback if you've been tagged for review, but it's not a requirement.

testutils: remove PrivilegedCheck

Now that check.C embeds testing.T we don't need separate testutils anymore.
This shows how we can introduce helpers that are available from both testing
and check style tests.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

docs: explain how to migrate from gocheck

Remove outdated documentation relating to gocheck and explain how existing
tests can be translated from gocheck to testing.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

golangci: prohibit use of gopkg.in/check.v1

Configure golangci-lint so that we don't add new code which uses the
deprecated package.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

all: replace gopkg.in/check.v1 with shim

There are a lot of tests in the codebase which use gopkg.in/check.v1, which
hasn't seen an update in the last three years. A while back it was decided
that we'd migrate away from the package and instead use plain old stdlib
testing.

In practice things aren't so simple. We have so many tests that migrating
them in one fell swoop is impossible. At the same time, the API exposed by
check is entirely incompatible with testing. It's not possible to write a
helper function which works for both check and testing. So far we've opted
to provide two separate helpers, for example PrivilegedTest and
PrivilegedCheck. This causes a lot of duplication and discourages sharing
helpers. It also makes refactoring code from check to testing harder since
the natural way to express the SetUp and TearDown behaviour of check is such
a helper function.

Additionally, check is a burden on anyone trying to run the tests: it has
completely separate flags which control what tests are being run. The output
also doesn't match what other go tooling expects, both for tests and
benchmarks.

To fix both of these, replace gopkg.in/check.v1 with a fork that removes
most command line flags and gets rid of the custom runner. Instead, suites
and tests are run as subtests. The other API should be mostly compatible.
Most importantly check.C now embeds testing.T. As a result, check.C
satisfies testing.TB, which allows us to access helpers written for the
testing package from within unmigrated tests.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
Make it easier to migrate off of gopkg.in/check.v1

@lmb lmb added the release-note/misc This PR makes changes that have no direct user impact. label May 16, 2023
@lmb
Copy link
Contributor Author

lmb commented May 16, 2023

/test

@lmb lmb force-pushed the checkmate branch 2 times, most recently from 8c1f19b to d77f3cf Compare May 16, 2023 15:56
@lmb
Copy link
Contributor Author

lmb commented May 16, 2023

/test

@lmb
Copy link
Contributor Author

lmb commented May 16, 2023

/test

@lmb
Copy link
Contributor Author

lmb commented May 17, 2023

/test

1 similar comment
@lmb
Copy link
Contributor Author

lmb commented May 17, 2023

/test

@lmb
Copy link
Contributor Author

lmb commented May 17, 2023

/test

@lmb lmb self-assigned this May 17, 2023
@lmb lmb changed the title all: make it easier to get rid of gopkg.in/check.v1 all: replace gopkg.in/check.v1 with shim May 17, 2023
@lmb lmb force-pushed the checkmate branch 7 times, most recently from e7c9042 to 7c3ac8a Compare May 17, 2023 16:11
@lmb
Copy link
Contributor Author

lmb commented May 17, 2023

/test

@lmb lmb force-pushed the checkmate branch 2 times, most recently from a63a6bb to 5b53733 Compare May 18, 2023 09:11
@lmb
Copy link
Contributor Author

lmb commented May 18, 2023

/test

@christarazi christarazi added area/CI Continuous Integration testing issue or flake area/CI-improvement Topic or proposal to improve the Continuous Integration workflow and removed release-note/misc This PR makes changes that have no direct user impact. labels May 18, 2023
Copy link
Member

@giorio94 giorio94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
@nathanjsweet nathanjsweet removed their request for review May 23, 2023 17:19
@lmb lmb force-pushed the checkmate branch 2 times, most recently from 811261a to 096315f Compare May 23, 2023 17:26
@lmb lmb changed the title all: replace gopkg.in/check.v1 with shim all: make it easier to migrate off of gopkg.in/check.v1 May 23, 2023
Copy link
Contributor

@tommyp1ckles tommyp1ckles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really good change - Azure LGTM

Copy link
Member

@christarazi christarazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹

Copy link
Contributor

@zacharysarah zacharysarah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lmb 👋🏻 Some style nits, otherwise docs LGTM. Approving with the understanding that changes must be made prior to merge.

Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
Documentation/contributing/testing/unit.rst Outdated Show resolved Hide resolved
@lmb
Copy link
Contributor Author

lmb commented May 24, 2023

/test

lmb and others added 4 commits May 24, 2023 15:19
There are a lot of tests in the codebase which use gopkg.in/check.v1,
which hasn't seen an update in the last three years. A while back it
was decided that we'd migrate away from the package and instead use
plain old stdlib testing.

In practice things aren't so simple. We have so many tests that
migrating them in one fell swoop is impossible. At the same time,
the API exposed by check is entirely incompatible with testing.
It's not possible to write a helper function which works for both
check and testing. So far we've opted to provide two separate
helpers, for example PrivilegedTest and PrivilegedCheck. This causes
a lot of duplication and discourages sharing helpers. It also
makes refactoring code from check to testing harder since the natural
way to express the SetUp and TearDown behaviour of check is
such a helper function.

Additionally, check is a burden on anyone trying to run the tests:
it has completely separate flags which control what tests are
being run. The output also doesn't match what other go tooling
expects, both for tests and benchmarks.

To fix both of these, replace gopkg.in/check.v1 with a fork that
removes most command line flags and gets rid of the custom runner.
Instead, suites and tests are run as subtests. The other API should
be mostly compatible. Most importantly check.C now embeds testing.T.
As a result, check.C satisfies testing.TB, which allows us to
access helpers written for the testing package from within unmigrated
tests.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
Remove outdated documentation relating to gocheck and explain how
existing tests can be translated from gocheck to testing.

Co-authored-by: ZSC <zacharysarah@users.noreply.github.com>
Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
Configure golangci-lint so that we don't add new code which uses
the deprecated package.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
Now that check.C embeds testing.T we don't need separate testutils
anymore. This shows how we can introduce helpers that are available
from both testing and check style tests.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
@lmb
Copy link
Contributor Author

lmb commented May 24, 2023

/test

Job 'Cilium-PR-K8s-1.26-kernel-net-next' failed:

Click to show.

Test Name

K8sDatapathServicesTest Checks N/S loadbalancing With host policy Tests NodePort

Failure Output

FAIL: Can not connect to service "tftp://[fd04::12]:31033/hello" from outside cluster (1/10)

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.26-kernel-net-next/145/

If it is a flake and a GitHub issue doesn't already exist to track it, comment /mlh new-flake Cilium-PR-K8s-1.26-kernel-net-next so I can create one.

Then please upload the Jenkins artifacts to that issue.

@lmb
Copy link
Contributor Author

lmb commented May 24, 2023

As per the discussion in the maintainers / community channel, tagging this ready to merge.

@lmb lmb added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label May 24, 2023
@joestringer joestringer merged commit 091202c into cilium:main May 24, 2023
57 of 58 checks passed
@lmb lmb deleted the checkmate branch May 24, 2023 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/CI Continuous Integration testing issue or flake area/CI-improvement Topic or proposal to improve the Continuous Integration workflow ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/ci This PR makes changes to the CI.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet