Report unit test coverage in presubmit checks

[liu-cong]

What would you like to be added:

Our presubmit unit test checks should report incremental test coverage, and fail the check if the coverage is below a certain threshold (say 80%), and allow the maintainers to override it (in certain cases it's reasonable to have a low coverage, e.g., code better tested in e2e tests, or to expedite a bug fix with follow up test coverage).

Why is this needed:

As the project becomes more mature, we should really emphasize on testing and reliability.

[kfswain]

@liu-cong for the short term, what do you think about just reporting the coverage, and a follow up issue to tighten up the CI?

I'd hate to implement this all in one swing and have a sudden bottleneck due to poor coverage. I think if we see where we are at first, and then act as needed (write more tests, or if we are in a good spot, add enforcement)

[liu-cong]

for the short term, what do you think about just reporting the coverage, and a follow up issue to tighten up the CI?

This is perfectly reasonable 👍 I think just having the coverage report will motivate people on thinking more on testing

[kfswain]

Issue context:
Below is a snippet of the output of make test:

?       sigs.k8s.io/gateway-api-inference-extension/api [no test files]
        sigs.k8s.io/gateway-api-inference-extension/api/v1alpha2                coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/applyconfiguration                coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/applyconfiguration/api/v1alpha2           coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/applyconfiguration/internal               coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned               coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned/fake          coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned/scheme                coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned/typed/api/v1alpha2            coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned/typed/api/v1alpha2/fake               coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/informers/externalversions                coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/informers/externalversions/api            coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/client-go/informers/externalversions/api/v1alpha2           coverage: 0.0% of statements
?       sigs.k8s.io/gateway-api-inference-extension/client-go/informers/externalversions/internalinterfaces     [no test files]
        sigs.k8s.io/gateway-api-inference-extension/client-go/listers/api/v1alpha2              coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/cmd/bbr             coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/cmd/epp             coverage: 0.0% of statements
?       sigs.k8s.io/gateway-api-inference-extension/hack        [no test files]
        sigs.k8s.io/gateway-api-inference-extension/internal/runnable           coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/internal/tls                coverage: 0.0% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/bbr/handlers    1.306s  coverage: 47.6% of statements
        sigs.k8s.io/gateway-api-inference-extension/pkg/bbr/metrics             coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/pkg/bbr/server              coverage: 0.0% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics     1.219s  coverage: 66.7% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/controller  1.774s  coverage: 73.0% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datastore   1.529s  coverage: 57.0% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/handlers    4.579s  coverage: 18.8% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics     1.596s  coverage: 97.8% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling  1.820s  coverage: 97.1% of statements
        sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types            coverage: 0.0% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/server      1.576s  coverage: 10.8% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/env    1.137s  coverage: 100.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/error          coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging                coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/pod            coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/testing                coverage: 0.0% of statements
        sigs.k8s.io/gateway-api-inference-extension/test/integration            coverage: 0.0% of statements
ok      sigs.k8s.io/gateway-api-inference-extension/test/integration/bbr        27.633s coverage: [no statements]
ok      sigs.k8s.io/gateway-api-inference-extension/test/integration/epp        96.572s coverage: [no statements]
        sigs.k8s.io/gateway-api-inference-extension/test/utils          coverage: 0.0% of statements

Some of these having 0% coverage (such as the /api or /client-go) is perfectly acceptable, others, less acceptable.
I'd consider this issue complete if we expose the coverage numbers better during test run.

[liu-cong]

Below is a snippet of the output of make test:

Thanks for sharing the example. This looks like total coverage, not incremental coverage, right?

To deliver the first phase, concretely we should:

• Set up a "ignore list" to bypass files such as /client-go to reduce noise.
• Also report incremental coverage
• (really nice-to-have) Highlight coverages below certain threshold so it's easy to catch