test/conformance/api/v1beta1.TestDomainMapping is being skipped in Prow but passing in Kind

[izabelacg]

Question

Can we consider removing the snippet that drops the enable-alpha and enable-beta flags from e2e-tests.sh script for gateway-api testing?

Context

Currently, tests such as test/conformance/api/v1beta1.TestDomainMapping and a few others tests related to alpha and beta features are currently being skipped in Prow, as observed in gateway-api-contour and gateway-api-istio. This skipping appears to be due to the removal of enable-alpha and enable-beta flags for gateway-api in the e2e-tests.sh script here:

serving/test/e2e-tests.sh

Lines 45 to 49 in 40088e3

	# Drop testing alpha and beta features with the Gateway API
	if [[ "${INGRESS_CLASS}" != *"gateway-api"* ]]; then
	E2E_TEST_FLAGS+=" -enable-alpha -enable-beta"
	fi
	fi

However, some tests that run in Kind (GitHub Actions) don't use the e2e-tests.sh script to run the tests and explicitly set the above-mentioned flags, resulting in successful test execution.

Important to note that in Kind, we only run end-to-end tests, not conformance tests. However, some alpha and beta features are successfully tested in Kind. The specific test mentioned in this issue, test/conformance/api/v1beta1.TestDomainMapping, is not run in Kind since it is a conformance test. Nevertheless, it was tested locally and passed.

serving/.github/workflows/kind-e2e.yaml

Lines 212 to 221 in 40088e3

	run: |
	gotestsum --format testname -- \
	-race -count=1 -parallel=1 -tags=e2e \
	-timeout=30m \
	${{ matrix.test-path }} \
	-skip-cleanup-on-fail \
	-disable-logstream \
	-enable-alpha -enable-beta \
	--ingress-class=${{ matrix.ingress-class || matrix.ingress }}.ingress.networking.knative.dev \
	${{ matrix.test-flags }}

Test PR with tests passing in Kind: #15202

[skonto]

Hi @izabelacg, @dprotaso removed the flags here: #14296 maybe the idea was to get all basic tests pass first on prow? Btw I am ok to discover failing stuff upfront.