[0.11 backport] integration: enforce features compat through env vars

[crazy-max]

backport of:

• integration: enforce features compat through env vars #3711
• integration: missing feature compat checks #3715

Not a clean cherry-pick. testBasicS3CacheImportExport, testBasicAzblobCacheImportExport and testBasicGhaCacheImportExport are missing on v0.11 branch.

[thaJeztah]

testBasicS3CacheImportExport -> those features are not available in 0.11, or 0.11 doesn't have those tests?

[crazy-max]

@thaJeztah on v0.11 branch there were no integration tests. It was just hack scripts that were running the tests in our workflow:

• https://github.com/moby/buildkit/tree/v0.11/hack/s3_test
• https://github.com/moby/buildkit/tree/v0.11/hack/azblob_test

This has changed for v-next so now it will run the tests on moby:

• integration: basic s3 cache test #3398
• integration: basic azblob cache test #3477

I'm still wondering why buildkit tests were failing with missing s3 backend as there is no integration tests for v0.11 🤔 cc @vvoland

[thaJeztah]

Would it make sense to backport the integration tests to improve coverage?

[tonistiigi]

Could we have a draft PR with CI run in moby to make sure this is all correct?

[vvoland]

I'm still wondering why buildkit tests were failing with missing s3 backend as there is no integration tests for v0.11 🤔

IIRC it was this one:

buildkit/client/client_test.go

Line 4437 in 3f612f6

Type: "s3",

[crazy-max]

I'm still wondering why buildkit tests were failing with missing s3 backend as there is no integration tests for v0.11 🤔

IIRC it was this one:

buildkit/client/client_test.go

Line 4437 in 3f612f6

Type: "s3",

Oh good catch I missed this one. Will open a follow-up on BuildKit master branch to fix that.

[crazy-max]

Could we have a draft PR with CI run in moby to make sure this is all correct?

moby/moby#45160

[tonistiigi]

is this all up-to-date with #3715 ?

[crazy-max]

is this all up-to-date with #3715 ?

@tonistiigi Yes