Add pull-from-mirror for adding per-mirror level restrictions #1411

QiWang19 · 2021-11-19T03:30:51Z

Close: #1407
Add pull-from-mirror: all, digest-only, tag-only for adding per-mirror level restrictions

The mirror-by-digest-only for primary is still allowed configuring,
and it is honored for compatibility

Signed-off-by: Qi Wang qiwan@redhat.com

QiWang19 · 2021-11-19T03:32:23Z

@mtrmac @vrothberg Could you review?

mtrmac

(A quick first glance only.)

pkg/sysregistriesv2/system_registries_v2.go

docs/containers-registries.conf.5.md

pkg/sysregistriesv2/system_registries_v2.go

pkg/sysregistriesv2/testdata/pull-sources-from-reference.conf

QiWang19 · 2021-11-30T18:21:03Z

@vrothberg @mtrmac PTAL

vrothberg

Other than the nit, LGTM

pkg/sysregistriesv2/system_registries_v2_test.go

QiWang19 · 2021-12-03T04:35:12Z

@vrothberg PTAL.

vrothberg

LGTM
@mtrmac PTAL

docs/containers-registries.conf.5.md

rhatdan · 2021-12-03T19:49:30Z

@mtrmac PTAL

QiWang19 · 2021-12-10T19:52:41Z

@mtrmac Could you review?

QiWang19 · 2022-01-05T13:24:07Z

@mtrmac Could you review?

mtrmac

I apologize for the late response.

This does work fine, and it’s perfectly sufficient for correctness.

OTOH it doesn’t implement openshift/enhancements#929 as currently proposed. That enhancement has a separate list of per-digest mirrors and per-tag mirrors, i.e. it is possible to have a tag-only mirror.

I can imagine, with absolutely no data, that some users might want to use “cheap” mirrors for digested references, and only use “more expensive”, better-secured, mirrors for tagged references. But even in that hypothetical case, I find it fairly hard to argue that they’d rather have a failed pull than use the “more expensive” mirror.

So, for c/image, I think this could well be sufficient — and users might set up the configuration so that digest-only mirrors are listed before the digest+tag mirrors, if they wanted to avoid use of a “costly” digest+tag mirror.

But it’s pretty surprising to introduce the OpenShift feature, and this underlying design for an implementation of the OpenShift feature, at the same time, but not have them do exactly the same thing. So either c/image should allow configuring tag-only mirrors, or the OpenShift CR should be changed so that the “tag” mirrors are documented to also be used for digested references.

It’s somewhat likely that there was an earlier conversation about this that I can’t remember/find; is there?

Except I can find #1407 (comment) , so it’s quite possible that the current inconsistent state of things is my fault. If that’s the case, I’m sorry.

docs/containers-registries.conf.5.md

mtrmac · 2022-01-15T06:33:29Z

docs/containers-registries.conf.5.md

+Also note that if all mirrors in the `mirror` array have `digest-only = true`, images referenced by a tag will only use the primary
+registry, failing if that registry is not accessible.


I don’t think we need the note about only using the primary twice; it’s a side point and not that important.

Maybe leave the note in the previous location (which is “outside” of the mirror-by-digest-only definition list entry anyway), and just write “if all mirrors are configured to be digest-only”, without mentioning a specific field name.

The digest-only copy will, I assume be removed — then the only copy of this note should be a separate paragraph, not a part of the mirrors-by-digest-only paragraph.

pkg/sysregistriesv2/system_registries_v2.go

pkg/sysregistriesv2/system_registries_v2_test.go

mtrmac · 2022-01-15T06:33:29Z

pkg/sysregistriesv2/system_registries_v2_test.go

@@ -675,6 +675,72 @@ func TestPullSourcesFromReference(t *testing.T) {
 	assert.Equal(t, 1, len(pullSources))
 }

+func TestPullSourcesMirrorFromReference(t *testing.T) {
+	sys := &types.SystemContext{


To make it easier for the future, could TestPullSourcesFromReference be modified to use the same table-driven format, and then these tests added to that function, without adding a new config file?

It’s a bit out of scope (especially with the testing for Location/Insecure), so I’m perfectly fine with doing that work myself later, in a separate PR.

ok, I'd leave that to you.

pkg/sysregistriesv2/system_registries_v2_test.go

QiWang19 · 2022-01-18T21:48:40Z

So either c/image should allow configuring tag-only mirrors, or the OpenShift CR should be changed so that the “tag” mirrors are documented to also be used for digested references.

Does the tag-only mirrors mean the registry reference without tag should be rejected in the case? The openshift/enhancements#929 proposed to allow using tags, not to limit the use to tag-only.

mtrmac · 2022-01-19T07:45:39Z

By tag-only mirrors I mean mirrors that aren’t used for digested references. Digested references would only use digest-only mirrors (if any), not the tag-only mirrors, and the primary (non-mirror) location.

The enhancement’s ImageSourceTagPolicy AFAICS only documents affecting tagged references; with this PR, that can’t be implemented as specified because the ISTP mirrors would be used for digest-only references as well.

Given ISDP (source -> mirrorDigest), ISTP (source -> mirrorTag), AFAICS OpenShift documents that source:latest should be looked up in (mirrorTag:latest, source:latest), and source@sha256… should be looked up in (mirrorDigest@sha256…, source@sha256…). With this PR, the former can’t be configured, the best we could do is (mirrorTag:latest, mirrorDigest:latest, source:latest).

I don’t have a very strong opinion on whether to change this implementation or the OpenShift CR.

QiWang19 · 2022-01-19T15:28:55Z

By tag-only mirrors I mean mirrors that aren’t used for digested references. Digested references would only use digest-only mirrors (if any), not the tag-only mirrors, and the primary (non-mirror) location.

With this PR, the former can’t be configured, the best we could do is (mirrorTag:latest, mirrorDigest:latest, source:latest).

The former one follows current implementation, like the testcase

I think you mean the digest example, source@sha256… should be looked up in (mirrorDigest@sha256…, source@sha256…) cannot be configured,
Current PR does (mirrorDigest@sha256, mirrorTag@sha256, source@sha256).

mtrmac · 2022-01-19T15:32:40Z

I think you mean the digest example

Yes, my mistake. Thanks.

QiWang19 · 2022-01-19T17:53:19Z

@mtrmac I fixed the reviews, PTAL.

mtrmac

Thanks! The implementation LGTM, just some test and documentation nits.

I don’t think it would be reasonable to block this PR on the inconsistency with OpenShift plans, per #1411 (review) . But I do hope there’s some plan to address that.

I suppose another c/image PR that adds another per-mirror option can land later, or something.

pkg/sysregistriesv2/system_registries_v2_test.go

mtrmac · 2022-01-22T14:46:32Z

pkg/sysregistriesv2/system_registries_v2_test.go

+		pullSource, err := registry.PullSourcesFromReference(digestedRef)
+		assert.Nil(t, err)
+		for i, s := range tc.digestSources {
+			assert.Equal(t, s, pullSource[i].Endpoint.Location)


Absolutely non-blocking: Testing pullSource[i].Reference would be more directly related to the end-user-wanted behavior.

But this works just fine, and I can clean that up later when unifying the two tests.

QiWang19 · 2022-03-24T04:36:09Z

@mtrmac @vrothberg PTAL

docs/containers-registries.conf.5.md

pkg/sysregistriesv2/system_registries_v2.go

QiWang19 · 2022-03-24T19:57:39Z

@vrothberg @mtrmac PTAL.

QiWang19 · 2022-03-28T16:57:49Z

@mtrmac Could you review it?

pkg/sysregistriesv2/system_registries_v2.go

QiWang19 · 2022-03-30T03:44:35Z

@vrothberg @mtrmac PTAL

vrothberg

Other than the nit, LGTM

vrothberg · 2022-03-30T08:27:34Z

pkg/sysregistriesv2/system_registries_v2.go

+			switch mirror.PullFromMirror {
+			case MirrorByDigestOnly:
+				if !isDigested {
+					continue
+				}
+				endpoints = append(endpoints, mirror)
+			case MirrorByTagOnly:
+				if isDigested {
+					continue
+				}
+				endpoints = append(endpoints, mirror)
+			default:
+				endpoints = append(endpoints, mirror)
+			}


Suggested change

switch mirror.PullFromMirror {

case MirrorByDigestOnly:

if !isDigested {

continue

}

endpoints = append(endpoints, mirror)

case MirrorByTagOnly:

if isDigested {

continue

}

endpoints = append(endpoints, mirror)

default:

endpoints = append(endpoints, mirror)

}

switch mirror.PullFromMirror {

case MirrorByDigestOnly:

if !isDigested {

continue

}

case MirrorByTagOnly:

if isDigested {

continue

}

}

endpoints = append(endpoints, mirror)

mtrmac

Design and overall implementation LGTM.

pkg/sysregistriesv2/system_registries_v2.go

pkg/sysregistriesv2/system_registries_v2_test.go

docs/containers-registries.conf.5.md

Close: containers#1407 Add pull-from-mirror: all, digest-only, tag-only for adding per-mirror level restrictions to image pull through mirrors. The `mirror-by-digest-only` for primary is still allowed configuring, and it is honored for compatibility Signed-off-by: Qi Wang <qiwan@redhat.com>

QiWang19 · 2022-03-30T18:22:06Z

@vrothberg @mtrmac PTAL. This is ready to get merged.

mtrmac

LGTM. Thanks!

vrothberg

LGTM

QiWang19 · 2022-03-31T15:21:45Z

@mtrmac @vrothberg can we have a new release for this feature?

mtrmac · 2022-04-01T12:48:29Z

Sure, a new release sounds fine. @vrothberg WDYT? Is there any reason not to, or anything urgent we should finish first?

vrothberg · 2022-04-04T07:13:55Z

Releasing now SGTM. Thanks for working on this, @QiWang19!

mtrmac · 2022-04-04T16:39:52Z

#1514 filed to release v5.21.0.

mtrmac reviewed Nov 19, 2021

View changes

pkg/sysregistriesv2/system_registries_v2.go Outdated Show resolved Hide resolved

pkg/sysregistriesv2/system_registries_v2.go Show resolved Hide resolved

QiWang19 force-pushed the digest-set-for-mirror branch from 05f001d to f1b55f8 Compare Nov 19, 2021

QiWang19 changed the title ~~set mirror-by-digest-only for each mirror instead of registry~~ Set digest-only for each mirror Nov 19, 2021

QiWang19 force-pushed the digest-set-for-mirror branch from f1b55f8 to 522c3f4 Compare Nov 19, 2021

vrothberg reviewed Nov 22, 2021

View changes

QiWang19 force-pushed the digest-set-for-mirror branch 2 times, most recently from 33e1e10 to cd80244 Compare Nov 30, 2021

vrothberg reviewed Dec 1, 2021

View changes

pkg/sysregistriesv2/system_registries_v2_test.go Outdated Show resolved Hide resolved

QiWang19 force-pushed the digest-set-for-mirror branch from cd80244 to dff5d35 Compare Dec 3, 2021

vrothberg reviewed Dec 3, 2021

View changes

TomSweeneyRedHat reviewed Dec 3, 2021

View changes

docs/containers-registries.conf.5.md Outdated Show resolved Hide resolved

QiWang19 force-pushed the digest-set-for-mirror branch 2 times, most recently from f23b1f1 to d3363ea Compare Dec 3, 2021

QiWang19 force-pushed the digest-set-for-mirror branch from d3363ea to ffb4bb3 Compare Dec 10, 2021

QiWang19 force-pushed the digest-set-for-mirror branch from ffb4bb3 to cfc5a3a Compare Dec 21, 2021

mtrmac reviewed Jan 15, 2022

View changes

QiWang19 force-pushed the digest-set-for-mirror branch from cfc5a3a to 6603bcd Compare Jan 18, 2022

mtrmac reviewed Jan 22, 2022

View changes

QiWang19 force-pushed the digest-set-for-mirror branch from 6603bcd to eea9df4 Compare Jan 24, 2022

QiWang19 force-pushed the digest-set-for-mirror branch 2 times, most recently from e73f234 to 71f0bfa Compare Mar 24, 2022

QiWang19 changed the title ~~Set digest-only for each mirror~~ Add pull-from-mirror for adding per-mirror level restrictions Mar 24, 2022

vrothberg reviewed Mar 24, 2022

View changes

docs/containers-registries.conf.5.md Outdated Show resolved Hide resolved

vrothberg reviewed Mar 24, 2022

View changes

docs/containers-registries.conf.5.md Outdated Show resolved Hide resolved

vrothberg reviewed Mar 24, 2022

View changes

docs/containers-registries.conf.5.md Outdated Show resolved Hide resolved

vrothberg reviewed Mar 24, 2022

View changes

QiWang19 force-pushed the digest-set-for-mirror branch 3 times, most recently from 84f966d to 165cb39 Compare Mar 24, 2022

QiWang19 force-pushed the digest-set-for-mirror branch from 165cb39 to 0549bd7 Compare Mar 28, 2022

vrothberg reviewed Mar 29, 2022

View changes

QiWang19 force-pushed the digest-set-for-mirror branch from 0549bd7 to be9bfd1 Compare Mar 29, 2022

vrothberg reviewed Mar 30, 2022

View changes

QiWang19 force-pushed the digest-set-for-mirror branch from be9bfd1 to 5d29fb7 Compare Mar 30, 2022

mtrmac reviewed Mar 30, 2022

View changes

QiWang19 force-pushed the digest-set-for-mirror branch from 5d29fb7 to 0642df2 Compare Mar 30, 2022

mtrmac approved these changes Mar 31, 2022

View changes

vrothberg approved these changes Mar 31, 2022

View changes

vrothberg merged commit f3198c2 into containers:main Mar 31, 2022
9 checks passed

Add pull-from-mirror for adding per-mirror level restrictions #1411

Add pull-from-mirror for adding per-mirror level restrictions #1411

QiWang19 commented Nov 19, 2021 •

edited

QiWang19 commented Nov 19, 2021

mtrmac left a comment

QiWang19 commented Nov 30, 2021

vrothberg left a comment

QiWang19 commented Dec 3, 2021

vrothberg left a comment

rhatdan commented Dec 3, 2021

QiWang19 commented Dec 10, 2021

QiWang19 commented Jan 5, 2022

mtrmac left a comment

mtrmac Jan 15, 2022

mtrmac Jan 22, 2022

mtrmac Jan 15, 2022

QiWang19 Jan 18, 2022

QiWang19 commented Jan 18, 2022

mtrmac commented Jan 19, 2022

QiWang19 commented Jan 19, 2022

mtrmac commented Jan 19, 2022

QiWang19 commented Jan 19, 2022

mtrmac left a comment

mtrmac Jan 22, 2022

QiWang19 commented Mar 24, 2022

QiWang19 commented Mar 24, 2022

QiWang19 commented Mar 28, 2022

QiWang19 commented Mar 30, 2022

vrothberg left a comment

vrothberg Mar 30, 2022

mtrmac left a comment

QiWang19 commented Mar 30, 2022

mtrmac left a comment

vrothberg left a comment

QiWang19 commented Mar 31, 2022

mtrmac commented Apr 1, 2022

vrothberg commented Apr 4, 2022

mtrmac commented Apr 4, 2022

		Also note that if all mirrors in the `mirror` array have `digest-only = true`, images referenced by a tag will only use the primary
		registry, failing if that registry is not accessible.

Add pull-from-mirror for adding per-mirror level restrictions #1411

Add pull-from-mirror for adding per-mirror level restrictions #1411

Conversation

QiWang19 commented Nov 19, 2021 • edited

QiWang19 commented Nov 19, 2021

mtrmac left a comment

QiWang19 commented Nov 30, 2021

vrothberg left a comment

QiWang19 commented Dec 3, 2021

vrothberg left a comment

rhatdan commented Dec 3, 2021

QiWang19 commented Dec 10, 2021

QiWang19 commented Jan 5, 2022

mtrmac left a comment

mtrmac Jan 15, 2022

Choose a reason for hiding this comment

mtrmac Jan 22, 2022

Choose a reason for hiding this comment

mtrmac Jan 15, 2022

Choose a reason for hiding this comment

QiWang19 Jan 18, 2022

Choose a reason for hiding this comment

QiWang19 commented Jan 18, 2022

mtrmac commented Jan 19, 2022

QiWang19 commented Jan 19, 2022

mtrmac commented Jan 19, 2022

QiWang19 commented Jan 19, 2022

mtrmac left a comment

mtrmac Jan 22, 2022

Choose a reason for hiding this comment

QiWang19 commented Mar 24, 2022

QiWang19 commented Mar 24, 2022

QiWang19 commented Mar 28, 2022

QiWang19 commented Mar 30, 2022

vrothberg left a comment

vrothberg Mar 30, 2022

Choose a reason for hiding this comment

mtrmac left a comment

QiWang19 commented Mar 30, 2022

mtrmac left a comment

vrothberg left a comment

QiWang19 commented Mar 31, 2022

mtrmac commented Apr 1, 2022

vrothberg commented Apr 4, 2022

mtrmac commented Apr 4, 2022

QiWang19 commented Nov 19, 2021 •

edited