[image-builder-bob] Introduce URL processing for non docker api urls #10266

princerachit · 2022-05-25T22:39:30Z

Description

Introduce a new function rewriteNonDockerAPIURL which can be used to rewrite URLs in bob proxy when the URL endpoint is NOT a docker api endpoint. See all docker registry endpoints here, it must start with /v2/.

As a side effect of this change gitpod will support Google Artifact Registry as a container registry.

Related Issue(s)

Fixes #10241

How to test

Pre requisite

You need a self hosted version of gitpod to test out GAR and Azure. You will need to setup container registry during installation.

GAR

For GAR you will need to create a GAR in a google project e.g. playground and then create a service account which has write permission to that registry. After that create a key for the service account and use it to login locally in your workspace.

Example below. You can read more here

cat my-svc-acc-key.json | docker login -u _json_key --password-stdin https://europe-docker.pkg.dev

You can also use workspace-preview cluster to test this.
Create a cluster and then copy the gitpod-config.yml from instance to your local workspace. You need to update the container registry config in this file before rendering using reapply.sh. See example gitpod-config.yaml change here.
Configure your kubeconfig to point to this cluster. Better remove any other entry from your kubeconfig.
Create a Script called reapply.sh
Create a file proxy-patch.json
Create a file trim.py. This will help in faster iteration of gitpod installation update'
Render kubectl manifests: ./reapply.sh render. Make Sure to update the reapply.sh file secret creation step first.
Deploy/Update gitpod using: ./reapply.sh deploy. Wait for a minute or two for the pods to come up.
Start a workspace in your preview env.

I have tested with following registries:

✅ Azure - Workspace-preview cluster - See Loom Video
✅ GAR - Workspace-preview cluster - See Loom Video
✅ GCR - Preview env. Through the werft build of this PR - See Loom Video

Logs

Sample log from Image Build pod:

{"level":"info","message":"serving request","req":"/v2/prince-tf-experiments/dazzle/workspace-images/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2022-05-26T13:03:20Z"}
2022/05/26 13:03:20 [DEBUG] HEAD https://europe-docker.pkg.dev/v2/prince-tf-experiments/dazzle/workspace-images/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545
{"host":"europe-docker.pkg.dev","level":"info","message":"authorizing registry access","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2022-05-26T13:03:20Z","user":"_json_key"}
2022/05/26 13:03:20 [DEBUG] HEAD https://europe-docker.pkg.dev/v2/prince-tf-experiments/dazzle/workspace-images/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545 (status: 401): retrying in 1s (3 left)
#5 pushing layers 14.4s done
#5 pushing manifest for localhost:8080/target:latest@sha256:3c93c7fc99a4471887ca128f0e18481e30b65ee6c63ae200697c11d3a1ffab84
{"grpc.code":"OK","grpc.method":"ContentStatus","grpc.service":"supervisor.StatusService","grpc.start_time":"2022-05-26T13:03:21Z","grpc.time_ms":0.037,"level":"info","message":"finished unary call with code OK","serviceContext":{"service":"supervisor","version":"commit-ab3f40db9e68ca471596056126f2d856ae59afe3"},"severity":"INFO","span.kind":"server","system":"grpc","time":"2022-05-26T13:03:21Z"}
{"level":"info","message":"serving request","req":"/v2/prince-tf-experiments/dazzle/workspace-images/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2022-05-26T13:03:21Z"}
2022/05/26 13:03:21 [DEBUG] PUT https://europe-docker.pkg.dev/v2/prince-tf-experiments/dazzle/workspace-images/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545
{"host":"europe-docker.pkg.dev","level":"info","message":"authorizing registry access","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2022-05-26T13:03:22Z","user":"_json_key"}
2022/05/26 13:03:22 [DEBUG] PUT https://europe-docker.pkg.dev/v2/prince-tf-experiments/dazzle/workspace-images/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545 (status: 401): retrying in 1s (3 left)
{"grpc.code":"OK","grpc.method":"ContentStatus","grpc.service":"supervisor.StatusService","grpc.start_time":"2022-05-26T13:03:22Z","grpc.time_ms":0.035,"level":"info","message":"finished unary call with code OK","serviceContext":{"service":"supervisor","version":"commit-ab3f40db9e68ca471596056126f2d856ae59afe3"},"severity":"INFO","span.kind":"server","system":"grpc","time":"2022-05-26T13:03:22Z"}
{"level":"info","message":"Original location: /v2/prince-tf-experiments/dazzle/workspace-images/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2022-05-26T13:03:23Z"}
{"level":"info","message":"Rewrote location: http://localhost:8080/v2/target/manifests/d260bb6ec50dd5b70a2acd883afb5cb39d9ad508dd0ddb5b2d25bdf8d9a87545","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2022-05-26T13:03:23Z"}
#5 pushing manifest for localhost:8080/target:latest@sha256:3c93c7fc99a4471887ca128f0e18481e30b65ee6c63ae200697c11d3a1ffab84 2.5s done
#5 DONE 16.9s
exit
{"alias":"5f7d75b3-73a1-44ed-a1b5-9417d0d8f666","level":"info","message":"closing terminal","serviceContext":{"service":"supervisor","version":"commit-ab3f40db9e68ca471596056126f2d856ae59afe3"},"severity":"INFO","time":"2022-05-26T13:03:23Z"}
{"level":"info","message":"received SIGTERM (or shutdown) - tearing down","serviceContext":{"service":"supervisor","version":"commit-ab3f40db9e68ca471596056126f2d856ae59afe3"},"severity":"INFO","time":"2022-05-26T13:03:23Z"}
{"level":"info","message":"shutting down API endpoint","serviceContext":{"service":"supervisor","version":"commit-ab3f40db9e68ca471596056126f2d856ae59afe3"},"severity":"INFO","time":"2022-05-26T13:03:23Z"}
{"alias":"5f7d75b3-73a1-44ed-a1b5-9417d0d8f666","level":"info","message":"terminal client left","serviceContext":{"service":"supervisor","version":"commit-ab3f40db9e68ca471596056126f2d856ae59afe3"},"severity":"INFO","time":"2022-05-26T13:03:23Z"}

🤙 This task ran as a workspace prebuild
🎉 Well done on saving 3 minutes

Release Notes

Add Support for Google Artifact Registry as Container Registry

Documentation

https://github.com/gitpod-io/website/issues/2128

sagor999

code looks good I think, but would love to see unit tests.

components/image-builder-bob/pkg/proxy/proxy.go

Process docker api urls starting with /v2/ differently from non-docker api url whose path does not start with /v2/.

sagor999

LGTM

/hold

adding hold though to coordinate with webapp @kylos101 @geropl on unpinning bob version in webapp cluster.

csweichel · 2022-05-30T08:29:56Z

components/image-builder-bob/pkg/proxy/proxy_test.go

+	for _, test := range tests {
+		t.Run(test.Name, func(t *testing.T) {
+			rewriteDockerAPIURL(&test.in.u, test.in.fromRepo, test.in.toRepo, test.in.host, test.in.tag)
+			if test.in.u.Path != test.u.Path {


minor nit: most tests use cmp.Diff to verify the result, e.g. here.

Makes for an easier read and improved consistency.

csweichel · 2022-05-31T20:22:01Z

cc @geropl

Can we unhold here? This might be important for a self-hosted install.

kylos101 · 2022-06-01T03:26:10Z

cc @geropl

Can we unhold here? This might be important for a self-hosted install.

@csweichel I think so? The pin was removed here. Am I missing anything, @geropl ?

Tomorrow (Wednesday), I'd like to rebase, unhold, and merge these PRs, so they can be tested in staging, in advance of Thursday's deploy. I'm just not sure how to initiate a deploy to staging, or, if that is automatic on merge to main.

[image-builder] replace reference to aliases with actual repo for cross mount blobs #10192
[image-builder-bob] Introduce URL processing for non docker api urls #10266 (this one)

We also need to revert this revert in a new PR (which I also plan to do tomorrow):

Revert "[image-builder-bob] Use separate auth for target and base" #10225, to redo [image-builder-bob] Use separate auth for target and base #10094

If a self-hosted install is in a pinch, rather than wait for this PR to get merged, I'd recommend creating a patch branch from the latest self-hosted release, and cherrypick #10266. I say that because it'd allow both work streams to continue independently, w/o creating dependencies on this PR being merged right away.

geropl · 2022-06-01T06:58:25Z

Can we unhold here?

💯 👍 @kylos101 @csweichel

iQQBot · 2022-06-01T15:55:49Z

mark here

csweichel · 2022-06-01T17:19:06Z

/hold cancel

roboquat added do-not-merge/work-in-progress do-not-merge/release-note-label-needed size/XS labels May 25, 2022

princerachit force-pushed the prs/support-gar branch from 424a311 to c19db5d Compare May 26, 2022 11:12

roboquat added size/M and removed size/XS labels May 26, 2022

princerachit changed the title ~~[image-builder-bob] Support Google Artifact registry~~ [image-builder-bob] Introduce URL processing for non docker api urls May 26, 2022

roboquat added release-note and removed do-not-merge/release-note-label-needed labels May 26, 2022

princerachit marked this pull request as ready for review May 26, 2022 13:35

princerachit requested a review from a team May 26, 2022 13:35

roboquat removed the do-not-merge/work-in-progress label May 26, 2022

github-actions bot added the team: workspace Issue belongs to the Workspace team label May 26, 2022

princerachit force-pushed the prs/support-gar branch from d698cec to 8811d94 Compare May 26, 2022 13:37

sagor999 suggested changes May 27, 2022

View reviewed changes

components/image-builder-bob/pkg/proxy/proxy.go Show resolved Hide resolved

components/image-builder-bob/pkg/proxy/proxy.go Outdated Show resolved Hide resolved

[image-builder] Intro non-docker url rewrite

1f9c1b9

Process docker api urls starting with /v2/ differently from non-docker api url whose path does not start with /v2/.

princerachit force-pushed the prs/support-gar branch from 8811d94 to 1f9c1b9 Compare May 27, 2022 06:44

roboquat added size/L and removed size/M labels May 27, 2022

princerachit requested review from sagor999 and a team May 27, 2022 06:45

sagor999 approved these changes May 28, 2022

View reviewed changes

roboquat added the do-not-merge/hold label May 28, 2022

csweichel reviewed May 30, 2022

View reviewed changes

roboquat removed the do-not-merge/hold label Jun 1, 2022

roboquat merged commit f2a2968 into main Jun 1, 2022

roboquat deleted the prs/support-gar branch June 1, 2022 17:19

roboquat added deployed: workspace Workspace team change is running in production deployed Change is completely running in production labels Jun 21, 2022

atduarte mentioned this pull request Jul 25, 2022

Image upload fails with 413 (request entity too large) #11561

Closed

utam0k mentioned this pull request Aug 1, 2022

registry-facade fails to authenticate against Google Artifact Registry #11454

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[image-builder-bob] Introduce URL processing for non docker api urls #10266

[image-builder-bob] Introduce URL processing for non docker api urls #10266

princerachit commented May 25, 2022 •

edited

sagor999 left a comment

sagor999 left a comment

csweichel May 30, 2022

csweichel commented May 31, 2022

kylos101 commented Jun 1, 2022

geropl commented Jun 1, 2022

iQQBot commented Jun 1, 2022

csweichel commented Jun 1, 2022

[image-builder-bob] Introduce URL processing for non docker api urls #10266

[image-builder-bob] Introduce URL processing for non docker api urls #10266

Conversation

princerachit commented May 25, 2022 • edited

Description

Related Issue(s)

How to test

Pre requisite

GAR

Logs

Release Notes

Documentation

sagor999 left a comment

Choose a reason for hiding this comment

sagor999 left a comment

Choose a reason for hiding this comment

csweichel May 30, 2022

Choose a reason for hiding this comment

csweichel commented May 31, 2022

kylos101 commented Jun 1, 2022

geropl commented Jun 1, 2022

iQQBot commented Jun 1, 2022

csweichel commented Jun 1, 2022

princerachit commented May 25, 2022 •

edited