[kots] Allow users to upload a `.docker/config.json` file #12174

Pothulapati · 2022-08-17T13:14:24Z

Description

This PR updates the kots UI to add a new config option to upload registry
credentials irrespective of the registry being used, which is then merged
into a single config.json file and passed as the registry secret which
is then used across the workspace image builder components.

Related Issue(s)

Fixes #12136

How to test

Upload a dockerconfigjson file through the new option,

and see those credentials in the secret by running

kubectl -n gitpod get secret builtin-registry-auth --output="jsonpath={.data.\.dockerconfigjson}" | base64 -d

Release Notes

[kots] Allow users to upload a `.docker/config.json` file

Documentation

Werft options:

/werft with-preview

Pothulapati · 2022-08-18T12:09:20Z

/werft run publish-to-kots

👍 started the job as gitpod-build-tar-kots-dockerconfig.20
(with .werft/ from main)

Pothulapati · 2022-08-18T12:10:04Z

install/kots/manifests/gitpod-installer-job.yaml

@@ -174,8 +184,7 @@ spec:
                kubectl create secret docker-registry container-registry \
                  --namespace "{{repl Namespace }}" \
                  --from-file=.dockerconfigjson=/tmp/container-registry-secret \
-                  -o yaml --dry-run=client | \
-                  kubectl replace --namespace "{{repl Namespace }}" --force -f -
+                  -o yaml --dry-run=client >  "${GITPOD_OBJECTS}/templates/gitpod.yaml"


Changed this to save the secret in the gitpod.yaml, instead of applying so that we can post process it if the user submits more docker configs through the new option

Seems a reasonable approach. This means that we'll have a secret at the start of the YAML file, but that shouldn't be a problem as it's what we're doing anyway

lucasvaltl · 2022-08-18T13:32:03Z

@Pothulapati what does the UX, specifically the user flow, look like here? When does a user set this and how - do they need to also set something else for this to take Affect?

Pothulapati · 2022-08-18T13:35:52Z

@lucasvaltl This adds a new config option in the registry section of the KOTS UI (irrespective of in-cluster or external registry) through which a user can upload a file.

do they need to also set something else for this to take Affect?

Not really, because this is automatically set everywhere to be used. They will have to obviously use the container image somewhere to actually use the functionality. 🤔

mrsimonemms · 2022-08-19T10:27:00Z

install/kots/manifests/gitpod-installer-job.yaml

@@ -79,6 +79,16 @@ spec:
              fi

              echo "Gitpod: Generate the base Installer config"
+              echo "Gitpod: Create a Helm template directory"


The echo "Gitpod: Generate the base Installer config" line should go above the /app/installer init > "${CONFIG_FILE}" so that the logging is representative of what's going on

mrsimonemms · 2022-08-19T10:28:23Z

install/kots/manifests/gitpod-installer-job.yaml

@@ -174,8 +184,7 @@ spec:
                kubectl create secret docker-registry container-registry \
                  --namespace "{{repl Namespace }}" \
                  --from-file=.dockerconfigjson=/tmp/container-registry-secret \
-                  -o yaml --dry-run=client | \
-                  kubectl replace --namespace "{{repl Namespace }}" --force -f -
+                  -o yaml --dry-run=client >  "${GITPOD_OBJECTS}/templates/gitpod.yaml"


Seems a reasonable approach. This means that we'll have a secret at the start of the YAML file, but that shouldn't be a problem as it's what we're doing anyway

mrsimonemms

/hold

Works beautifully. I've respectfully made a couple of suggestions on the config wording, but approved so you can merge when ready

install/kots/manifests/kots-config.yaml

Signed-off-by: Tarun Pothulapati <tarun@gitpod.io>

Pothulapati · 2022-08-19T13:23:02Z

This is how the option looks now!

lucasvaltl · 2022-08-19T13:28:48Z

install/kots/manifests/kots-config.yaml

+          when: '{{repl ConfigOptionEquals "reg_docker_config_enable" "1" }}'
+          type: file
+          required: true
+          help_text: Docker [config JSON file](https://docs.docker.com/engine/reference/commandline/cli/#sample-configuration-file) with auth credentials used to access private registries, for workspace images.


Suggested change

help_text: Docker [config JSON file](https://docs.docker.com/engine/reference/commandline/cli/#sample-configuration-file) with auth credentials used to access private registries, for workspace images.

help_text: Docker [config JSON file](https://docs.docker.com/engine/reference/commandline/cli/#sample-configuration-file) with auth credentials used to access private registries used for pulling base workspace images.

credentials used to access private registries used for pulling base workspace images.

Happy to update, but used to <> used for <> sounds a bit confusing?

lucasvaltl · 2022-08-19T13:30:47Z

install/kots/manifests/kots-config.yaml

+          help_text: This is useful when you have base workspace images in private registries other than the above configured ones.
+
+        - name: reg_docker_config
+          title: Registry credentials


Suggested change

title: Registry credentials

title: Private base image registry credentials

lucasvaltl · 2022-08-19T13:32:46Z

install/kots/manifests/kots-config.yaml

+          title: Configure additional registry credentials for pulling workspace images
+          type: bool
+          default: "0"
+          help_text: This is useful when you have base workspace images in private registries other than the above configured ones.


Suggested change

help_text: This is useful when you have base workspace images in private registries other than the above configured ones.

help_text: This is useful when you want to use base workspace images in private registries other than the above configured ones or the [default base workspace images stored on Docker Hub](https://github.com/gitpod-io/workspace-images).

lucasvaltl

Added some suggestions that you can merge right in. Otherwise LGTM, although I think we should hack around KOTS a bit to create a better separation between the push and pull registries here.

mrsimonemms · 2022-08-19T13:38:35Z

/unhold

Follow up to #12174, and improves the help_text around the newly added fields. Signed-off-by: Tarun Pothulapati <tarun@gitpod.io>

…owList` Follow upto #12174 This PR updates the installer logic to also load the auth's reigstry URL's into `.containerRegistry.privateBaseImageAllowList`. Signed-off-by: Tarun Pothulapati <tarun@gitpod.io>

…owList` Follow upto #12174 This PR updates the installer logic to also load the auth's reigstry URL's into `.containerRegistry.privateBaseImageAllowList`. Signed-off-by: Tarun Pothulapati <tarun@gitpod.io> Co-authored-by: Simon Emms <simon@gitpod.io> Co-authored-by: Simon Emms <simon@gitpod.io>

…owList` Follow upto #12174 This PR updates the installer logic to also load the auth's reigstry URL's into `.containerRegistry.privateBaseImageAllowList`. Signed-off-by: Tarun Pothulapati <tarun@gitpod.io> Co-authored-by: Simon Emms <simon@gitpod.io> Co-authored-by: Nandaja Varma <nandaja.varma@gmail.com>

roboquat added do-not-merge/work-in-progress release-note size/S labels Aug 17, 2022

Pothulapati force-pushed the tar/kots-dockerconfig branch from 5a04f3f to e963107 Compare August 18, 2022 04:57

roboquat added size/M and removed size/S labels Aug 18, 2022

Pothulapati force-pushed the tar/kots-dockerconfig branch 11 times, most recently from 81ecde6 to 21b2ed1 Compare August 18, 2022 12:07

Pothulapati marked this pull request as ready for review August 18, 2022 12:08

Pothulapati requested a review from a team August 18, 2022 12:08

roboquat removed the do-not-merge/work-in-progress label Aug 18, 2022

github-actions bot added the team: delivery Issue belongs to the self-hosted team label Aug 18, 2022

Pothulapati commented Aug 18, 2022

View reviewed changes

Pothulapati requested a review from mrsimonemms August 18, 2022 12:10

Pothulapati force-pushed the tar/kots-dockerconfig branch from 21b2ed1 to 107b284 Compare August 18, 2022 14:12

Pothulapati added the do-not-merge/work-in-progress label Aug 19, 2022

Pothulapati force-pushed the tar/kots-dockerconfig branch from 107b284 to e429538 Compare August 19, 2022 10:25

mrsimonemms reviewed Aug 19, 2022

View reviewed changes

Pothulapati force-pushed the tar/kots-dockerconfig branch 4 times, most recently from 706b2a6 to 4caf78e Compare August 19, 2022 12:22

mrsimonemms approved these changes Aug 19, 2022

View reviewed changes

install/kots/manifests/kots-config.yaml Outdated Show resolved Hide resolved

install/kots/manifests/kots-config.yaml Outdated Show resolved Hide resolved

roboquat added the do-not-merge/hold label Aug 19, 2022

[kots] Add new config to upload dockerConfigJson file

5f32584

Signed-off-by: Tarun Pothulapati <tarun@gitpod.io>

Pothulapati force-pushed the tar/kots-dockerconfig branch from 4caf78e to 5f32584 Compare August 19, 2022 13:13

lucasvaltl reviewed Aug 19, 2022

View reviewed changes

lucasvaltl approved these changes Aug 19, 2022

View reviewed changes

roboquat removed the do-not-merge/hold label Aug 19, 2022

Pothulapati removed the do-not-merge/work-in-progress label Aug 19, 2022

roboquat merged commit 2e8fb27 into main Aug 19, 2022

roboquat deleted the tar/kots-dockerconfig branch August 19, 2022 13:39

Pothulapati added a commit that referenced this pull request Aug 25, 2022

[kots] Add clearer descriptions for docker_config_cred field

b4e335f

Follow up to #12174, and improves the help_text around the newly added fields. Signed-off-by: Tarun Pothulapati <tarun@gitpod.io>

Pothulapati mentioned this pull request Aug 25, 2022

[kots] Add clearer descriptions for docker_config_cred field #12392

Closed

1 task

Pothulapati mentioned this pull request Aug 25, 2022

[kots] load dockerConfigJson reigstry names into privateBaseImageAllowList #12398

Merged

1 task

Pothulapati mentioned this pull request Sep 2, 2022

Load test Single Cluster Reference Architecture #10869

Closed

Pothulapati mentioned this pull request Sep 13, 2022

[installer] Support adding image pull secrets even when the internal (in-cluster) registry is used #10792

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[kots] Allow users to upload a `.docker/config.json` file #12174

[kots] Allow users to upload a `.docker/config.json` file #12174

Pothulapati commented Aug 17, 2022 •

edited

Pothulapati commented Aug 18, 2022 •

edited by werft-gitpod-dev-com bot

Pothulapati Aug 18, 2022

mrsimonemms Aug 19, 2022

lucasvaltl commented Aug 18, 2022

Pothulapati commented Aug 18, 2022 •

edited by werft-gitpod-dev-com bot

mrsimonemms Aug 19, 2022

mrsimonemms Aug 19, 2022

mrsimonemms left a comment

Pothulapati commented Aug 19, 2022

lucasvaltl Aug 19, 2022 •

edited

Pothulapati Aug 19, 2022

lucasvaltl Aug 19, 2022

lucasvaltl Aug 19, 2022 •

edited

lucasvaltl left a comment

mrsimonemms commented Aug 19, 2022

	help_text: Docker [config JSON file](https://docs.docker.com/engine/reference/commandline/cli/#sample-configuration-file) with auth credentials used to access private registries, for workspace images.
	help_text: Docker [config JSON file](https://docs.docker.com/engine/reference/commandline/cli/#sample-configuration-file) with auth credentials used to access private registries used for pulling base workspace images.

	title: Registry credentials
	title: Private base image registry credentials

	help_text: This is useful when you have base workspace images in private registries other than the above configured ones.
	help_text: This is useful when you want to use base workspace images in private registries other than the above configured ones or the [default base workspace images stored on Docker Hub](https://github.com/gitpod-io/workspace-images).

[kots] Allow users to upload a .docker/config.json file #12174

[kots] Allow users to upload a .docker/config.json file #12174

Conversation

Pothulapati commented Aug 17, 2022 • edited

Description

Related Issue(s)

How to test

Release Notes

Documentation

Werft options:

Pothulapati commented Aug 18, 2022 • edited by werft-gitpod-dev-com bot

Pothulapati Aug 18, 2022

Choose a reason for hiding this comment

mrsimonemms Aug 19, 2022

Choose a reason for hiding this comment

lucasvaltl commented Aug 18, 2022

Pothulapati commented Aug 18, 2022 • edited by werft-gitpod-dev-com bot

mrsimonemms Aug 19, 2022

Choose a reason for hiding this comment

mrsimonemms Aug 19, 2022

Choose a reason for hiding this comment

mrsimonemms left a comment

Choose a reason for hiding this comment

Pothulapati commented Aug 19, 2022

lucasvaltl Aug 19, 2022 • edited

Choose a reason for hiding this comment

Pothulapati Aug 19, 2022

Choose a reason for hiding this comment

lucasvaltl Aug 19, 2022

Choose a reason for hiding this comment

lucasvaltl Aug 19, 2022 • edited

Choose a reason for hiding this comment

lucasvaltl left a comment

Choose a reason for hiding this comment

mrsimonemms commented Aug 19, 2022

[kots] Allow users to upload a `.docker/config.json` file #12174

[kots] Allow users to upload a `.docker/config.json` file #12174

Pothulapati commented Aug 17, 2022 •

edited

Pothulapati commented Aug 18, 2022 •

edited by werft-gitpod-dev-com bot

Pothulapati commented Aug 18, 2022 •

edited by werft-gitpod-dev-com bot

lucasvaltl Aug 19, 2022 •

edited

lucasvaltl Aug 19, 2022 •

edited