Skip to content

Conversation

@mszostok
Copy link
Collaborator

@mszostok mszostok commented Sep 3, 2021

Description

Currently, we build images on local machine, archive them, mount to kind/k3d cluster and load into cluster.
This PR adds support for local registry, so we build images on local machine and push them to local registry.
As a result, we re-use native docker support for pushing only changed layers and also re-use layers between images. Thanks to that, we reduce time spent on sharing images between localhost and k3d cluster in all scenarios (install/upgrade and when the cluster is deleted as the local registry stores data on localhost).

K3d and local registry containers are connected with the same network.

Changes proposed in this pull request:

  • Reduce the e2e test image size from 420MB to 74.9MB
  • Add opt to create local registry via capact env create k3d --enable-registry
  • Add opt to use local registry via capact install --environment k3d --enable-registry
  • Remove registry if available via capact env delete k3d
  • Handle verbose mode during building/pushing images. If v=2 not specified, output is more concise and consistent.
  • Move k3d config to dedicated tmpl file and use Go template to render it instead of fmt.Sprintf

Kind with loading images
build

Install:

✓ Building image local/argo-actions:dev [took 3s]
✓ Building image local/argo-runner:dev [took 1s]
✓ Building image local/e2e-test:dev [took 1s]
✓ Building image local/gateway:dev [took 1s]
✓ Building image local/hub-js:dev [took 2s]
✓ Building image local/k8s-engine:dev [took 2s]
✓ Building image local/populator:dev [took 3s]
✓ Loading Docker images into kind cluster [took 91s]

Upgrade: (no changes)

✓ Building image local/argo-actions:dev-5143 [took 2s]
✓ Building image local/argo-runner:dev-5143 [took 1s]
✓ Building image local/e2e-test:dev-5143 [took 1s]
✓ Building image local/gateway:dev-5143 [took 1s]
✓ Building image local/hub-js:dev-5143 [took 1s]
✓ Building image local/k8s-engine:dev-5143 [took 2s]
✓ Building image local/populator:dev-5143 [took 2s]
✓ Loading Docker images into kind cluster [took 82s]

K3d with local registry
registry

Install:

Installing Capact on cluster...
 ✓ Building image capact-registry.localhost:5000/argo-actions:dev [took 6s]
 ✓ Building image capact-registry.localhost:5000/argo-runner:dev [took 4s]
 ✓ Building image capact-registry.localhost:5000/e2e-test:dev [took 17s]
 ✓ Building image capact-registry.localhost:5000/gateway:dev [took 3s]
 ✓ Building image capact-registry.localhost:5000/hub-js:dev [took 2s]
 ✓ Building image capact-registry.localhost:5000/k8s-engine:dev [took 4s]
 ✓ Building image capact-registry.localhost:5000/populator:dev [took 6s]
 ✓ Pushing capact-registry.localhost:5000/argo-actions:dev [took 1s]
 ✓ Pushing capact-registry.localhost:5000/argo-runner:dev [took 2s]
 ✓ Pushing capact-registry.localhost:5000/e2e-test:dev [took 3s]
 ✓ Pushing capact-registry.localhost:5000/gateway:dev [took 0s]
 ✓ Pushing capact-registry.localhost:5000/hub-js:dev [took 17s]
 ✓ Pushing capact-registry.localhost:5000/k8s-engine:dev [took 2s]
 ✓ Pushing capact-registry.localhost:5000/populator:dev [took 4s]

Upgrade (no changes):

 ✓ Building image capact-registry.localhost:5000/argo-actions:dev-29541 [took 2s]
 ✓ Building image capact-registry.localhost:5000/argo-runner:dev-29541 [took 1s]
 ✓ Building image capact-registry.localhost:5000/e2e-test:dev-29541 [took 1s]
 ✓ Building image capact-registry.localhost:5000/gateway:dev-29541 [took 1s]
 ✓ Building image capact-registry.localhost:5000/hub-js:dev-29541 [took 1s]
 ✓ Building image capact-registry.localhost:5000/k8s-engine:dev-29541 [took 1s]
 ✓ Building image capact-registry.localhost:5000/populator:dev-29541 [took 2s]
 ✓ Pushing capact-registry.localhost:5000/argo-actions:dev-29541 [took 0s]
 ✓ Pushing capact-registry.localhost:5000/argo-runner:dev-29541 [took 0s]
 ✓ Pushing capact-registry.localhost:5000/e2e-test:dev-29541 [took 0s]
 ✓ Pushing capact-registry.localhost:5000/gateway:dev-29541 [took 0s]
 ✓ Pushing capact-registry.localhost:5000/hub-js:dev-29541 [took 0s]
 ✓ Pushing capact-registry.localhost:5000/k8s-engine:dev-29541 [took 0s]
 ✓ Pushing capact-registry.localhost:5000/populator:dev-29541 [took 0s]

https://asciinema.org/a/432775

Related issue(s)

@mszostok mszostok added enhancement New feature or request area/cli Relates to CLI labels Sep 3, 2021
@mszostok mszostok force-pushed the k3d-env/local-registry branch 2 times, most recently from bb51fcb to 89e418c Compare September 3, 2021 13:49
@mszostok mszostok changed the title K3d env/local registry Add local registry support for k3d environment Sep 3, 2021
@mszostok mszostok added the WIP Work in progress label Sep 3, 2021
@mszostok mszostok force-pushed the k3d-env/local-registry branch from d940c2f to 5e03ea4 Compare September 4, 2021 17:20
@mszostok mszostok removed the WIP Work in progress label Sep 4, 2021
@Trojan295 Trojan295 self-assigned this Sep 6, 2021
@Trojan295
Copy link
Contributor

Trojan295 commented Sep 6, 2021

One thing I noticed: if I don't have the registry:2 image pulled already locally, then I get an error:

❯ capact env create k3d --enable-registry
Creating cluster dev-capact...
 ✗ Creating local registry under /home/damian/.config/capact/local_registry
Error: Error response from daemon: No such image: registry:2

Executing docker pull registry:2 before solves the problem.

@mszostok mszostok force-pushed the k3d-env/local-registry branch 2 times, most recently from 489dfbb to 6305a68 Compare September 7, 2021 20:39
@mszostok
Copy link
Collaborator Author

mszostok commented Sep 7, 2021

No the image is pulled when needed 👍

For now, I added TODO comment for supporting port change, as I run into an issue with it and wasn't able to solve it with a given timebox

@mszostok mszostok force-pushed the k3d-env/local-registry branch from 6305a68 to 1c88042 Compare September 7, 2021 20:49
@mszostok mszostok merged commit aaeabac into capactio:main Sep 8, 2021
@mszostok mszostok deleted the k3d-env/local-registry branch September 8, 2021 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/cli Relates to CLI enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants