Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add task to support running apko builds via Tekton. #5

Merged
merged 1 commit into from Feb 21, 2022

Conversation

mattmoor
Copy link
Member

@mattmoor mattmoor commented Feb 19, 2022

To build/install the task definition (w/ apko) you can run:

KO_DOCKER_REPO=ghcr.io/mattmoor ko apply -Bf config/

This task definition implements the interface defined in github.com/mattmoor/mink
to support uploading local source and directing where to publish images (so it can
be used with mink apply as well).

You can invoke this task imperatively with the following mink command:

$ mink run task apko --as=apko -- --path=./examples/nginx.yaml

2022/02/19 19:21:58 building image 'gcr.io/mattmoor-chainguard/images' from config file './examples/nginx.yaml'
2022/02/19 19:21:58 build context:
2022/02/19 19:21:58   image configuration: {{[https://dl-cdn.alpinelinux.org/alpine/edge/main] [/etc/apk/keys/alpine-devel@lists.alpinelinux.org-4a6a0840.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5243ef4b.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-6165ee59.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-61666e3f.rsa.pub] [alpine-baselayout nginx]} {service-bundle  map[nginx:/usr/sbin/nginx -c /etc/nginx/nginx.conf -g "daemon off;"]}}
2022/02/19 19:21:58   working directory: /tmp/apko-2587090319
2022/02/19 19:21:58   tarball path:
2022/02/19 19:21:58 doing pre-flight checks
2022/02/19 19:21:58 building image fileystem in /tmp/apko-2587090319
2022/02/19 19:21:58 initializing apk database
2022/02/19 19:21:58 running: /sbin/apk add --initdb --root /tmp/apko-2587090319
2022/02/19 19:21:58 [apk] OK: 0 MiB in 0 packages
2022/02/19 19:21:58 initializing apk keyring
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-4a6a0840.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5243ef4b.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-6165ee59.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-61666e3f.rsa.pub
2022/02/19 19:21:58 initializing apk repositories
2022/02/19 19:21:58 initializing apk world
2022/02/19 19:21:58 synchronizing with desired apk world
2022/02/19 19:21:58 running: /sbin/apk fix --root /tmp/apko-2587090319 --no-cache --update-cache
2022/02/19 19:21:58 [apk] fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
(1/12) Installing musl (1.2.2-r7)
(2/12) Installing busybox (1.35.0-r2)
Executing busybox-1.35.0-r2.post-install
(3/12) Installing alpine-baselayout (3.2.0-r19)
Executing alpine-baselayout-3.2.0-r19.pre-install
Executing alpine-baselayout-3.2.0-r19.post-install
(4/12) Installing libcrypto1.1 (1.1.1m-r2)
(5/12) Installing pcre (8.45-r2)
(6/12) Installing libssl1.1 (1.1.1m-r2)
(7/12) Installing zlib (1.2.11-r3)
(8/12) Installing nginx (1.20.2-r0)
Executing nginx-1.20.2-r0.pre-install
Executing nginx-1.20.2-r0.post-install
(9/12) Installing skalibs (2.11.1.0-r0)
(10/12) Installing s6-ipcserver (2.11.0.1-r0)
(11/12) Installing execline (2.8.2.0-r0)
(12/12) Installing s6 (2.11.0.1-r0)
Executing s6-2.11.0.1-r0.pre-install
Executing busybox-1.35.0-r2.trigger
OK: 8 MiB in 12 packages
2022/02/19 19:21:58 generating supervision tree
2022/02/19 19:21:58 simple service: nginx => /usr/sbin/nginx -c /etc/nginx/nginx.conf -g "daemon off;"
2022/02/19 19:21:58   supervision dir: /tmp/apko-2587090319/sv/nginx
2022/02/19 19:21:58 finished building filesystem in /tmp/apko-2587090319
2022/02/19 19:21:59 built image layer tarball as /tmp/apko-1984638129.tar.gz
2022/02/19 19:21:59 building OCI image 'gcr.io/mattmoor-chainguard/images' from layer '/tmp/apko-1984638129.tar.gz'
2022/02/19 19:21:59 OCI layer digest: sha256:5b6a06005c97c31df81455008c3c92078a50c8b07bc394a98c2519f2945c47ff
2022/02/19 19:21:59 OCI layer diffID: sha256:775a08d0e0b33a95dff11e6ffe1511605e9e6ef2646ab39353deeb3344b75577
2022/02/19 19:21:59 output OCI image file to foo.tar.gz

2022/02/19 19:22:02 pushed blob: sha256:3ed57839e782bc6b1c084f25433ef6d92647ec9906070d2ecc48ec31cfa84650
2022/02/19 19:22:02 pushed blob: sha256:5b6a06005c97c31df81455008c3c92078a50c8b07bc394a98c2519f2945c47ff
2022/02/19 19:22:03 gcr.io/mattmoor-chainguard/images: digest: sha256:a2fdd03e2be33056b469fe472917e1c2d19670de04da7963e566cd078771c02e size: 427

gcr.io/mattmoor-chainguard/images@sha256:a2fdd03e2be33056b469fe472917e1c2d19670de04da7963e566cd078771c02e

@mattmoor
Copy link
Member Author

mattmoor commented Feb 19, 2022

It's notable that I configured the apko serviceaccount with GKE workload identity to authorize pushing.

I'm still trying to sort out what port the nginx serves on, so I can try composing this with kn service create nginx --image=$(mink run task apko -- --path=./examples/nginx.yaml) (as well as showing mink apply) 🤩

@mattmoor
Copy link
Member Author

mattmoor commented Feb 19, 2022

Ah it's just 80 (not 8080): https://nginx.default.mattmoor.io

@mattmoor
Copy link
Member Author

mattmoor commented Feb 19, 2022

(I'm taking it down, since it's surely running as root)

To build/install the task definition (w/ apko) you can run:
```
KO_DOCKER_REPO=ghcr.io/mattmoor ko apply -Bf config/
```

This task definition implements the interface defined in `github.com/mattmoor/mink`
to support uploading local source and directing where to publish images (so it can
be used with `mink apply` as well).

You can invoke this task imperatively with the following `mink` command:
```
$ mink run task apko --as=apko -- --path=./examples/nginx.yaml

2022/02/19 19:21:58 building image 'gcr.io/mattmoor-chainguard/images' from config file './examples/nginx.yaml'
2022/02/19 19:21:58 build context:
2022/02/19 19:21:58   image configuration: {{[https://dl-cdn.alpinelinux.org/alpine/edge/main] [/etc/apk/keys/alpine-devel@lists.alpinelinux.org-4a6a0840.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5243ef4b.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-6165ee59.rsa.pub /etc/apk/keys/alpine-devel@lists.alpinelinux.org-61666e3f.rsa.pub] [alpine-baselayout nginx]} {service-bundle  map[nginx:/usr/sbin/nginx -c /etc/nginx/nginx.conf -g "daemon off;"]}}
2022/02/19 19:21:58   working directory: /tmp/apko-2587090319
2022/02/19 19:21:58   tarball path:
2022/02/19 19:21:58 doing pre-flight checks
2022/02/19 19:21:58 building image fileystem in /tmp/apko-2587090319
2022/02/19 19:21:58 initializing apk database
2022/02/19 19:21:58 running: /sbin/apk add --initdb --root /tmp/apko-2587090319
2022/02/19 19:21:58 [apk] OK: 0 MiB in 0 packages
2022/02/19 19:21:58 initializing apk keyring
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-4a6a0840.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5243ef4b.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-6165ee59.rsa.pub
2022/02/19 19:21:58 installing key /etc/apk/keys/alpine-devel@lists.alpinelinux.org-61666e3f.rsa.pub
2022/02/19 19:21:58 initializing apk repositories
2022/02/19 19:21:58 initializing apk world
2022/02/19 19:21:58 synchronizing with desired apk world
2022/02/19 19:21:58 running: /sbin/apk fix --root /tmp/apko-2587090319 --no-cache --update-cache
2022/02/19 19:21:58 [apk] fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
(1/12) Installing musl (1.2.2-r7)
(2/12) Installing busybox (1.35.0-r2)
Executing busybox-1.35.0-r2.post-install
(3/12) Installing alpine-baselayout (3.2.0-r19)
Executing alpine-baselayout-3.2.0-r19.pre-install
Executing alpine-baselayout-3.2.0-r19.post-install
(4/12) Installing libcrypto1.1 (1.1.1m-r2)
(5/12) Installing pcre (8.45-r2)
(6/12) Installing libssl1.1 (1.1.1m-r2)
(7/12) Installing zlib (1.2.11-r3)
(8/12) Installing nginx (1.20.2-r0)
Executing nginx-1.20.2-r0.pre-install
Executing nginx-1.20.2-r0.post-install
(9/12) Installing skalibs (2.11.1.0-r0)
(10/12) Installing s6-ipcserver (2.11.0.1-r0)
(11/12) Installing execline (2.8.2.0-r0)
(12/12) Installing s6 (2.11.0.1-r0)
Executing s6-2.11.0.1-r0.pre-install
Executing busybox-1.35.0-r2.trigger
OK: 8 MiB in 12 packages
2022/02/19 19:21:58 generating supervision tree
2022/02/19 19:21:58 simple service: nginx => /usr/sbin/nginx -c /etc/nginx/nginx.conf -g "daemon off;"
2022/02/19 19:21:58   supervision dir: /tmp/apko-2587090319/sv/nginx
2022/02/19 19:21:58 finished building filesystem in /tmp/apko-2587090319
2022/02/19 19:21:59 built image layer tarball as /tmp/apko-1984638129.tar.gz
2022/02/19 19:21:59 building OCI image 'gcr.io/mattmoor-chainguard/images' from layer '/tmp/apko-1984638129.tar.gz'
2022/02/19 19:21:59 OCI layer digest: sha256:5b6a06005c97c31df81455008c3c92078a50c8b07bc394a98c2519f2945c47ff
2022/02/19 19:21:59 OCI layer diffID: sha256:775a08d0e0b33a95dff11e6ffe1511605e9e6ef2646ab39353deeb3344b75577
2022/02/19 19:21:59 output OCI image file to foo.tar.gz

2022/02/19 19:22:02 pushed blob: sha256:3ed57839e782bc6b1c084f25433ef6d92647ec9906070d2ecc48ec31cfa84650
2022/02/19 19:22:02 pushed blob: sha256:5b6a06005c97c31df81455008c3c92078a50c8b07bc394a98c2519f2945c47ff
2022/02/19 19:22:03 gcr.io/mattmoor-chainguard/images: digest: sha256:a2fdd03e2be33056b469fe472917e1c2d19670de04da7963e566cd078771c02e size: 427

gcr.io/mattmoor-chainguard/images@sha256:a2fdd03e2be33056b469fe472917e1c2d19670de04da7963e566cd078771c02e
```
@mattmoor
Copy link
Member Author

mattmoor commented Feb 19, 2022

Alright, I was able to mink apply the following:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: nginx
spec:
  template:
    spec:
      containers:
      - image: task://apko?path=./examples/nginx.yaml
        ports:
        - containerPort: 80

With:

$ mink apply --as=apko -f examples/nginx.ksvc.yaml 
service.serving.knative.dev/nginx created

- output.tar.gz
- $(params["dev.mink.images.target"])

# mink wants just the digest in its result, but the above produces
Copy link
Contributor

@imjasonh imjasonh Feb 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should mink take the Tekton form if given? That seems to be the thing we're recommending.

Copy link
Member Author

@mattmoor mattmoor Feb 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems to be the thing we're recommending

By "recommending" I assume you mean "The awful spelunking Chains does today", or is there something else?

... it was my understanding from the way the discussion on tektoncd/community#504 went that Tekton has no such official "recommendations" today (this is what I was trying to fix, but I gave up).

Copy link
Contributor

@imjasonh imjasonh Feb 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, yeah, chains' spelunking.

Whether or not it's a documented recommended format, yet, that seems to be the way we're recommending tools produce their output. It seems like a useful thing for mink to get on board with (and it strengthens the case for making it a documented recommendation)

Copy link
Contributor

@imjasonh imjasonh Feb 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can send a PR for mink when I get a chance.

Copy link
Member Author

@mattmoor mattmoor Feb 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imjasonh I'd MUCH rather see us do something better in Chains (and Tekton in general), and I don't think that ship has sailed. Part of my desire for a stronger convention upstream for these things is that when matching Task interface characteristics, domain-scoping makes things (relatively) unambiguous. I don't think IMAGES meets that bar.

image: ko://chainguard.dev/apko/cmd/apko
workingDir: /workspace
args:
- build
Copy link
Contributor

@imjasonh imjasonh Feb 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there plans to have apko push by itself?

Copy link
Member Author

@mattmoor mattmoor Feb 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly, but the beauty of the task is that we can just change it to use that once it can 😉

... or perhaps we allow the apko step to run hermetically (it would likely need some sort of prefetch step) and keep things this way.

Copy link
Member

@kaniini kaniini Feb 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, apko publish is coming next.

Copy link
Member Author

@mattmoor mattmoor Feb 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's gonna be sweet. We should add in all of @imjasonh cool universal cred helper stuff 🤩

@imjasonh
Copy link
Contributor

imjasonh commented Feb 19, 2022

(I'm taking it down, since it's surely running as root)

Aw you're no fun.

- output.tar.gz

- name: push-image
# This is a version of "crane" with k8schain linked as
Copy link
Contributor

@imjasonh imjasonh Feb 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not technically true since google/go-containerregistry#1250

Copy link
Member Author

@mattmoor mattmoor Feb 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I know, it is just a lot shorter to write 😅

@kaniini
Copy link
Member

kaniini commented Feb 20, 2022

(I'm taking it down, since it's surely running as root)

the Alpine nginx configuration runs as nginx, even in this configuration :)

@kaniini
Copy link
Member

kaniini commented Feb 20, 2022

This is really neat! I see no problems with it, so feel free to merge it when the nitpicks are figured out :)

.ko.yaml Show resolved Hide resolved
@kaniini kaniini merged commit f6cea28 into chainguard-dev:main Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants