Dockerfree build and ecs-init make integration #3149
Conversation
agent-container/agent-config.json
Outdated
| @@ -0,0 +1,2 @@ | |||
| {"author":"Amazon Web Services, Inc.","config":{"Cmd":["/agent"],"ArgsEscaped":true},"created":"2014-12-12T01:12:53.332832423Z","history":[{"created":"2014-12-12T01:12:53.332832423Z","author":"Amazon Web Services, Inc.","created_by":"[] + [] === \"\"","empty_layer":true}],"os":"linux","rootfs":{"type":"layers","diff_ids":["sha256:~~digest~~"]}} | |||
There was a problem hiding this comment.
why are these static files? shouldn't they be generated at build-time?
There was a problem hiding this comment.
The diff_ids field is being generated at build-time. Otherwise I agree maybe we can update the history.created with an actual timestamp.
These are generated by Docker otherwise -- if we don't have Docker, we need to create/provide these config files manually.
Bottlerocket uses static configs to build its pause container: https://github.com/bottlerocket-os/bottlerocket/blob/develop/packages/ecs-agent/pause-config.json
There was a problem hiding this comment.
So do we have to ensure these files get rebuilt/committed each time then from current agent changes? Might need some sort of check to reinforce that
There was a problem hiding this comment.
Updated to string replace and use the date util to match expected formatting: sed -i "s/~~timestamp~~/$(date +"%FT%T.%NZ")/g" image/config.json
Otherwise we'll use https://github.com/aws/amazon-ecs-agent/blob/master/VERSION to fill in our version.
| DIGEST=$(sha256sum image/rootfs/layer.tar | sed -e 's/ .*//') | ||
| install -m 0644 ./agent-container/agent-image-VERSION image/rootfs/VERSION | ||
| install -m 0644 ./agent-container/agent-config.json image/config.json | ||
| sed -i "s/~~digest~~/${DIGEST}/" image/config.json |
There was a problem hiding this comment.
is it possible for us to use jq inside of the koji build? if it is we could maybe turn these into jq templates? That way the json is always guaranteed to be valid. It looks something like this:
template.jq
{
"descriptionContentType": "text/plain",
"title": $title,
"description": "$description"
}
then pass the parameters via jq like:
jq -n --arg description "My Description" --arg title "My Title" -f template.jq
There was a problem hiding this comment.
I don't want to add any dependencies unless it's absolutely necessary. I updated to use sed to replace the tokens.
| # requires glibc-static | ||
|
|
||
| dockerfree-pause: | ||
| GOOS=linux GOARCH=amd64 ./scripts/build-pause |
There was a problem hiding this comment.
can you clarify why you are setting GOOS/GOARCH here? What if this is being built on arm?
There was a problem hiding this comment.
there's still more work to build arm -- I'll update the description to say this is focused on amd linux.
| then | ||
| cp $thisCertFile . | ||
| else | ||
| exit 1 |
There was a problem hiding this comment.
maybe echo some sort of error message to help debug in case this happens
|
can you update the title? Now it's in review :D |
fierlion
changed the title
* Resolve go modules/vendor conflicts and add dockerfree-all make target * add build-time configuration and update scripts/comments
* Resolve go modules/vendor conflicts and add dockerfree-all make target * add build-time configuration and update scripts/comments
* Resolve go modules/vendor conflicts and add dockerfree-all make target * add build-time configuration and update scripts/comments
Summary
This integrates ecs-init specific Make targets and also adds docker-free targets to build the pause container and agent container without Docker. This will allow us to build ECS RPMs in multiple constrained/secure build environments (like Koji).
Implementation details
Each of the more complex make targets calls an external bash script in
/scripts. Otherwise, most of the init-specific targets are simple enough to be inlined.All integration changes are additive so they will not mutate or interrupt our existing processes.
The changes were built and tested on an EC2 instance based on the latest ECS-Optimized AMI.
The changes were tested with go version 1.17.2. There's still more work to make the changes compatible with older versions of golang.
Testing
All make targets have been tested manually. The init specific targets have been integrated into the github workflow testing
New tests cover the changes: yes -- github init-specific workflows added.
Description for the changelog
Add docker-free make targets and integrate generic rpm build.
Licensing
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.