-
Notifications
You must be signed in to change notification settings - Fork 188
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore: Containers reduced to ~100MB total. ~30s installation. (#3487)
TLDR: Want to test?
```
for img in aztec-sandbox cli noir; do docker pull aztecprotocol/${img}:cl_sandbox_cli_layer_share; done
```
More wonderful improvements:
* We build a common prod image (`yarn-project-prod`) shared between
sandbox and cli, so we only pull the data once for both images. The
actual cli/sandbox images now just specify how to actually run. This
gets the compressed image size down to about ~110mb. Took about 50s to
pull/extract sandbox, cli and nargo on crappy office connection (30s on
home connection).
* We introduce `buildx` for the creation of multiarch images, allowing
us to ditch the doubling of jobs and manifest creation phases of
multiarch images.
* This makes most sense for small containers. It doesn't replace the
traditional flow for large project builds such as `noir` as it uses
virtualisation.
* Multiarch is configured by a new `multiarch` property added to the
build manifest. If set to `buildx` it uses buildx virtualisation, if set
to `host` it's expected that two host machines with different arches do
the build (as traditionally).
* This also goes back to a pattern of there being *no* arch postfix by
default on image tags, instead we only apply the postfix to images
specifically where `multiarch: host` is enabled. The ecr manifest job is
then expected to immediately create the postfixless version.
* We separate some `calculate_content_hash` stuff into
`calculate_rebuild_files` to aid debugging rebuild pattern stuff.
* `deploy_dockerhub` is extensively simplified and sped up by using
`skopeo` to perform direct registry to registry multiarch image copies
and retags.
* The concept of commit messages with `[ci force-release]` is
introduced, and at present if enabled, images will pushed to dockerhub
with the normalised branch name as a tag. Npm stuff does nothing still.
* We separate the concept of `deploy` and `release`, where a release is
a publishing of artefacts, and a deploy involves deployment of services
etc in AWS.
* `noir` has stricter rebuild patterns specified to prevent aggressive
rebuilding.
* Change some sync fs calls to async.- Loading branch information
1 parent
63dd0c8
commit b49cef2
Showing
27 changed files
with
319 additions
and
288 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,21 +1,8 @@ | ||
| #!/bin/bash | ||
|
|
||
| [ -n "${BUILD_SYSTEM_DEBUG:-}" ] && set -x # conditionally trace | ||
| set -eu | ||
|
|
||
| REPOSITORY=$1 | ||
| COMMIT_HASH=${2:-${COMMIT_HASH:-$(git rev-parse HEAD)}} | ||
|
|
||
| # Get list of rebuild patterns, concat them with regex 'or' (|), and double escape \ for awk -v. | ||
| AWK_PATTERN=$(query_manifest rebuildPatterns $REPOSITORY | tr '\n' '|' | sed 's/\\/\\\\/g') | ||
| # Remove the trailing '|'. | ||
| AWK_PATTERN=${AWK_PATTERN%|} | ||
|
|
||
| cd "$(git rev-parse --show-toplevel)" | ||
| set -euo pipefail | ||
|
|
||
| # an example line is | ||
| # An example line is: | ||
| # 100644 da9ae2e020ea7fe3505488bbafb39adc7191559b 0 yarn-project/world-state/tsconfig.json | ||
| # this format is beneficial as it grabs the hashes from git efficiently | ||
| # we will next filter by our rebuild patterns | ||
| # then we pipe the hash portion of each file to git hash-object to produce our content hash | ||
| git ls-tree -r $COMMIT_HASH | awk -v pattern="($AWK_PATTERN)" '$4 ~ pattern {print $3}' | git hash-object --stdin | ||
| # Extract the hashes and pipe the hash portion of each file to git hash-object to produce our content hash. | ||
| calculate_rebuild_files "$@" | awk '{print $3}' | git hash-object --stdin |
Oops, something went wrong.