make BUILDSYS_OUTPUT_DIR work with overrides #963
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Ben Cressey <bcressey@amazon.com>
webern
approved these changes
Jun 30, 2020
tjkirch
approved these changes
Jun 30, 2020
Any variable that refers to another variable needs this treatment, unless it's not one we expect to override on the command line. Most of the other ones in |
zmrow
added a commit
to zmrow/bottlerocket
that referenced
this pull request
Jul 23, 2020
Simplify repo creation and tie it into the existing `cargo make` build system using a new `repo` target. This is intended to replace the common pattern of calls to tuftool and updata - either creating a repo or extending an existing one, adding the latest built artifacts, then updating the manifest to match. **Usage:** `cargo make repo` depends on the `build` target, so if your goal is building an update for a repo you don't have to separately build it first. It uses the same `BUILDSYS_VARIANT` and `BUILDSYS_ARCH` variables to determine the metadata for the update added to the repo. **Requirements:** * An Infra.toml file, based on Infra.toml.example, listing paths to keys, existing repos, etc. * Release.toml updated with a new version **Optional further configuration:** * Repo expiration policy - by default, uses a policy file with 2 week target and snapshot expiration and 1 week timestamp expiration. * Wave policy - same policy files you give to updata today; defaults to "default" wave policy. * Release start time - when waves start and when expiration starts counting down; defaults to now. * Can select which named repo and signing key to use from Infra.toml. **Design decisions:** * Built repo metadata is written to a directory like /build/repos/bottlerocket-0.4.1-5880e5d/aws-k8s-1.15/x86_64 so that you can prepare repos for multiple releases in parallel. Targets are written to a shared directory like /build/repos/bottlerocket-0.4.1-5880e5d/targets - they're unique across variants and arches so there's no conflict. The directory structure as a whole can be synced to your final repo location; it's the structure expected by Bottlerocket and updog. * buildsys uses environment variables set by cargo-make; we opted instead for more standard arg parsing. It seems more likely that someone would use pubsys separately from cargo-make, and pubsys has more input information, so arg parsing was clearer. * cargo-make environment variable expansion is done in phases, and you can't refer to a variable defined in the same section if you intend to let the user override the earlier variable on the command line. If you do, the variable won't expand, as seen in bottlerocket-os#963. Because of this, until we figure out a better strategy, a couple of variables can't be overridden - the path to Release.toml (which we made a variable in this change) and the repo output directory. Co-authored-by: Zac Mrowicki <mrowicki@amazon.com> Co-authored-by: Tom Kirchner <tjk@amazon.com>
webern
pushed a commit
to bottlerocket-os/twoliter
that referenced
this pull request
Jul 10, 2023
Simplify repo creation and tie it into the existing `cargo make` build system using a new `repo` target. This is intended to replace the common pattern of calls to tuftool and updata - either creating a repo or extending an existing one, adding the latest built artifacts, then updating the manifest to match. **Usage:** `cargo make repo` depends on the `build` target, so if your goal is building an update for a repo you don't have to separately build it first. It uses the same `BUILDSYS_VARIANT` and `BUILDSYS_ARCH` variables to determine the metadata for the update added to the repo. **Requirements:** * An Infra.toml file, based on Infra.toml.example, listing paths to keys, existing repos, etc. * Release.toml updated with a new version **Optional further configuration:** * Repo expiration policy - by default, uses a policy file with 2 week target and snapshot expiration and 1 week timestamp expiration. * Wave policy - same policy files you give to updata today; defaults to "default" wave policy. * Release start time - when waves start and when expiration starts counting down; defaults to now. * Can select which named repo and signing key to use from Infra.toml. **Design decisions:** * Built repo metadata is written to a directory like /build/repos/bottlerocket-0.4.1-5880e5d/aws-k8s-1.15/x86_64 so that you can prepare repos for multiple releases in parallel. Targets are written to a shared directory like /build/repos/bottlerocket-0.4.1-5880e5d/targets - they're unique across variants and arches so there's no conflict. The directory structure as a whole can be synced to your final repo location; it's the structure expected by Bottlerocket and updog. * buildsys uses environment variables set by cargo-make; we opted instead for more standard arg parsing. It seems more likely that someone would use pubsys separately from cargo-make, and pubsys has more input information, so arg parsing was clearer. * cargo-make environment variable expansion is done in phases, and you can't refer to a variable defined in the same section if you intend to let the user override the earlier variable on the command line. If you do, the variable won't expand, as seen in bottlerocket-os/bottlerocket#963. Because of this, until we figure out a better strategy, a couple of variables can't be overridden - the path to Release.toml (which we made a variable in this change) and the repo output directory. Co-authored-by: Zac Mrowicki <mrowicki@amazon.com> Co-authored-by: Tom Kirchner <tjk@amazon.com>
webern
pushed a commit
to bottlerocket-os/twoliter
that referenced
this pull request
Aug 11, 2023
Simplify repo creation and tie it into the existing `cargo make` build system using a new `repo` target. This is intended to replace the common pattern of calls to tuftool and updata - either creating a repo or extending an existing one, adding the latest built artifacts, then updating the manifest to match. **Usage:** `cargo make repo` depends on the `build` target, so if your goal is building an update for a repo you don't have to separately build it first. It uses the same `BUILDSYS_VARIANT` and `BUILDSYS_ARCH` variables to determine the metadata for the update added to the repo. **Requirements:** * An Infra.toml file, based on Infra.toml.example, listing paths to keys, existing repos, etc. * Release.toml updated with a new version **Optional further configuration:** * Repo expiration policy - by default, uses a policy file with 2 week target and snapshot expiration and 1 week timestamp expiration. * Wave policy - same policy files you give to updata today; defaults to "default" wave policy. * Release start time - when waves start and when expiration starts counting down; defaults to now. * Can select which named repo and signing key to use from Infra.toml. **Design decisions:** * Built repo metadata is written to a directory like /build/repos/bottlerocket-0.4.1-5880e5d/aws-k8s-1.15/x86_64 so that you can prepare repos for multiple releases in parallel. Targets are written to a shared directory like /build/repos/bottlerocket-0.4.1-5880e5d/targets - they're unique across variants and arches so there's no conflict. The directory structure as a whole can be synced to your final repo location; it's the structure expected by Bottlerocket and updog. * buildsys uses environment variables set by cargo-make; we opted instead for more standard arg parsing. It seems more likely that someone would use pubsys separately from cargo-make, and pubsys has more input information, so arg parsing was clearer. * cargo-make environment variable expansion is done in phases, and you can't refer to a variable defined in the same section if you intend to let the user override the earlier variable on the command line. If you do, the variable won't expand, as seen in bottlerocket-os/bottlerocket#963. Because of this, until we figure out a better strategy, a couple of variables can't be overridden - the path to Release.toml (which we made a variable in this change) and the repo output directory. Co-authored-by: Zac Mrowicki <mrowicki@amazon.com> Co-authored-by: Tom Kirchner <tjk@amazon.com>
webern
pushed a commit
to webern/twoliter
that referenced
this pull request
Aug 14, 2023
Simplify repo creation and tie it into the existing `cargo make` build system using a new `repo` target. This is intended to replace the common pattern of calls to tuftool and updata - either creating a repo or extending an existing one, adding the latest built artifacts, then updating the manifest to match. **Usage:** `cargo make repo` depends on the `build` target, so if your goal is building an update for a repo you don't have to separately build it first. It uses the same `BUILDSYS_VARIANT` and `BUILDSYS_ARCH` variables to determine the metadata for the update added to the repo. **Requirements:** * An Infra.toml file, based on Infra.toml.example, listing paths to keys, existing repos, etc. * Release.toml updated with a new version **Optional further configuration:** * Repo expiration policy - by default, uses a policy file with 2 week target and snapshot expiration and 1 week timestamp expiration. * Wave policy - same policy files you give to updata today; defaults to "default" wave policy. * Release start time - when waves start and when expiration starts counting down; defaults to now. * Can select which named repo and signing key to use from Infra.toml. **Design decisions:** * Built repo metadata is written to a directory like /build/repos/bottlerocket-0.4.1-5880e5d/aws-k8s-1.15/x86_64 so that you can prepare repos for multiple releases in parallel. Targets are written to a shared directory like /build/repos/bottlerocket-0.4.1-5880e5d/targets - they're unique across variants and arches so there's no conflict. The directory structure as a whole can be synced to your final repo location; it's the structure expected by Bottlerocket and updog. * buildsys uses environment variables set by cargo-make; we opted instead for more standard arg parsing. It seems more likely that someone would use pubsys separately from cargo-make, and pubsys has more input information, so arg parsing was clearer. * cargo-make environment variable expansion is done in phases, and you can't refer to a variable defined in the same section if you intend to let the user override the earlier variable on the command line. If you do, the variable won't expand, as seen in bottlerocket-os/bottlerocket#963. Because of this, until we figure out a better strategy, a couple of variables can't be overridden - the path to Release.toml (which we made a variable in this change) and the repo output directory. Co-authored-by: Zac Mrowicki <mrowicki@amazon.com> Co-authored-by: Tom Kirchner <tjk@amazon.com>
webern
pushed a commit
to webern/twoliter
that referenced
this pull request
Aug 17, 2023
Simplify repo creation and tie it into the existing `cargo make` build system using a new `repo` target. This is intended to replace the common pattern of calls to tuftool and updata - either creating a repo or extending an existing one, adding the latest built artifacts, then updating the manifest to match. **Usage:** `cargo make repo` depends on the `build` target, so if your goal is building an update for a repo you don't have to separately build it first. It uses the same `BUILDSYS_VARIANT` and `BUILDSYS_ARCH` variables to determine the metadata for the update added to the repo. **Requirements:** * An Infra.toml file, based on Infra.toml.example, listing paths to keys, existing repos, etc. * Release.toml updated with a new version **Optional further configuration:** * Repo expiration policy - by default, uses a policy file with 2 week target and snapshot expiration and 1 week timestamp expiration. * Wave policy - same policy files you give to updata today; defaults to "default" wave policy. * Release start time - when waves start and when expiration starts counting down; defaults to now. * Can select which named repo and signing key to use from Infra.toml. **Design decisions:** * Built repo metadata is written to a directory like /build/repos/bottlerocket-0.4.1-5880e5d/aws-k8s-1.15/x86_64 so that you can prepare repos for multiple releases in parallel. Targets are written to a shared directory like /build/repos/bottlerocket-0.4.1-5880e5d/targets - they're unique across variants and arches so there's no conflict. The directory structure as a whole can be synced to your final repo location; it's the structure expected by Bottlerocket and updog. * buildsys uses environment variables set by cargo-make; we opted instead for more standard arg parsing. It seems more likely that someone would use pubsys separately from cargo-make, and pubsys has more input information, so arg parsing was clearer. * cargo-make environment variable expansion is done in phases, and you can't refer to a variable defined in the same section if you intend to let the user override the earlier variable on the command line. If you do, the variable won't expand, as seen in bottlerocket-os/bottlerocket#963. Because of this, until we figure out a better strategy, a couple of variables can't be overridden - the path to Release.toml (which we made a variable in this change) and the repo output directory. Co-authored-by: Zac Mrowicki <mrowicki@amazon.com> Co-authored-by: Tom Kirchner <tjk@amazon.com>
webern
pushed a commit
to webern/twoliter
that referenced
this pull request
Aug 24, 2023
Simplify repo creation and tie it into the existing `cargo make` build system using a new `repo` target. This is intended to replace the common pattern of calls to tuftool and updata - either creating a repo or extending an existing one, adding the latest built artifacts, then updating the manifest to match. **Usage:** `cargo make repo` depends on the `build` target, so if your goal is building an update for a repo you don't have to separately build it first. It uses the same `BUILDSYS_VARIANT` and `BUILDSYS_ARCH` variables to determine the metadata for the update added to the repo. **Requirements:** * An Infra.toml file, based on Infra.toml.example, listing paths to keys, existing repos, etc. * Release.toml updated with a new version **Optional further configuration:** * Repo expiration policy - by default, uses a policy file with 2 week target and snapshot expiration and 1 week timestamp expiration. * Wave policy - same policy files you give to updata today; defaults to "default" wave policy. * Release start time - when waves start and when expiration starts counting down; defaults to now. * Can select which named repo and signing key to use from Infra.toml. **Design decisions:** * Built repo metadata is written to a directory like /build/repos/bottlerocket-0.4.1-5880e5d/aws-k8s-1.15/x86_64 so that you can prepare repos for multiple releases in parallel. Targets are written to a shared directory like /build/repos/bottlerocket-0.4.1-5880e5d/targets - they're unique across variants and arches so there's no conflict. The directory structure as a whole can be synced to your final repo location; it's the structure expected by Bottlerocket and updog. * buildsys uses environment variables set by cargo-make; we opted instead for more standard arg parsing. It seems more likely that someone would use pubsys separately from cargo-make, and pubsys has more input information, so arg parsing was clearer. * cargo-make environment variable expansion is done in phases, and you can't refer to a variable defined in the same section if you intend to let the user override the earlier variable on the command line. If you do, the variable won't expand, as seen in bottlerocket-os/bottlerocket#963. Because of this, until we figure out a better strategy, a couple of variables can't be overridden - the path to Release.toml (which we made a variable in this change) and the repo output directory. Co-authored-by: Zac Mrowicki <mrowicki@amazon.com> Co-authored-by: Tom Kirchner <tjk@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue number:
N/A
Description of changes:
In 805bef2 we added a separate directory for images, but the component variables were not correctly expanded when overridden on the command line. Now they are.
Testing done:
Terms of contribution:
By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.