New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run custom commands at build time #335
Conversation
Before commit: a `Builder` object was required to calculate the environment variables to pass to external scripts. This is a nuissance when no `Builder` is available (e.g., when loading an mfgimage, or for the upcomimg per-package commands feature). After commit: Environment variables are divided into three sets (basic, settings, slot). The sets are divided according to the inputs required to generate them. Now the appropriate variables can be calculated without a `Builder` object.
Prior to this change, ShellInteractiveCommand() would never return an error. Now, this function returns an error if the specified command exits with a nonzero status. This change makes the function consistent with the other "shell command" functions in the util package.
It is simpler to just use `sort.Slice()` rather than create a sorter type.
This commit adds the ability to run custom commands at build time. This PR adds the ability to run custom commands at build time. A package specifies custom commands in its `pkg.yml` file. There are three types of commands: 1. pre_build_cmds (run before the build) 2. pre_link_cmds (run after compilation, before linking) 3. post_build_cmds (run after the build) Example (apps/blinky/pkg.yml): ``` pkg.pre_build_cmds: scripts/pre_build1.sh: 100 scripts/pre_build2.sh: 200 pkg.pre_link_cmds: scripts/pre_link.sh: 500 pkg.post_build_cmds: scripts/post_build.sh: 100 ``` For each command, the string on the left specifies the command to run. The number on the right indicates the command's relative ordering. When newt builds this example, it performs the following sequence: * scripts/pre_build1.sh * scripts/pre_build2.sh * [compile] * scripts/pre_link.sh * [link] * scripts/post_build.sh If other packages specify custom commands, those commands would also be executed during the above sequence. For example, if another package specifies a pre command with an ordering of 150, that command would run immediately after `pre_build1.sh`. In the case of a tie, the commands are run in lexicographic order. All commands are run from the project's base directory. In the above example, the `scripts` directory is a sibling of `targets`. See <apache#335> for details.
035787e
to
63a7289
Compare
This commit adds the ability to run custom commands at build time. This PR adds the ability to run custom commands at build time. A package specifies custom commands in its `pkg.yml` file. There are three types of commands: 1. pre_build_cmds (run before the build) 2. pre_link_cmds (run after compilation, before linking) 3. post_build_cmds (run after the build) Example (apps/blinky/pkg.yml): ``` pkg.pre_build_cmds: scripts/pre_build1.sh: 100 scripts/pre_build2.sh: 200 pkg.pre_link_cmds: scripts/pre_link.sh: 500 pkg.post_build_cmds: scripts/post_build.sh: 100 ``` For each command, the string on the left specifies the command to run. The number on the right indicates the command's relative ordering. When newt builds this example, it performs the following sequence: * scripts/pre_build1.sh * scripts/pre_build2.sh * [compile] * scripts/pre_link.sh * [link] * scripts/post_build.sh If other packages specify custom commands, those commands would also be executed during the above sequence. For example, if another package specifies a pre command with an ordering of 150, that command would run immediately after `pre_build1.sh`. In the case of a tie, the commands are run in lexicographic order. All commands are run from the project's base directory. In the above example, the `scripts` directory is a sibling of `targets`. See <apache#335> for details.
63a7289
to
d1ffc46
Compare
This commit adds the ability to run custom commands at build time. This PR adds the ability to run custom commands at build time. A package specifies custom commands in its `pkg.yml` file. There are three types of commands: 1. pre_build_cmds (run before the build) 2. pre_link_cmds (run after compilation, before linking) 3. post_build_cmds (run after the build) Example (apps/blinky/pkg.yml): ``` pkg.pre_build_cmds: scripts/pre_build1.sh: 100 scripts/pre_build2.sh: 200 pkg.pre_link_cmds: scripts/pre_link.sh: 500 pkg.post_build_cmds: scripts/post_build.sh: 100 ``` For each command, the string on the left specifies the command to run. The number on the right indicates the command's relative ordering. When newt builds this example, it performs the following sequence: * scripts/pre_build1.sh * scripts/pre_build2.sh * [compile] * scripts/pre_link.sh * [link] * scripts/post_build.sh If other packages specify custom commands, those commands would also be executed during the above sequence. For example, if another package specifies a pre command with an ordering of 150, that command would run immediately after `pre_build1.sh`. In the case of a tie, the commands are run in lexicographic order. All commands are run from the project's base directory. In the above example, the `scripts` directory is a sibling of `targets`. See <apache#335> for details.
d1ffc46
to
962ce2b
Compare
We don't currently support custom commands in cmake output. Print a warning if the user runs the "target cmake" command on a target that specified custom commands.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few minor comments.
@@ -0,0 +1,235 @@ | |||
package builder |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apache license missing.
settings map[string]string) map[string]string { | ||
return lpkg.PreLinkCmds(settings) | ||
}) | ||
r.postBuildCmdCfg = extcmd.Read("post_build_cmds", lpkgs, &r.cfg, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After reading the code, I feel like this should be 'post_link_cmds' rather than 'post_build_cmds'. I think that would better communicate the order in which scripts are executed. First glance I expected order to be pre_build -> post_build -> pre_link, and I was wondering what happened with the post_link step :).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @mkiiskila. For some reason I have a preference for post_build
, but I agree there is some ambiguity there, so I did the right thing and renamed it to post_link
:).
This commit adds the ability to run custom commands at build time. This PR adds the ability to run custom commands at build time. A package specifies custom commands in its `pkg.yml` file. There are three types of commands: 1. pre_build_cmds (run before the build) 2. pre_link_cmds (run after compilation, before linking) 3. post_build_cmds (run after the build) Example (apps/blinky/pkg.yml): ``` pkg.pre_build_cmds: scripts/pre_build1.sh: 100 scripts/pre_build2.sh: 200 pkg.pre_link_cmds: scripts/pre_link.sh: 500 pkg.post_build_cmds: scripts/post_build.sh: 100 ``` For each command, the string on the left specifies the command to run. The number on the right indicates the command's relative ordering. When newt builds this example, it performs the following sequence: * scripts/pre_build1.sh * scripts/pre_build2.sh * [compile] * scripts/pre_link.sh * [link] * scripts/post_build.sh If other packages specify custom commands, those commands would also be executed during the above sequence. For example, if another package specifies a pre command with an ordering of 150, that command would run immediately after `pre_build1.sh`. In the case of a tie, the commands are run in lexicographic order. All commands are run from the project's base directory. In the above example, the `scripts` directory is a sibling of `targets`. See <#335> for details.
This PR adds the ability to run custom commands at build time.
A package specifies custom commands in its
pkg.yml
file. There are three types of commands:EXAMPLE
Example (apps/blinky/pkg.yml):
For each command, the string on the left specifies the command to run. The number on the right indicates the command's relative ordering.
When newt builds this example, it performs the following sequence:
If other packages specify custom commands, those commands would also be executed during the above sequence. For example, if another package specifies a pre command with an ordering of 150, that command would run immediately after
pre_build1.sh
. In the case of a tie, the commands are run in lexicographic order.All commands are run from the project's base directory. In the above example, the
scripts
directory is a sibling oftargets
.CUSTOM BUILD INPUTS
A custom pre-build or pre-link command can produce files that get fed into the current build.
Pre-build commands can generate any of the following:
.c
files for newt to compile..a
files for newt to link..h
files that any package can include.Pre-link commands can only generate .a files.
.c
and.a
files should be written to$MYNEWT_USER_SRC_DIR
(or any subdirectory within)..h
files should be written to$MYNEWT_USER_INCLUDE_DIR
. The directory structure used here is directly reflected by the includer. E.g., if a script writes to:then a source file can include this header with:
DETAILS
1. Environment variables
In addition to the usual environment variables defined for debug and download scripts, newt defines the following env vars for custom commands:
.a
file..o
and.a
files get written.These environment variables are defined for each process that a custom command runs in. They are not defined in the newt process itself. So, the following snippet will not produce the expected output:
BAD Example (apps/blinky/pkg.yml):
You can execute
sh
here instead if you need access to the environment variables, but it is probably saner to just use a script.2. Detect changes in custom build inputs
To avoid unnecessary rebuilds, newt detects if custom build inputs have changed since the previous build. If none of the inputs have changed, then they do not get rebuilt. If any of them of them have changed, they all get rebuilt.
The
$MYNEWT_USER_[...]
directories are actually temp directories. After the pre-build commands have run, newt compares the contents of the temp directory with those of the actual user directory. If any differences are detected, newt replaces the user directory with the temp directory, triggering a rebuild of its contents. The same procedure is used for pre-link commands.3. Paths
Custom build inputs get written to the following directories:
Custom commands should not write to these directories. They should use the
$MYNEWT_USER_[...]
environment variables instead.