-
Notifications
You must be signed in to change notification settings - Fork 95
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This commit adds the ability to run custom commands at build time.
A package specifies custom commands in its `pkg.yml` file. There are
two types of commands: 1) pre_cmds (run before the build), and 2)
post_cmds (run after the build).
EXAMPLE
Example (apps/blinky/pkg.yml):
pkg.pre_cmds:
scripts/pre_build1.sh: 100
scripts/pre_build2.sh: 200
pkg.post_cmds:
scripts/post_build.sh: 100
For each command, the string on the left specifies the command to run.
The number on the right indicates the command's relative ordering.
When newt builds this example, it performs the following sequence:
scripts/pre_build1.sh
scripts/pre_build2.sh
[compile]
[link]
scripts/post_build.sh
If other packages specify custom commands, those commands would also be
executed during the above sequence. For example, if another package
specifies a pre command with an ordering of 150, that command would run
immediately after `pre_build1.sh`. In the case of a tie, the commands
are run in lexicographic order.
All commands are run from the project's base directory. In the above
example, the `scripts` directory would be a sibling of `targets`.
CUSTOM BUILD INPUTS
A custom pre-build command can produce files that get fed into the
current build. A command can generate any of the following:
1) .c files for newt to compile.
2) .a files for newt to link.
3) .h files that any package can include.
.c and .a files should be written to "$MYNEWT_USER_SRC_DIR" (or any
subdirectory within).
.h files should be written to "$MYNEWT_USER_INCLUDE_DIR". The directory
structure used here is directly reflected by the includer. E.g., if a
script writes to:
$MYNEWT_USER_INCLUDE_DIR/foo/bar.h
then a source file can include this header with:
#include "foo/bar.h"
DETAILS
1. Environment variables
In addition to the usual environment variables defined for debug and
download scripts, newt defines the following env vars for custom
commands:
* MYNEWT_USER_SRC_DIR: Path where build inputs get written.
* MYNEWT_USER_INCLUDE_DIR: Path where globally-accessible headers get
written.
* MYNEWT_PKG_NAME: The full name of the package that specifies
the command being executed.
* MYNEWT_APP_BIN_DIR: The path where the current target's binary
gets written.
These environment variables are defined for each processes that a custom
command runs in. They are *not* defined in the newt process itself.
So, the following snippet will not produce the expected output:
BAD Example (apps/blinky/pkg.yml):
pkg.pre_cmds:
'echo $MYNEWT_USER_SRC_DIR': 100
You can execute `sh` here instead if you need access to the environment
variables, but it is probably saner to just use a script.
2. Detect changes in custom build inputs
To avoid unnecessary rebuilds, newt detects if custom build inputs have
changed since the previous build. If none of the inputs have changed,
then they do not get rebuilt. If any of them of them have changed, they
all get rebuilt.
The $MYNEWT_USER_[...] base is actually a temp directory. After the
pre-build commands have run, newt compares the contents of the temp
directory with those of the actual user directory. If any differences
are detected, newt replaces the user directory with the temp directory,
triggering a rebuild of its contents.
3. Paths
Custom build inputs get written to the following directories:
* bin/targets/<target>/user/src
* bin/targets/<target>/user/include
Custom commands should not write to these directories. They should use
the $MYNEWT_USER_[...] environment variables instead.- Loading branch information
1 parent
bd8f133
commit 035787e
Showing
13 changed files
with
493 additions
and
23 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| package builder | ||
|
|
||
| import ( | ||
| "io/ioutil" | ||
| "os" | ||
| "os/exec" | ||
|
|
||
| "github.com/kballard/go-shellquote" | ||
| log "github.com/sirupsen/logrus" | ||
| "mynewt.apache.org/newt/newt/stage" | ||
| "mynewt.apache.org/newt/util" | ||
| ) | ||
|
|
||
| // artifactsAreSame indicates whether the artifacts produced by the set of | ||
| // pre-build user scripts are identical to those from the previous build. If | ||
| // the artifacts are unchanged, there is no need to pass updated copies into | ||
| // the build. | ||
| func artifactsAreSame(fromSrc string, fromInc string, | ||
| toSrc string, toInc string) (bool, error) { | ||
|
|
||
| srcEq, err := util.DirsAreEqual(fromSrc, toSrc) | ||
| if err != nil { | ||
| return false, err | ||
| } | ||
| if !srcEq { | ||
| return false, nil | ||
| } | ||
|
|
||
| incEq, err := util.DirsAreEqual(fromInc, toInc) | ||
| if err != nil { | ||
| return false, err | ||
| } | ||
| if !incEq { | ||
| return false, nil | ||
| } | ||
|
|
||
| return true, nil | ||
| } | ||
|
|
||
| // execExtCmds executes a set of user scripts. | ||
| func execExtCmds(sf stage.StageFunc, env map[string]string) error { | ||
| envs := EnvVarsToSlice(env) | ||
| toks, err := shellquote.Split(sf.Name) | ||
| if err != nil { | ||
| return util.FmtNewtError( | ||
| "invalid command string: \"%s\": %s", sf.Name, err.Error()) | ||
| } | ||
|
|
||
| // If the command is in the user's PATH, expand it to its real location. | ||
| cmd, err := exec.LookPath(toks[0]) | ||
| if err == nil { | ||
| toks[0] = cmd | ||
| } | ||
|
|
||
| if err := util.ShellInteractiveCommand(toks, envs, true); err != nil { | ||
| return err | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
|
|
||
| // execPreCmds runs the target's set of pre-build user commands. It is an | ||
| // error if any command fails (exits with a nonzero status). | ||
| func (t *TargetBuilder) execPreCmds(env map[string]string) error { | ||
| // Create temporary directories where scripts can put build inputs. | ||
| tmpDir, err := ioutil.TempDir("", "mynewt-user") | ||
| if err != nil { | ||
| return util.ChildNewtError(err) | ||
| } | ||
| defer func() { | ||
| log.Debugf("removing user dir: %s", tmpDir) | ||
| os.RemoveAll(tmpDir) | ||
| }() | ||
|
|
||
| tmpSrcDir := UserTempSrcDir(tmpDir) | ||
| log.Debugf("creating user src dir: %s", tmpSrcDir) | ||
| if err := os.MkdirAll(tmpSrcDir, 0755); err != nil { | ||
| return util.ChildNewtError(err) | ||
| } | ||
|
|
||
| tmpIncDir := UserTempIncludeDir(tmpDir) | ||
| log.Debugf("creating user include dir: %s", tmpIncDir) | ||
| if err := os.MkdirAll(tmpIncDir, 0755); err != nil { | ||
| return util.ChildNewtError(err) | ||
| } | ||
|
|
||
| for _, sf := range t.res.PreCmdCfg.StageFuncs { | ||
| uenv := UserEnvVars(tmpDir, sf.Pkg, t.target.Name(), | ||
| t.appPkg.FullName()) | ||
| for k, v := range uenv { | ||
| env[k] = v | ||
| } | ||
| if err := execExtCmds(sf, env); err != nil { | ||
| return err | ||
| } | ||
| } | ||
|
|
||
| // Compare the artifacts just produced (temp directory) to those from the | ||
| // previous build (user bin directory). If they are different, replace the | ||
| // old with the new so that they get relinked during this build. | ||
|
|
||
| srcDir := UserSrcDir(t.target.Name()) | ||
| incDir := UserIncludeDir(t.target.Name()) | ||
|
|
||
| eq, err := artifactsAreSame(srcDir, incDir, tmpSrcDir, tmpIncDir) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| if !eq { | ||
| os.RemoveAll(srcDir) | ||
| if err := os.Rename(tmpSrcDir, srcDir); err != nil { | ||
| return util.ChildNewtError(err) | ||
| } | ||
|
|
||
| os.RemoveAll(incDir) | ||
| if err := os.Rename(tmpIncDir, incDir); err != nil { | ||
| return util.ChildNewtError(err) | ||
| } | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
|
|
||
| // execPostCmds runs the target's set of post-build user commands. It is an | ||
| // error if any command fails (exits with a nonzero status). | ||
| func (t *TargetBuilder) execPostCmds(env map[string]string) error { | ||
| for _, sf := range t.res.PostCmdCfg.StageFuncs { | ||
| if err := execExtCmds(sf, env); err != nil { | ||
| return err | ||
| } | ||
| } | ||
|
|
||
| return nil | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.