Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Post-build script execution #545

Open
vosen opened this issue Sep 10, 2014 · 50 comments
Open

Post-build script execution #545

vosen opened this issue Sep 10, 2014 · 50 comments

Comments

@vosen
Copy link

@vosen vosen commented Sep 10, 2014

Currently, cargo execute scrips before the build starts with the build field. I propose renaming build to pre_build and adding post_build (which would run after every successful build). It's useful for general postprocessing: running executable packers, zipping files, copying stuff, logging, etc.

@alexcrichton
Copy link
Member

@alexcrichton alexcrichton commented Sep 10, 2014

This will likely happen as part of an implementation of cargo install, but cargo build is not meant to be a general purpose build system for these sorts of applications.

@DiamondLovesYou
Copy link
Contributor

@DiamondLovesYou DiamondLovesYou commented Jan 2, 2015

What about integration tests? For example, PNaCl modules can be run on the host machine, but they have to be translated into a NaCl beforehand and then run with sel_ldr (from the NaCl SDK).

@jan-hudec
Copy link

@jan-hudec jan-hudec commented Mar 12, 2015

@DiamondLovesYou, would #1411 work for the tests?

@caitp
Copy link

@caitp caitp commented Jul 5, 2015

A "post_build" script would be useful for things like creating a MacOS bundle, rather than having a separate script to do that work separately. "install" could do this too, but maybe not as useful for testing there?

@vvanders
Copy link

@vvanders vvanders commented Sep 26, 2015

+1 For rust projects that linked dynamically from other languages being able to deploy(move) the .so after a successful build would be nice.

@tbelaire
Copy link

@tbelaire tbelaire commented Nov 22, 2015

This would be useful for running

arm-none-eabi-objcopy -O binary ${TARGET}.elf ${TARGET}.gba
gbafix ${TARGET}.gba

To fix up the headers and such after building my project.

@Boscop
Copy link

@Boscop Boscop commented Jun 16, 2016

+1 for post build scripts

moises-silva added a commit to friends-of-freeswitch/mod_prometheus that referenced this issue Aug 6, 2016
I'd prefer if cargo would generate mod_prometheus.so
instead of libmod_prometheus.so but I did not find
a way to tell cargo or rustc to not prefix the binary
with 'lib'.

A post-build step on cargo would be also an option if
this were implemented: rust-lang/cargo#545
@mkroman
Copy link
Contributor

@mkroman mkroman commented Sep 4, 2016

This will likely happen as part of an implementation of cargo install, but cargo build is not meant to be a general purpose build system for these sorts of applications.

It doesn't have to mean that cargo will be a general purpose build system.
An example where a post-build action could be used for cargo is triggering a build of a crate in a sub-directory once the main crate has been built.

Crate B depends on crate A, and is in a sub-directory of crate A, and it automatically gets built every time crate A has been built.

@Boscop
Copy link

@Boscop Boscop commented Sep 4, 2016

But what if crate B also depends on sub crates C and D? Shouldn't the build of dependencies be caused by building the crate that depends on them in a "by need" manner?
I would say the post build command should not involve building any related crates or calling cargo (can be easily enforced), only for post processing of the lib/exe that was just built. To keep it compartmentalized / clean.

@JinShil
Copy link

@JinShil JinShil commented Oct 29, 2016

This would be useful for embedded systems programming. I often run arm-none-eabi-size to verify my binary size with each build, and occasionally other utilities from binutils to verify symbol tables, etc...

@steveklabnik
Copy link
Member

@steveklabnik steveklabnik commented Oct 29, 2016

Don't forget that there's nothing that says you have to use a non-Rust solution for these kinds of things; any "cargo-foo" executable on your $PATH works as "cargo foo", and can be written in any language, including Rust. Your custom command could invoke "cargo build" as a step first and then do anything.

On Oct 29, 2016, 13:35 -0400, Mike notifications@github.com, wrote:

This would be useful for embedded systems programming. I often run arm-none-eabi-size to verify my binary size with each build, and occasionally other utilities from binutils to verify symbol tables, etc...


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub (#545 (comment)), or mute the thread (https://github.com/notifications/unsubscribe-auth/AABsimputy3ucYbBxCoVRqmOdAF8GQPnks5q44P0gaJpZM4CgUcP).

@JinShil
Copy link

@JinShil JinShil commented Oct 30, 2016

Don't forget that there's nothing that says you have to use a non-Rust solution for these kinds of things; any "cargo-foo" executable on your $PATH works as "cargo foo", and can be written in any language, including Rust. Your custom command could invoke "cargo build" as a step first and then do anything.

Yes, that is what I, and probably others, are already doing. It would just be more convenient to integrate this into the already familiar cargo build.

@ssokolow
Copy link

@ssokolow ssokolow commented Oct 30, 2016

Not to mention that, in my experience, if you try to enforce your will too strongly, you start to get behaviours similar to what has been standard in the home inkjet printer market for well over a decade now.

(Where the first thing you see when you open the box is a slip that warns you, in big letters, to never plug the printer in until after you've installed the software off their CD, because Microsoft's automated driver installation workflow doesn't allow them to install their fancy, tray-resident, bloated, talking print drivers and their official position is that, if Microsoft's installer has had a chance to run, your OS is now broken and must be repaired first.)

Better to have an official way to run a post-build script than to see everyone and their dog reinventing broken wheels... especially since, as a Stubborn Person™, I can easily foresee these horrors:

  • Crates which bail out early and insist that, even if they're being used as a dependency, you must run their own special cargo mybuild wrapper instead of cargo build (possibly not as an inherent thing, but as a result of depending on something else by the same author which has that build requirement).
  • Developers recognizing the ugliness of the former situation and producing a "better" solution by hacking (subtly buggy) post-builds onto cargo build using pre-build scripts which leave something running in the background to trigger the post-build using inotify or equivalent.

(Also, keep in mind that, as Rust sees more corporate uptake, we'll see more "I'm too busy to learn the 'proper' way, so I'll just hack something in" WTFs. That's just a basic rule of programming in general.)

@Kixunil
Copy link

@Kixunil Kixunil commented Jan 29, 2017

In embedded development it's useful to run objcopy to produce hex file or something similar. Adding this to post-build script would be much cleaner than reinventing wheel with my own shell script.

@yasammez
Copy link

@yasammez yasammez commented Mar 19, 2017

This would be useful for stuff like

SignTool sign /v /s PrivateCertStore /n MyTestCrt /t http://timestamp.verisign.com/scripts/timstamp.dll driver.sys

when developing Windows drivers in Testmode.

@fschutt
Copy link
Contributor

@fschutt fschutt commented Apr 2, 2017

This would be useful for packaging debian applications, especially when combined with configuration
Like a configuration deploy which builds the crate in release mode, then creates the folders needed for a package (control files, etc.) and runs dpkg on it.

@kornelski
Copy link
Contributor

@kornelski kornelski commented Apr 4, 2017

cargo install touches files outside the current directory, so it doesn't feel like a proper post-build step. If I want to strip the built binary, would I have to work on a binary already copied to /bin?

@Boscop
Copy link

@Boscop Boscop commented Apr 20, 2017

Why not do this:
If there is a postbuild.rs file, it gets executed after build. (similar to the build.rs file before.)
It would make sense to execute the script also if the build failed. The build status can be passed to it in an env var.

@Kixunil
Copy link

@Kixunil Kixunil commented Apr 22, 2017

@Boscop I like that idea but I worry that executing another binary every compilation might slow down iterative development. Maybe using postbuild-success.rs for only success, postbuild-failure.rs for only failure and postbuild.rs for both?

@Boscop
Copy link

@Boscop Boscop commented Apr 22, 2017

I think there won't really be a slowdown compared to having different scripts for success/failure because if your postbuid.rs script checks the build status first and then returns if it's not success/failure, it won't really slow things down. The postbuild.rs script won't have to be recompiled every time, only when it changes. So calling it will just incur the cost of starting an executable that immediately returns.
Having one postbuild.rs script follows the KISS principle, most projects won't even have a postbuild.rs script, and for those that will have one, I think having one script fulfills their needs.
If they notice a slowdown, we can always introduce the option to replace the postbuild.rs script by the other two later. But let's first have just the postbuild.rs and see if that's enough.

@Kixunil
Copy link

@Kixunil Kixunil commented Apr 24, 2017

That sounds reasonable.

@tanis2000
Copy link

@tanis2000 tanis2000 commented Jun 9, 2017

@Boscop idea of the postbuild.rs script sounds like a very good way of managing post build actions. Even though the general opinion seems to be split between this and avoiding to turn cargo into a general purpose make system, I feel that this won't hurt.

@JoeyAcc
Copy link

@JoeyAcc JoeyAcc commented Jun 26, 2017

Not that I'm advocating for it per se, but Cargo already is a general purpose build system due to the cargo plugins (e.g. cargo tree) being able to do whatever they want. That that is merely a user-friendly(ish) facade for "random turing-complete script" doesn't diminish that in any way.

So the real question becomes: How ergonomic do we want to make this general-purpose build system?

@ssokolow
Copy link

@ssokolow ssokolow commented Aug 25, 2017

How would that help to run something after the dependency is compiled without either forcing all downstream dependencies to also explicitly use/call cargo-make for/in their build automation or resorting to the ugly hackery I worry about where people fire off an inotify-based "watch for build completion" daemon in the pre-build script?

@JoeyAcc
Copy link

@JoeyAcc JoeyAcc commented Aug 25, 2017

After the dependency is compiled this solution wouldn't do a thing of course.
But if the dependency is a Cargo project, then Cargo can build it however specified in its Cargo.toml.

So this wouldn't help with e.g. pre-existing native (C/C++) dependencies, but it could help with some Cargo projects I think.

@ssokolow
Copy link

@ssokolow ssokolow commented Aug 25, 2017

And the whole point I've been trying to make is that, in certain contexts, there are tasks which Cargo.toml doesn't understand which must take place after the build process.

(eg. modifications to the built ELF/Mach-O/PE files where include! and friends aren't applicable because the required format for the modifications is defined by some entity outside the developer's control, such as the operating system's loader.)

To summarize my first comment, if you don't accomodate these enough, you'll get horrendous hacks from people who care more about getting work done than tying themselves in knots to make the perfect codebase. (Such as the example I mentioned of firing off something in build.rs which sits around in the background, waiting for the build artifact to appear so it can manipulate it further.)

@kornelski
Copy link
Contributor

@kornelski kornelski commented Sep 2, 2017

RFC for build system integration may be relevant to this.

@Boscop
Copy link

@Boscop Boscop commented Dec 2, 2017

What would be the best way to do post build script execution by starting a process that waits in the background; should that process use the notify crate to wait for a write event on target/release/<package name>.exe (for executables on windows)?

Would it make sense to publish a generic post build task runner tool, installable as a cargo binary?

@Kixunil
Copy link

@Kixunil Kixunil commented Dec 5, 2017

@Boscop sounds very hacky to me. Anyone wanting to implement it would spend their time better by writing PR agains Cargo.

@SoniEx2
Copy link

@SoniEx2 SoniEx2 commented Dec 9, 2017

Need to cat a zip to a bin. This would be useful, even if restricted to application/binary builds.

@gnzlbg
Copy link

@gnzlbg gnzlbg commented Feb 23, 2018

I need to do something with my binary after I build it and before it is installed.

Without a post build script cargo install installs the wrong thing.

@FredrikAleksander
Copy link

@FredrikAleksander FredrikAleksander commented Mar 26, 2018

In one of my projects, I need to generate a symbol map after building, and it is pretty annoying this cannot be done via Cargo. My current solution is using gnu make instead of Cargo, and have make call cargo when it needs to build rust. This is a really hacky solution, and it would be easily solved with a post build script

@dancrossnyc
Copy link

@dancrossnyc dancrossnyc commented Jun 18, 2018

+1 on the need for this or something very like it. I need to run objcopy on a binary after build.

@timburks
Copy link

@timburks timburks commented Jul 6, 2018

I'm also using rust to build macOS apps, currently by calling cargo from a shell script.

A suggestion that I haven't seen yet is to name the post-build script final.rs and run it after all builds, perhaps with environment variables to pass build status. This seems very non-disruptive to current usage.

[package]
# ...
build = "build.rs"
final = "final.rs"
@kornelski
Copy link
Contributor

@kornelski kornelski commented Mar 15, 2019

cc #3483

@antekone
Copy link

@antekone antekone commented Aug 7, 2019

A post-build step would be very useful in environments which require Cargo to build the project, but don't support setting post-build action by themselves, i.e. Intellij IDEA with Rust plugin.

It's possible to create bash scripts that simply invoke cargo build and then perform selected action if return code was 0, and then add this script as a launch configuration, but still it would be very convenient if selected action would be executed directly after each build of the project.

Example of such post-build command is scp'ing the generated executable to a different machine, which contains large set of sample files that are impossible to transfer to the development machine.

@nox
Copy link
Contributor

@nox nox commented Sep 30, 2019

Note that with the advent of pipelined compilation, there is also a need for a post-build-but-still-produces-artifacts-required-for-linking scripts.

For example, the script crate in Servo depends on mozjs, which depends on mozjs_sys, which invokes make to build the JavaScript engine from Firefox, SpiderMonkey.

For obvious reasons, the script crate needs the Rust bindings for SpiderMonkey, but its compilation can start before SpiderMonkey itself finishes building. Unfortunately, as SpiderMonkey is built as part of mozjs_sys' build script, the pipeline is not as good as it could be.

@ianrrees
Copy link

@ianrrees ianrrees commented Aug 7, 2020

I'm a new-to-Rust embedded developer, currently tinkering with a little ARM M0+ so there isn't much use in the ELF that cargo build currently generates. This proposal seems like it would be helpful - cargo-binutils is great, cargo objcopy --release -- -O binary firmware.bin produces my desired output at this stage. But, it seems awkward to require a different command from a simple cargo build to get the desired output.

Since this ticket is still open, I assume this is still up for consideration?

@anderejd
Copy link

@anderejd anderejd commented Aug 7, 2020

In the .NET world we have dotnet publish in addition to the regular dotnet build, perhaps an official cargo publish would be a less controversial command to allow post-build scripts for?

EDIT: I just realized my brain fart above. cargo publish is taken for uploading to crates.io. Perhaps calling it cargo package would work?

@kornelski
Copy link
Contributor

@kornelski kornelski commented Aug 7, 2020

npm has lots of scripts for various lifecycle events, including "prepublish"/"prepublishOnly", but use-cases for that are notably different from general post-build:

  • Compiling CoffeeScript source code into JavaScript.
  • Creating minified versions of JavaScript source code.
  • Fetching remote resources that your package will use.

but there's no "CoffeeRust" yet. I thought that maybe I could use a prepublish hook to pre-generate some lookup tables (to avoid building them in build.rs), but I couldn't either: I need them in cargo test too.

cargo publish can't be used for publishing compiled binaries, so tasks like stripping binaries and creating macOS framework and app bundles wouldn't be satisfied by a prepublish hook.

@steveklabnik
Copy link
Member

@steveklabnik steveklabnik commented Aug 7, 2020

@ianrrees the general idea has been stuck in "RFC hell" for a while; I am doing the same thing (roughly) as you, but using https://github.com/matklad/cargo-xtask (which links to one of the proposals for stuff in this space) as a polyfill in the meantime.

@ianrrees
Copy link

@ianrrees ianrrees commented Aug 7, 2020

@steveklabnik is there an RFC for post-build scripts? I read through RFC 2136 which @kornelski mentioned in the thread above, and didn't get the impression that this proposed change would conflict. That also made me wonder whether this feature would be big enough to warrant an RFC, assuming that the existing build script functionality weren't broken.

@steveklabnik
Copy link
Member

@steveklabnik steveklabnik commented Aug 12, 2020

If there is one, it's not one that's well-known to me.

I am not on the Cargo team, but IMHO, this does deserve an RFC, because even if it is a small feature on its own, it would need to be coherently designed alongside any other desired lifecycle hooks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.