Let Cargo packages modify the state of the build environment for other (non dependent) packages.

[TyOverby]

Build scripts are heavily dependent on the state of the users machine. Like programs that rely heavily on mutable state, making gaurantees that their machine is in the correct state to build a program is difficult. With a package manager, much of this difficulty goes away when the state that you care about is "this rust library needs to be installed". But when we care about "this C library needs to be installed", and a rust dependency requires that state before it can even compile, then you can either:

• Force the users of that library to configure their machine correctly
• Force the library author to build and install the C library
• Let the user configure their project to build and install the C library.

I'm trying for an approach where by including another dependency in their Cargo.toml file, they could have the native libraries built that are required by their other rust dependencies.

Title was: Export multiple C dynamic and static libraries
I have a project where the entire purpose is to compile the C and C++ dependencies required for various projects in the Piston project. So instead of telling the user to "download and build sdl2", it will download and attempt to automate that as much as possible with the goal of either successfully installing the whole thing without user interaction, or failing with a helpful error message and steps that the user can take to resolve those errors.

Right now I have it automated such that when I run cargo build it runs the build step which runs my compile.sh which downloads and compiles all the projects and throws all library files in ./lib/.

My question is, is it possible to simply throw these static and dynamic library files in a directory (/target/???) and have them available to link with the rest of the libraries in the compilation?

Right now the process involves installing these libraries at the system level (sudo make install), and I'd like to be able to build and install and link them locally.

[alexcrichton]

The build step has the environment variables DEPS_DIR and OUT_DIR set for the script which is the location that the outputs are expected to go. Cargo will always have those two directories on the link path to rustc, but installation is a fairly separate story.

There's currently no equivalent of cargo install, but that would definitely need to take into account installing native dynamic libraries next to any rust libraries.

Does that help?

[TyOverby]

Thanks @alexcrichton, that might solve my problems. When I copy the library files over into $OUT_DIR, will those show up for other dependencies in my Cargo.toml list?

For example

...
[dependencies.native_code] # Produces foo.so
git = "..."

[dependencies.rust_code] # Needs foo.so
git = "..."

Will this setup work? Or will I have to convince the author of rust_code to depend on my native_code package?

I really hope that either that setup works, or Cargo adopt a new directive next to dependencies (install maybe?) that is used to create the global state of the build before dependencies are compiled.

[alexcrichton]

I'm not sure how that setup would work. If the package rust_code depends on native_code, then there's currently no way for you to inject a dependency, and I don't think that one is planned.

There's a number of issues around a command such as cargo install, and I imagine that we'll cross that bridge when we get there.

[alexcrichton]

When I copy the library files over into $OUT_DIR, will those show up for other dependencies in my Cargo.toml list?

No, currently the build command is invoked before a package is built, but that's the only current state of affairs.

[TyOverby]

I suppose the issue that I'm having is this: Build scripts are heavily dependent on the state of the users machine. Like programs that rely heavily on mutable state, making gaurantees that their machine is in the correct state to build a program is difficult. With a package manager, much of this difficulty goes away when the state that you care about is "this rust library needs to be installed". But when we care about "this C library needs to be installed", and a rust dependency requires that state before it can even compile, then you can either:

• Force the users of that library to configure their machine correctly
• Force the library author to have his cargo build step build and install the C library
• Let the user configure their project to build and install the C library.

I'm trying for an approach where by including another dependency in their Cargo.toml file, they could have the native libraries built that are required by their other rust dependencies.

[alexcrichton]

I believe that having another dependency would always be possible, but this would imply that you need some means of saying that package foo didn't explicitly depend on package bar, but for ordering reasons you want foo to depend on bar.

Basically, you have to ensure that the dependencies are built in the right order, and cargo does not currently allow a method to do that without modifying the source package.

This seems like a fairly different request from the original issue description/title, would you be ok opening a separate issue with this feature request? Although I may have also overlooked something!

[TyOverby]

Haha, maybe I just got better at describing my problem!

I believe that having another dependency would always be possible, but this would imply that you need some means of saying that package foo didn't explicitly depend on package bar, but for ordering reasons you want foo to depend on bar.

Yep that's exactly what I want. But in this example bar is a package that builds C libraries.

Basically, you have to ensure that the dependencies are built in the right order, and cargo does not currently allow a method to do that without modifying the source package.

Yep. Would it make sense for packages to have optional "package priority"? Or maybe the user could specify the order? I'm leaning towards the first option, but I could see how that would get out of hand.

This seems like a fairly different request from the original issue description/title, would you be ok opening a separate issue with this feature request? Although I may have also overlooked something!

Hmm... Maybe I should just change the title? I'm open to suggestions.

[alexcrichton]

Hmm... Maybe I should just change the title? I'm open to suggestions.

Either way is ok, but be sure to keep the original title/description somewhere to maintain the history!

Yep. Would it make sense for packages to have optional "package priority"? Or maybe the user could specify the order?

Interesting! These both seem like interesting ideas!

[TyOverby]

Well, now that I know that it's somewhat possible, I've updated the title and description.

[pnkfelix]

From the description here, it sounds like the current way @TyOverby is planning to solve their problem is to make the build = script copy the native library files into $OUT_DIR. (This approach seemed to solve my problem as well, at least for local hacking with sdl2.)

It seems like a potentially more flexible approach would be to allow the build script to communicate back to cargo the set of -L directives that cargo should add to its rustc invocations. E.g. if the build script were expected to have some standard output format (or to produce such output to a dedicated file that cargo could read in after the build script runs).

Any thoughts on the options here?

[alexcrichton]

That is definitely a possibility, and seems pretty reasonable. Cargo is not currently super ambitious when building native code as it basically only has the bare-bones interface of build + -L. I'd want to sit down and consider all options like this before pushing forward and implementing an arbitrary one, however.

[TyOverby]

@pnkfelix: That's exactly what I was trying to do.

[alexcrichton]

Times have come a long way since this was opened and with the now landed build script support (#792), I'm going to close this for now as I think this is basically implemented to the extent that we'd like to implement it for now.

Specifically @pnkfelix's suggestion is now implemented where build scripts can provide feedback to the compiler about where native dependencies are located.