System dependencies

[cmr]

Fairly long discussion: https://gist.github.com/cmr/10e1633ba7e9569af4b9

The takeway, I think, is that there will be a pkg-* class of packages. These will integrate with pkg-config to query how to link to a given system dependency, with the ability to fall back on a bundled version that is then built.

[alexcrichton]

@wycats and I also discussed this at great length, and we ended up coming to a very similar but slightly tweaked solution. The main idea is a syntax extension: https://github.com/alexcrichton/link-config.

At compile-time the syntax extension detects pkg-config or any other necessary tool and it has an escape hatch for custom logic. Two blocks are generated for dynamic/static deps, and through cargo you will specify what's static and what's dynamic.

It's still a work in progress, but I don't think that we need a pkg-* section of packages, nor does cargo need to know anything about pkg-config by default.

[tomaka]

In fact I had a similar idea a few days ago, but with CMake: https://github.com/Tomaka17/cmake-rs

The problem I encountered is that I couldn't detect the path where to put the libraries (ie. target/deps).
It would help if cargo passed some useful environment variables while building.

[alexcrichton]

It would help if cargo passed this directory as an environment variable while building.

Cargo passes DEPS_DIR and OUT_DIR by default

[tomaka]

Cargo passes DEPS_DIR and OUT_DIR by default

But this is only for the build= command, right?

If you implement a syntax extension, then you need to pass these variables when building regular targets, except if your idea is to write build = "rustc link_to_lib.rs"

[alexcrichton]

The purpose of the syntax extension is to discover native dependencies, not to build them currently. This could be expanded in the future, however.

[tomaka]

This solution does not solve the problem posed by libraries like rust_stb_image.
stb_image is a one-file C library that is not and will never be part of any distribution. It has to be built and linked statically.

[alexcrichton]

That is the intention of the build= command. For example:

// Cargo.toml
build = "make -f build.mk"

// build.mk
all:
    $(CC) -c -o $(DEPS_DIR)/stb_image.o stb_image.c
    ar crus $(DEPS_DIR)/libstb_image.a $(DEPS_DIR)/stb_image.o

// lib.rs
#[link(name = "stb_image", kind = "static")]
extern { /* ... */ }

[pnkfelix]

@alexcrichton can you elaborate how the syntax extension will get cargo to add an additional -L argument to the necessary rustc invocations? Or is that not the intention here -- i.e. is there some option besides adding new directories to the linker search path that you are intending to use here?

[alexcrichton]

That's one of the drawbacks of the current syntax extension. You can compile a crate locally by adding extra -L flags via #[link_args], but it sadly doesn't affect downstream crates.

[wycats]

@alexcrichton we should decide on some canonical solution to this problem, implement what we need to, and then document it.

This seems like a somewhat common problem, and even if the solution is somewhat involved, having a clear path for people (ideally as portable as possible) would help a lot.

[tbu-]

pkg-config is normally not available under Windows unfortunately.

[alexcrichton]

Now that #792 is landed, I think this is basically done. I've included documentation for our solution to this problem in that PR (will soon be pushed to doc.crates.io).

If you have any questions, feel free to ask me!