New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't pass flags to the compiler #60

Closed
mahkoh opened this Issue Jun 25, 2014 · 37 comments

Comments

Projects
None yet
@mahkoh

mahkoh commented Jun 25, 2014

Updated description

Arbitrary flags cannot be passed through to the compiler.

Original Description

-O has to be the default.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Jun 25, 2014

Member

There will be toooooons of these eventually, yes.

Member

steveklabnik commented Jun 25, 2014

There will be toooooons of these eventually, yes.

@npryce

This comment has been minimized.

Show comment
Hide comment
@npryce

npryce Jul 4, 2014

Until this is implemented, cargo cannot be used for cross compilation.

npryce commented Jul 4, 2014

Until this is implemented, cargo cannot be used for cross compilation.

@olsonjeffery

This comment has been minimized.

Show comment
Hide comment
@olsonjeffery

olsonjeffery Jul 5, 2014

+1 for passing flags to the compiler as a critical feature

olsonjeffery commented Jul 5, 2014

+1 for passing flags to the compiler as a critical feature

@japaric

This comment has been minimized.

Show comment
Hide comment
@japaric

japaric Jul 6, 2014

Member

-O has to be the default.

FWIW, I've forked cargo to always pass the "-O" flag to rustc.

Member

japaric commented Jul 6, 2014

-O has to be the default.

FWIW, I've forked cargo to always pass the "-O" flag to rustc.

@huonw

This comment has been minimized.

Show comment
Hide comment
@huonw

huonw Jul 6, 2014

Member

-O should probably be the default for dependencies since they are compiled rarely, but it would be nice for cargo build to default to the fastest option for the main code (i.e. no -O).

Member

huonw commented Jul 6, 2014

-O should probably be the default for dependencies since they are compiled rarely, but it would be nice for cargo build to default to the fastest option for the main code (i.e. no -O).

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jul 7, 2014

Member

#140 has added cargo build --release to build with optimizations

Member

alexcrichton commented Jul 7, 2014

#140 has added cargo build --release to build with optimizations

@alexchandel

This comment has been minimized.

Show comment
Hide comment
@alexchandel

alexchandel Jul 18, 2014

+1 for passing flags like --opt-level=3, -Z lto and --emit=obj via command line or manifest file.

alexchandel commented Jul 18, 2014

+1 for passing flags like --opt-level=3, -Z lto and --emit=obj via command line or manifest file.

@alexchandel

This comment has been minimized.

Show comment
Hide comment
@alexchandel

alexchandel Jul 18, 2014

And -Z no-landing-pads.

alexchandel commented Jul 18, 2014

And -Z no-landing-pads.

@alexcrichton alexcrichton changed the title from Can't pass flags to the compiler / no optimization possible to Can't pass flags to the compiler Aug 3, 2014

@xldenis

This comment has been minimized.

Show comment
Hide comment
@xldenis

xldenis Aug 4, 2014

@alexchandel you seem to be able to do this, right?

xldenis commented Aug 4, 2014

@alexchandel you seem to be able to do this, right?

@alexchandel

This comment has been minimized.

Show comment
Hide comment
@alexchandel

alexchandel Aug 4, 2014

Thankfully, Cargo doesn't crash when passed extra unrecognized flags. For libraries, I partially hack around this issue by calling rustc afterwards on the last thing cargo builds.

Please add some target and emission specifications. The lib.crate-types key is completely ignored, AS IS #![crate_type = "staticlib"].

Cargo isn't the last step in many build cycles, and the inability to output object files or to compile static libraries is a huge inconvenience.

alexchandel commented Aug 4, 2014

Thankfully, Cargo doesn't crash when passed extra unrecognized flags. For libraries, I partially hack around this issue by calling rustc afterwards on the last thing cargo builds.

Please add some target and emission specifications. The lib.crate-types key is completely ignored, AS IS #![crate_type = "staticlib"].

Cargo isn't the last step in many build cycles, and the inability to output object files or to compile static libraries is a huge inconvenience.

@thehydroimpulse

This comment has been minimized.

Show comment
Hide comment
@thehydroimpulse

thehydroimpulse Aug 6, 2014

Being able to output object files and such would be super awesome!

thehydroimpulse commented Aug 6, 2014

Being able to output object files and such would be super awesome!

@bfops

This comment has been minimized.

Show comment
Hide comment
@bfops

bfops Aug 9, 2014

I tried hacking together a fix for this at https://github.com/bfops/cargo. If people are willing to test for me, you can add

extra_rustc_args = ["--foo", "bar"]

to your Cargo.toml. No support yet for passing directly from the command line.

bfops commented Aug 9, 2014

I tried hacking together a fix for this at https://github.com/bfops/cargo. If people are willing to test for me, you can add

extra_rustc_args = ["--foo", "bar"]

to your Cargo.toml. No support yet for passing directly from the command line.

@olsonjeffery

This comment has been minimized.

Show comment
Hide comment
@olsonjeffery

olsonjeffery Aug 9, 2014

👍 to arbitrary rustc flags

olsonjeffery commented Aug 9, 2014

👍 to arbitrary rustc flags

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Aug 10, 2014

Member

It should be noted that the design of cargo is currently to explicitly not allow arbitrary flags. It is very easy to have compilations go awry very quickly, and packages specifying flags for themselves is likely a decision that should not be up to the author but rather the builder.

We certainly plan on allowing more flavorful configuration that is possible today through profiles, but it will likely not just be "pass these flags through to the compiler".

Member

alexcrichton commented Aug 10, 2014

It should be noted that the design of cargo is currently to explicitly not allow arbitrary flags. It is very easy to have compilations go awry very quickly, and packages specifying flags for themselves is likely a decision that should not be up to the author but rather the builder.

We certainly plan on allowing more flavorful configuration that is possible today through profiles, but it will likely not just be "pass these flags through to the compiler".

@npryce

This comment has been minimized.

Show comment
Hide comment
@npryce

npryce Aug 10, 2014

@bfops - putting compiler flags into the TOML file won't work. Compiler flags have to be passed to Cargo from outside.

For example, when cross-compiling a single program for different targets I need to pass different command-line flags (target triple, location of linker, etc.) for each target.

npryce commented Aug 10, 2014

@bfops - putting compiler flags into the TOML file won't work. Compiler flags have to be passed to Cargo from outside.

For example, when cross-compiling a single program for different targets I need to pass different command-line flags (target triple, location of linker, etc.) for each target.

@npryce

This comment has been minimized.

Show comment
Hide comment
@npryce

npryce Aug 10, 2014

@alexcrichton - I think you should always allow the user to "break glass in case of emergency". At the moment, I cannot use Cargo as a build tool. So, for anyone doing cross compilation, it's useless.

If you disallow passing flags to the rustc compiler you essentially couple the release cycles of Cargo and rustc, because Cargo will have to be changed to expose any new capabilities of rustc. But if you're going to couple Cargo and Rustc releases, you may as well pass compiler flags through, because Cargo can have built-in knowledge of the flags accepted by the rustc version it's coupled to, and be able to detect and reject invalid flags.

npryce commented Aug 10, 2014

@alexcrichton - I think you should always allow the user to "break glass in case of emergency". At the moment, I cannot use Cargo as a build tool. So, for anyone doing cross compilation, it's useless.

If you disallow passing flags to the rustc compiler you essentially couple the release cycles of Cargo and rustc, because Cargo will have to be changed to expose any new capabilities of rustc. But if you're going to couple Cargo and Rustc releases, you may as well pass compiler flags through, because Cargo can have built-in knowledge of the flags accepted by the rustc version it's coupled to, and be able to detect and reject invalid flags.

@bfops

This comment has been minimized.

Show comment
Hide comment
@bfops

bfops Aug 10, 2014

@npryce - I agree, flags in the TOML file isn't comprehensive. Once I'm done my exams, I'll try to continue development on my fork if another one hasn't taken off yet. Pull requests are always welcome too!

I'm not sure I entirely understand your use case, sorry. With the current set-up, you'd have to have different targets for windows builds, linux builds, etc. Each target can have its own flags. But yes, right now in my fork they have to be baked into the TOML file.

If arguments are specified on the command line (e.g. via --rustc_args=), should they be passed to one target? All of them? Tests? Examples? The dependencies as well?

bfops commented Aug 10, 2014

@npryce - I agree, flags in the TOML file isn't comprehensive. Once I'm done my exams, I'll try to continue development on my fork if another one hasn't taken off yet. Pull requests are always welcome too!

I'm not sure I entirely understand your use case, sorry. With the current set-up, you'd have to have different targets for windows builds, linux builds, etc. Each target can have its own flags. But yes, right now in my fork they have to be baked into the TOML file.

If arguments are specified on the command line (e.g. via --rustc_args=), should they be passed to one target? All of them? Tests? Examples? The dependencies as well?

@npryce

This comment has been minimized.

Show comment
Hide comment
@npryce

npryce Aug 11, 2014

@bfops In my case, I'm cross-compiling from Ubuntu x86 to Raspbian on the Raspberry Pi, but there are lots of different single-board Linux computers (Beaglebone, etc.) that my programs could run on. Each target typically has its own toolchain -- C compiler, linker, C stdlib, etc. When using Rust, the C compiler is not used, but the linker and C stdlib is.

At the moment I use Make for the build. I have a make include file per target (e.g. targets/raspi.mk) that defines the rustc flags for that target. My main makefile includes targets/$(target).mk. And I can pass the target to the make command: make target=raspi.

How Cargo should work, I don't know. Maybe the project's top-level cargo file should have a way of defining compiler arguments per target and be told the target as a command-line parameter.

Or, just say that Cargo will never be flexible enough for all uses and don't support cross-compilation at all. Let Cargo do dependency resolution well and make it easy to invoke from an existing build tool like Make.

npryce commented Aug 11, 2014

@bfops In my case, I'm cross-compiling from Ubuntu x86 to Raspbian on the Raspberry Pi, but there are lots of different single-board Linux computers (Beaglebone, etc.) that my programs could run on. Each target typically has its own toolchain -- C compiler, linker, C stdlib, etc. When using Rust, the C compiler is not used, but the linker and C stdlib is.

At the moment I use Make for the build. I have a make include file per target (e.g. targets/raspi.mk) that defines the rustc flags for that target. My main makefile includes targets/$(target).mk. And I can pass the target to the make command: make target=raspi.

How Cargo should work, I don't know. Maybe the project's top-level cargo file should have a way of defining compiler arguments per target and be told the target as a command-line parameter.

Or, just say that Cargo will never be flexible enough for all uses and don't support cross-compilation at all. Let Cargo do dependency resolution well and make it easy to invoke from an existing build tool like Make.

@bfops

This comment has been minimized.

Show comment
Hide comment
@bfops

bfops Aug 12, 2014

@npryce My current cargo fork allows specifying rustc command line flags on a per-target basis, the remaining step would be to make the target a command-line parameter, which seems like a good feature either way. I'll look into it.

bfops commented Aug 12, 2014

@npryce My current cargo fork allows specifying rustc command line flags on a per-target basis, the remaining step would be to make the target a command-line parameter, which seems like a good feature either way. I'll look into it.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 4, 2014

Member

I'm going to close this issue as-is, as I don't think that it's particularly actionable any more. Support for per-project profiles recently landed which is how cargo plans to support customizing the build of a project.

Right now profiles are pretty bare, only supporting opt-level and debug, but the plan is to make them much richer by adding a variety of other options. At this time we don't plan on allowing, through the normal cargo build interface, arbitrary flags to be passed to the compiler. The plan is to have a composable set of sub-commands which will be used to customize a build of cargo if you'd like.

If you currently have a use case which cargo isn't supporting, then please feel free to open a bug about it! Specific problems are greatly appreciated, and the general idea is that build-related options (like most of those in -C) should be exposed through profiles.

So in summary:

  • It is not currently planned for arbitrary flags to be passed through the manifest/cargo build
  • I'm closing this so more specific follow-up bugs can be opened.
  • New finer-grained commands will be added to customize how cargo builds packages, and this is how truly arbitrary flags will be able to be passed. This is not currently implemented today, but it's certainly planned.
Member

alexcrichton commented Sep 4, 2014

I'm going to close this issue as-is, as I don't think that it's particularly actionable any more. Support for per-project profiles recently landed which is how cargo plans to support customizing the build of a project.

Right now profiles are pretty bare, only supporting opt-level and debug, but the plan is to make them much richer by adding a variety of other options. At this time we don't plan on allowing, through the normal cargo build interface, arbitrary flags to be passed to the compiler. The plan is to have a composable set of sub-commands which will be used to customize a build of cargo if you'd like.

If you currently have a use case which cargo isn't supporting, then please feel free to open a bug about it! Specific problems are greatly appreciated, and the general idea is that build-related options (like most of those in -C) should be exposed through profiles.

So in summary:

  • It is not currently planned for arbitrary flags to be passed through the manifest/cargo build
  • I'm closing this so more specific follow-up bugs can be opened.
  • New finer-grained commands will be added to customize how cargo builds packages, and this is how truly arbitrary flags will be able to be passed. This is not currently implemented today, but it's certainly planned.
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 18, 2014

Member

#595 is an example of a subcommand which may help here, I'm curious about opinions on it!

Member

alexcrichton commented Sep 18, 2014

#595 is an example of a subcommand which may help here, I'm curious about opinions on it!

@buster

This comment has been minimized.

Show comment
Hide comment
@buster

buster Dec 10, 2014

Currently i have a little rust program, that i would like to be compiled dynamically linked, because the resulting binary becomes huge if not.
So, that is not yet possible with cargo?

buster commented Dec 10, 2014

Currently i have a little rust program, that i would like to be compiled dynamically linked, because the resulting binary becomes huge if not.
So, that is not yet possible with cargo?

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@buster

This comment has been minimized.

Show comment
Hide comment
@buster

buster Dec 10, 2014

@steveklabnik , actually i want to build a binary with "-C prefer-dynamic"...

buster commented Dec 10, 2014

@steveklabnik , actually i want to build a binary with "-C prefer-dynamic"...

@prdoyle

This comment has been minimized.

Show comment
Hide comment
@prdoyle

prdoyle Feb 12, 2015

+1 for "-C prefer-dynamic". Anybody figured out the right way to do that yet? 550 KB "hello world" is a dealbreaker for me.

prdoyle commented Feb 12, 2015

+1 for "-C prefer-dynamic". Anybody figured out the right way to do that yet? 550 KB "hello world" is a dealbreaker for me.

@alexchandel

This comment has been minimized.

Show comment
Hide comment
@alexchandel

alexchandel Feb 14, 2015

Ugh, I need a -C ban-dynamic. @buster @prdoyle The big hello world is due to rust#22159. If you hack out allocation, it comes down to 4k (on OS X x64). Bear in mind though that the 500 kB is the amortized cost of jemalloc & friends: larger executables will have much better mileage / far fewer bytes per LOC than small ones because of this.

alexchandel commented Feb 14, 2015

Ugh, I need a -C ban-dynamic. @buster @prdoyle The big hello world is due to rust#22159. If you hack out allocation, it comes down to 4k (on OS X x64). Bear in mind though that the 500 kB is the amortized cost of jemalloc & friends: larger executables will have much better mileage / far fewer bytes per LOC than small ones because of this.

@prdoyle

This comment has been minimized.

Show comment
Hide comment
@prdoyle

prdoyle Feb 14, 2015

Sure, I get that it will be amortized, but it would be nice if Rust could be used for small programs too, even if they allocate some memory.

prdoyle commented Feb 14, 2015

Sure, I get that it will be amortized, but it would be nice if Rust could be used for small programs too, even if they allocate some memory.

@alexchandel

This comment has been minimized.

Show comment
Hide comment
@alexchandel

alexchandel Feb 15, 2015

@prdoyle jemalloc could definitely use some optimization. I can't imagine how all 500kB could be necessary for a single malloc. Though many heap allocations could be optimized out. Yet on most platforms Rust programs are dynamically linked to the system library (glibc/libSystem/kernel32/etc), which uses 5-10MB of process memory. If we had our own system library, then a 500kB runtime wouldn't be so bad. Especially since with static linking & LTO, most programs would use a fraction of that.

alexchandel commented Feb 15, 2015

@prdoyle jemalloc could definitely use some optimization. I can't imagine how all 500kB could be necessary for a single malloc. Though many heap allocations could be optimized out. Yet on most platforms Rust programs are dynamically linked to the system library (glibc/libSystem/kernel32/etc), which uses 5-10MB of process memory. If we had our own system library, then a 500kB runtime wouldn't be so bad. Especially since with static linking & LTO, most programs would use a fraction of that.

@MarkSwanson

This comment has been minimized.

Show comment
Hide comment
@MarkSwanson

MarkSwanson Mar 4, 2015

+1 for LTO; we're thinking of building small shared objects in rust and deploying them inside Linux containers that have tight memory restrictions.

MarkSwanson commented Mar 4, 2015

+1 for LTO; we're thinking of building small shared objects in rust and deploying them inside Linux containers that have tight memory restrictions.

@huonw

This comment has been minimized.

Show comment
Hide comment
@huonw
Member

huonw commented Mar 4, 2015

@maxlapshin

This comment has been minimized.

Show comment
Hide comment
@maxlapshin

maxlapshin May 25, 2015

This is very nice, but: erszcz/erlang-rust-nif#3
cargo cannot compile under MacOS X without adding specific flags.

maxlapshin commented May 25, 2015

This is very nice, but: erszcz/erlang-rust-nif#3
cargo cannot compile under MacOS X without adding specific flags.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton May 26, 2015

Member

@maxlapshin would cargo rustc suffice for your use case?

Member

alexcrichton commented May 26, 2015

@maxlapshin would cargo rustc suffice for your use case?

@zjjott

This comment has been minimized.

Show comment
Hide comment
@zjjott

zjjott Jun 19, 2015

Holy cargo....

It should be noted that the design of cargo is currently to explicitly not allow arbitrary flags. It is very easy to have compilations go awry very quickly,

I just want to know,a project like servo,how to use cargo for minimize version using prefer-dynamic args?

zjjott commented Jun 19, 2015

Holy cargo....

It should be noted that the design of cargo is currently to explicitly not allow arbitrary flags. It is very easy to have compilations go awry very quickly,

I just want to know,a project like servo,how to use cargo for minimize version using prefer-dynamic args?

@PeteX

This comment has been minimized.

Show comment
Hide comment
@PeteX

PeteX Nov 20, 2016

I found this bug and got confused, so I thought I'd add a note for the benefit of other visitors. As of November 2016 it's perfectly possible to pass arbitrary flags to rustc. If you want to use dynamic libraries, for example, you can do either:

cargo rustc -- -C prefer-dynamic

or

RUSTFLAGS='-C prefer-dynamic' cargo build

Hope this helps someone!

PeteX commented Nov 20, 2016

I found this bug and got confused, so I thought I'd add a note for the benefit of other visitors. As of November 2016 it's perfectly possible to pass arbitrary flags to rustc. If you want to use dynamic libraries, for example, you can do either:

cargo rustc -- -C prefer-dynamic

or

RUSTFLAGS='-C prefer-dynamic' cargo build

Hope this helps someone!

@mulkieran

This comment has been minimized.

Show comment
Hide comment
@mulkieran

mulkieran Dec 2, 2016

@PeteX It helped me, thanks!

mulkieran commented Dec 2, 2016

@PeteX It helped me, thanks!

@learnopengles

This comment has been minimized.

Show comment
Hide comment
@learnopengles

learnopengles Jan 11, 2017

There also seems to be flags like CARGO_TARGET_{target-triple in all caps and with "-" replaced by "_"}_LINKER which are still needed if, for example, you want to cross-compile the tests.

learnopengles commented Jan 11, 2017

There also seems to be flags like CARGO_TARGET_{target-triple in all caps and with "-" replaced by "_"}_LINKER which are still needed if, for example, you want to cross-compile the tests.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Apr 4, 2018

Member

It should be noted that the design of cargo is currently to explicitly not allow arbitrary flags. It is very easy to have compilations go awry very quickly, and packages specifying flags for themselves is likely a decision that should not be up to the author but rather the builder.

I think this is a good argument for published dependencies, but less so for toplevel Cargo.tomls. We already have profiles for this, and ideally they'd be able to specify more. You can always override things via .cargo/config or editing the Cargo.toml.

This becomes frustrating because IDEs no longer work with projects forced to use RUSTFLAGS as part of their build process (e.g. Servo) since their build process is more than cargo build. At best they don't work, at worst they clobber builds to boot.

Really, even the ability to override this in .cargo/config ONLY would be quite useful. But it sounds very much like a profile thing.

Edit: My bad, it's possible to do this in the config already

Member

Manishearth commented Apr 4, 2018

It should be noted that the design of cargo is currently to explicitly not allow arbitrary flags. It is very easy to have compilations go awry very quickly, and packages specifying flags for themselves is likely a decision that should not be up to the author but rather the builder.

I think this is a good argument for published dependencies, but less so for toplevel Cargo.tomls. We already have profiles for this, and ideally they'd be able to specify more. You can always override things via .cargo/config or editing the Cargo.toml.

This becomes frustrating because IDEs no longer work with projects forced to use RUSTFLAGS as part of their build process (e.g. Servo) since their build process is more than cargo build. At best they don't work, at worst they clobber builds to boot.

Really, even the ability to override this in .cargo/config ONLY would be quite useful. But it sounds very much like a profile thing.

Edit: My bad, it's possible to do this in the config already

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment