Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: -C export-executable-symbols #2841

Open
wants to merge 4 commits into
base: master
from

Conversation

@MaulingMonkey
Copy link

MaulingMonkey commented Dec 29, 2019

Rendered

Add the ability to export symbols from executables, not just dylibs, via a new compiler flag: -C export-executable-symbols.

(Bonus: I've also implemented and used this flag as a proof-of-concept to make sure I am indeed solving my problems with this suggestion, but I'm happy to rewrite or discard that if another approach seems better.)

@retep998

This comment has been minimized.

Copy link
Member

retep998 commented Dec 29, 2019

How would the user specify this in the normal cargo world without custom rustc flags?

@MaulingMonkey

This comment has been minimized.

Copy link
Author

MaulingMonkey commented Dec 29, 2019

Specifying a .cargo/config file alongside/above Cargo.toml in the project tree works, but it'd be possible to extend Cargo.toml manifests as well - the most similar looking options all live under [profile.*]. Could be a new option:

[profile.dev]
export-executable-symbols = true

On the other hand, this is a niche enough use case, that just leaving it as a custom rustc flag might be more than enough. For my own use cases, I'm likely to want to apply the setting to an entire workspace once and forget about it - instead of configuring it on a per-profile or per-package basis - so I'm likely to resort to an in-tree .cargo/config regardless.

@tanriol

This comment has been minimized.

Copy link

tanriol commented Dec 30, 2019

Would be useful to me too. So far for me on Linux is mostly just works, except that I needed to add a code path that may call the same exported functions from the executable itself because otherwise they were discarded as dead code. Would love to get rid of that hack.

@pnkfelix

This comment has been minimized.

Copy link
Member

pnkfelix commented Dec 31, 2019

Its sort of unfortunate that this RFC is written in a fashion where it sounds like it is saying "just do what dylibs do, but on executables", when at least currently, the semantics of dylibs are ... in flux at best.

(That is, to my knowledge, there is not much specification for dylibs themselves beyond "keep rustc working, and maybe try to keep compiler-plugins working for a little while longer." For some examples of the complications involved, see the discussion on rust-lang/rust#37530.)

So let me ask some basic questions, where I would like an answer more specific than "just do what dylibs do":

  • Is this going to export solely symbols for items marked pub ? If so, it might be good to make that point explicit here. (At least, I don't think I saw visibility mentioned anywhere.)

  • Do you need fine grain control over which symbols are exported and which ones aren't?

  • Potentially related to the previous bullet: Why not make this an attribute in the source rather than a compiler-flag? Are there cases where embedding this info into the Rust source itself would be a mistake or an anti-pattern?

    • An attribute could here mean either something like #[export] that you'd put on every item whose symbol you want exported, or it could mean an attribute like #![export_all_symbols] that you'd just write once in the root main.rs file.
    • (Even if you need the behavior to be applied to dependencies in your crate graph, you could still get that semantics via an attribute with a global-effect included in the source code of the executable crate itself.)
@retep998

This comment has been minimized.

Copy link
Member

retep998 commented Dec 31, 2019

The behavior of which symbols to export should match what we do for cdylib because anything else would be inconsistent and confusing. I'd love if we could improve that behavior but I don't think this RFC is the place to do that.

A crate wide attribute to toggle exporting symbols is a valid alternative and should go in the alternatives section. I am neutral on crate wide attribute vs rustc flag and would be happy with either.

@MaulingMonkey

This comment has been minimized.

Copy link
Author

MaulingMonkey commented Dec 31, 2019

Is this going to export solely symbols for items marked pub ? If so, it might be good to make that point explicit here. (At least, I don't think I saw visibility mentioned anywhere.)

I'm interested in explicitly annotated items:

  • #[no_mangle] pub extern "..." fn s, which I'm after right now for JNI interop.
  • #[used] #[no_mangle] pub static s have possible uses, although I don't have any immediate need for these.

However, pub here means public to the mod, not necessairly the crate. Exposing these publicly all the way to the crate root, while doable, would be awkward in the likely event that the item names are generated by macros for the purpouse of mangling to some other language's naming standard.

The current behavior seems to be to export anything tagged #[no_mangle] (including main and rust_eh_personality from Rust itself), but otherwise ignoring visibility and ABI entirely.

Do you need fine grain control over which symbols are exported and which ones aren't?

I don't, although I could see some proprietary devs not wanting to export the entire universe. They'd likely have similar concerns with dylibs, however.

Potentially related to the previous bullet: Why not make this an attribute in the source rather than a compiler-flag? Are there cases where embedding this info into the Rust source itself would be a mistake or an anti-pattern?

A source attribute would probably work fine? My only possible concern would be ease of implementation - piping that information up to the linking stage - although I'd be happy to take a stab at it if that seems preferable.

An attribute could here mean either something like #[export] that you'd put on every item whose symbol you want exported, or it could mean an attribute like #![export_all_symbols] that you'd just write once in the root main.rs file.

Finer grained control over exports could be useful to dylibs as well, for those who want fine grained control. E.g. one might have middleware looking to link against a specific known #[no_mangle] symbol for allocation or logging purpouses - yet wish to deter reverse engineering by statically linking in said middleware, and not exporting the symbol dynamically, even when building a dynamic library for Android, while still needing to export the main entry points for Android to call into.

@Lokathor

This comment has been minimized.

Copy link

Lokathor commented Dec 31, 2019

Remember that using no_mangle effectively puts the symbol at the crate root because module location within a crate is performed with mangling.

@gilescope

This comment has been minimized.

Copy link

gilescope commented Jan 3, 2020

Would this allow us to write integration tests against an exe? Currently we can only do it against libs. (I.e. tests in the tests dir). This leads to a lot of exes being changed into libs just for test purposes...

@MaulingMonkey

This comment has been minimized.

Copy link
Author

MaulingMonkey commented Jan 4, 2020

Would this allow us to write integration tests against an exe? Currently we can only do it against libs. (I.e. tests in the tests dir). This leads to a lot of exes being changed into libs just for test purposes...

Not on its own.

# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

On a technical level, this just involves preventing an early bailout when

This comment has been minimized.

Copy link
@nagisa

nagisa Jan 12, 2020

Contributor

This is not a reference level explanation of the feature, but rather an approach to implementation. For RFC process this is not too useful and is prone to becoming invalid and impossible to understand in the future or in context of alternative implementations.

Instead this section should be exhaustively enumerating interactions with other language features (@pnkfelix mentioned for instance visibility) and describing how the relevant portions (symbols) of produced artifacts change as a result.

Alternatives:

- Unconditionally export symbols from executables instead of introducing a new compiler flag.
- Introduce a crate-level attribute instead of a compiler flag (`#![export_all_symbols]`? `#![export_symbols]`?)

This comment has been minimized.

Copy link
@nagisa

nagisa Jan 12, 2020

Contributor

As mentioned in discussion already, add the alternative: introduce an item-level attribute to only "in-executable export" specific items.

# Prior art
[prior-art]: #prior-art

C and C++ compilers can already do this via `__declspec(dllexport)` annotations.

This comment has been minimized.

Copy link
@nagisa

nagisa Jan 12, 2020

Contributor

dllexport was a part of a previous (closed) RFC #1296 and discussed fairly extensively. Might be worth it to look through that RFC to see if we didn’t miss anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.