Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-exporting C symbols for cdylib #2771

Open
qinsoon opened this issue Sep 8, 2016 · 22 comments
Open

Re-exporting C symbols for cdylib #2771

qinsoon opened this issue Sep 8, 2016 · 22 comments
Labels
A-ffi A-linkage T-lang

Comments

@qinsoon
Copy link

@qinsoon qinsoon commented Sep 8, 2016

In an extern block, C symbols can be imported to Rust. When building Rust library into dylib, the symbols stay visible and can be used from C. However, when building into newly introduced cdylib, the imported C symbols are no longer visible. I am not aware of a mechanism to re-export the C symbols.

@petrochenkov
Copy link
Contributor

@petrochenkov petrochenkov commented Sep 8, 2016

Does making them pub help? (More precisely, making them nameable from outside of the crate.)

@qinsoon
Copy link
Author

@qinsoon qinsoon commented Sep 9, 2016

@petrochenkov Declaring pub doesn't help. I assume it is because pub does not mean they are accessible from native, thus their symbols can still be safely removed.

@retep998
Copy link
Member

@retep998 retep998 commented Sep 27, 2017

Can we please have some focus brought to this. For many applications which combine both C and Rust and are trying to create a cdylib, the lack of this feature is a blocker.

@michaelwoerister
Copy link
Member

@michaelwoerister michaelwoerister commented Oct 2, 2017

It seems clear that we need this functionality in one form or other. Ideally we'd have an RFC to determine who this should be implemented exactly. @aturon, how do we handle this, it being the impl period now? Or maybe we can decide on a straightforward way how to express this without an RFC.

@michaelwoerister
Copy link
Member

@michaelwoerister michaelwoerister commented Oct 2, 2017

I'll leave this nominated for now so it doesn't fall off the radar. Hopefully we'll have a clearer picture on how to proceed before the next dev-tools meeting.

@aturon
Copy link
Member

@aturon aturon commented Oct 16, 2017

I think we could approach the procedural issue here in a few ways:

  1. We could land the functionality under a feature gate, and have an RFC prior to stabilization
  2. We could have a public discussion on thread, involving lang team, with fcp signoff
  3. We could just do an RFC :)

It's not like RFCs are prohibited per se, more that we're trying to focus the teams on executing already-planned work. This is small enough that it's probably not a big distraction.

I do think that in general this area of Rust could use more design focus, FWIW.

@retep998
Copy link
Member

@retep998 retep998 commented Oct 16, 2017

Currently the recommended way to export a function that you define yourself from a cdylib or staticlib is a #[no_mangle] pub extern fn foo(...) { ... } at the crate root. So I'd like to propose a solution that is as close to the existing forms as possible with minimal changes to the language:

  • Declare the external functions directly at the crate root and make them public: extern { pub fn foo(...); }
  • If the external function is declared elsewhere then publicly re-export into the crate root: pub use foo::bar;

The crate root being the root of the cdylib or staticlib crate itself.

@aturon
Copy link
Member

@aturon aturon commented Oct 16, 2017

cc @rust-lang/lang, adding this to our list of issues that need to be triaged.

@joshtriplett
Copy link
Member

@joshtriplett joshtriplett commented Oct 19, 2017

We discussed this one in the lang team meeting. This does need an RFC; how quickly that progresses will depend on how smoothly and non-bikesheddy the RFC goes. The approach of making symbols public if pub and #[no_mangle] seems reasonable.

Also, we should stop automatically exporting such symbols from a dylib, if possible. That may be difficult without regressions, but it's a serious library hygiene issue; it prevents hiding ABI changes for a dependency inside your own library, without breaking your own ABI.

@petrochenkov
Copy link
Contributor

@petrochenkov petrochenkov commented Oct 19, 2017

The approach of making symbols public if pub and #[no_mangle] seems reasonable.

Oh, this is great. What is done now (if I remember correctly), e.g. heuristic-based link-time visibility based on sum of factors like crate type, ABI (non-"Rust") and no_mangle even on private items is a mess and kinda breaks the spirit of Rust privacy (even if link-time visibility is a separate issue from compile-time one).

@joshtriplett
Copy link
Member

@joshtriplett joshtriplett commented Oct 19, 2017

I do think it'd be nice to have a broader RFC that explicitly defines when symbols should be exported, across all use-cases.

@dbrgn
Copy link

@dbrgn dbrgn commented Feb 1, 2018

I just stumbled over this. Is there any workaround that we can use in the meantime?

In the worst case I could wrapp all 3rd party functions with my own:

extern crate saltyrtc_client_ffi;

pub type salty_event_loop_t = saltyrtc_client_ffi::salty_event_loop_t;

#[no_mangle]
pub extern "C" fn salty_event_loop_new() -> *mut salty_event_loop_t {
    saltyrtc_client_ffi::salty_event_loop_new()
}

...but I hope there's a better way than this.

If not, this is really a stumbling block when trying to reuse code from FFI compatible crates...

@Michael-F-Bryan
Copy link

@Michael-F-Bryan Michael-F-Bryan commented Feb 7, 2018

I just encountered this issue at work. We've got a ffi_utils crate which contains general things like error handling, null pointer checks, and exception safety and I was hoping the original symbols would be exported from a cdylib which pulls in the ffi_utils crate.

@dbrgn for now I've found a really crummy workaround.... I'm creating a macro which will manually re-export the symbols. So you "only" need to call export_error_handling functions!() to make sure the symbols are included in the final DLL.

It feels horrible, but it works...


#[doc(hidden)]
#[macro_export]
macro_rules! export_c_symbol {
    (fn $name:ident($( $arg:ident : $type:ty ),*) -> $ret:ty) => {
        #[no_mangle]
        pub unsafe extern "C" fn $name($( $arg : $type),*) -> $ret {
            $crate::error_handling::$name($( $arg ),*)
        }
    };
    (fn $name:ident($( $arg:ident : $type:ty ),*)) => {
        export_c_symbol!(fn $name($( $arg : $type),*) -> ());
    }
}

/// As a workaround for rust-lang/rust#6342, you can use this macro to make sure
/// the symbols for `ffi_utils`'s error handling are correctly exported in your
/// `cdylib`.
#[macro_export]
macro_rules! export_error_handling_functions {
    () => {
    export_c_symbol!(fn clear_last_error());
    export_c_symbol!(fn last_error_length() -> ::libc::c_int);
    export_c_symbol!(fn last_error_length_utf16() -> ::libc::c_int);
    export_c_symbol!(fn error_message_utf8(buf: *mut ::libc::c_char, length: ::libc::c_int) -> ::libc::c_int);
    export_c_symbol!(fn error_message_utf16(buf: *mut u16, length: ::libc::c_int) -> ::libc::c_int);
    };
}

@dbrgn
Copy link

@dbrgn dbrgn commented Feb 7, 2018

@Michael-F-Bryan thanks. yeah, there's probably currently no way around wrapping these functions.

by the way, you should try to avoid two #[no_mangle] functions with the same name, even across crate boundaries: https://users.rust-lang.org/t/including-third-party-no-mangle-functions-in-a-cdylib/15388/2

@Michael-F-Bryan
Copy link

@Michael-F-Bryan Michael-F-Bryan commented Feb 7, 2018

by the way, you should try to avoid two #[no_mangle] functions with the same name, even across crate boundaries

Yep. The first time I ran the test suite with that macro I got lots of linker errors due to duplicate symbols. I "fixed" that by removing the #[no_mangle] from the original functions so the compiler would still mangle them.

Interestingly, I never got the "duplicate symbols" error when compiling the DLL in release mode. I'm assuming rustc/LLVM inlines the original functions and then strips out their now "unused" function symbols before they get to the linking stage.

@dbrgn
Copy link

@dbrgn dbrgn commented Feb 7, 2018

It doesn't happen when linking together two libraries with rustc, but it happened to me when trying to link two independent shared libraries (compiled from Rust) into an iOS app. There might be a way to extract the Rust stdlib into a separate shared library (similar to the way it's done with libc, if I understand this correctly), but I haven't found a way to do this so far.

But that's probably offtopic here :)

@agalakhov
Copy link

@agalakhov agalakhov commented Sep 5, 2018

Stepped into this issue as well. Unfortunately the solution with function wrapper does not work for me as I have to reexport arrays.

@tanriol
Copy link

@tanriol tanriol commented Sep 22, 2018

The workaround does not work for me either as I need to (re)export variadic functions which cannot be defined in Rust.

@joshtriplett
Copy link
Member

@joshtriplett joshtriplett commented Sep 22, 2018

@tanriol While it doesn't address your immediate issue, in case you haven't already seen it, #2137 and rust-lang/rust#44930 may help you in the future.

ghost referenced this issue in bytecodealliance/lucet Feb 22, 2019
This required some shenanigans with the probestack definition to get it exported in the final
`lucet_runtime.so`. I left a TODO in the code to revert it once [this Rust
issue](https://github.com/rust-lang/rust/issues/36342) is fixed.
ghost referenced this issue in bytecodealliance/lucet Feb 23, 2019
Moving the stack probe into the compiled Lucet modules lets us dodge Rust's current inability to
reexport dynamic symbols (see <https://github.com/rust-lang/rust/issues/36342>). It loses a small
amount of fidelity that we got with stack overflow traps previously, as we can't distinguish a stack
overflow originating in the stack probe from anywhere else.

The C test suites are now parameterized over region, much like the Rust test suites.
@marmeladema
Copy link

@marmeladema marmeladema commented Apr 10, 2019

I encountered this issue on Linux using rust 1.33.0. The trick to set lto to true and incremental to false works fine if the crate type is only cdylib, if you add rlib for example crate-type = ["cdylib", "rlib"] then no imported symbols are re-exported.

@Mark-Simulacrum Mark-Simulacrum transferred this issue from rust-lang/rust Sep 22, 2019
@Mark-Simulacrum Mark-Simulacrum added the A-ffi label Sep 22, 2019
@Centril Centril added T-lang A-linkage labels Sep 22, 2019
@DianaNites
Copy link

@DianaNites DianaNites commented Jan 16, 2020

I just ran into, and spent hours, debugging this. :(

In my case I can trivially wrap it but it'd be nice if I could tell Rust not to throw away my symbol :(

Michael-F-Bryan added a commit to Michael-F-Bryan/ffi_helpers that referenced this issue Apr 7, 2020
@devsnek
Copy link

@devsnek devsnek commented Oct 10, 2020

So right now it is not possible to re-export, for example, libc::malloc?

jrose-signal added a commit to signalapp/libsignal that referenced this issue Nov 20, 2020
When building a cdylib, Rust currently doesn't export public symbols
on Linux if they come from a dependency. However, enabling LTO gets
around this by performing the filtering step after the merging.

rust-lang/rfcs#2771
@rust-lang rust-lang deleted a comment from p499 Nov 22, 2020
jrose-signal added a commit to signalapp/libsignal that referenced this issue Dec 2, 2020
When building a cdylib, Rust currently doesn't export public symbols
on Linux if they come from a dependency. However, enabling LTO gets
around this by performing the filtering step after the merging.

rust-lang/rfcs#2771
jrose-signal added a commit to signalapp/libsignal that referenced this issue Dec 10, 2020
When building a cdylib, Rust currently doesn't export public symbols
on Linux if they come from a dependency. However, enabling LTO gets
around this by performing the filtering step after the merging.

rust-lang/rfcs#2771
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ffi A-linkage T-lang
Projects
None yet
Development

No branches or pull requests