Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-exporting C symbols for cdylib #2771

Open
qinsoon opened this issue Sep 8, 2016 · 21 comments
Open

Re-exporting C symbols for cdylib #2771

qinsoon opened this issue Sep 8, 2016 · 21 comments

Comments

@qinsoon
Copy link

@qinsoon qinsoon commented Sep 8, 2016

In an extern block, C symbols can be imported to Rust. When building Rust library into dylib, the symbols stay visible and can be used from C. However, when building into newly introduced cdylib, the imported C symbols are no longer visible. I am not aware of a mechanism to re-export the C symbols.

@petrochenkov

This comment has been minimized.

Copy link
Contributor

@petrochenkov petrochenkov commented Sep 8, 2016

Does making them pub help? (More precisely, making them nameable from outside of the crate.)

@qinsoon

This comment has been minimized.

Copy link
Author

@qinsoon qinsoon commented Sep 9, 2016

@petrochenkov Declaring pub doesn't help. I assume it is because pub does not mean they are accessible from native, thus their symbols can still be safely removed.

@retep998

This comment has been minimized.

Copy link
Member

@retep998 retep998 commented Sep 27, 2017

Can we please have some focus brought to this. For many applications which combine both C and Rust and are trying to create a cdylib, the lack of this feature is a blocker.

@michaelwoerister

This comment has been minimized.

Copy link
Contributor

@michaelwoerister michaelwoerister commented Oct 2, 2017

It seems clear that we need this functionality in one form or other. Ideally we'd have an RFC to determine who this should be implemented exactly. @aturon, how do we handle this, it being the impl period now? Or maybe we can decide on a straightforward way how to express this without an RFC.

@michaelwoerister

This comment has been minimized.

Copy link
Contributor

@michaelwoerister michaelwoerister commented Oct 2, 2017

I'll leave this nominated for now so it doesn't fall off the radar. Hopefully we'll have a clearer picture on how to proceed before the next dev-tools meeting.

@aturon

This comment has been minimized.

Copy link
Member

@aturon aturon commented Oct 16, 2017

I think we could approach the procedural issue here in a few ways:

  1. We could land the functionality under a feature gate, and have an RFC prior to stabilization
  2. We could have a public discussion on thread, involving lang team, with fcp signoff
  3. We could just do an RFC :)

It's not like RFCs are prohibited per se, more that we're trying to focus the teams on executing already-planned work. This is small enough that it's probably not a big distraction.

I do think that in general this area of Rust could use more design focus, FWIW.

@retep998

This comment has been minimized.

Copy link
Member

@retep998 retep998 commented Oct 16, 2017

Currently the recommended way to export a function that you define yourself from a cdylib or staticlib is a #[no_mangle] pub extern fn foo(...) { ... } at the crate root. So I'd like to propose a solution that is as close to the existing forms as possible with minimal changes to the language:

  • Declare the external functions directly at the crate root and make them public: extern { pub fn foo(...); }
  • If the external function is declared elsewhere then publicly re-export into the crate root: pub use foo::bar;

The crate root being the root of the cdylib or staticlib crate itself.

@aturon

This comment has been minimized.

Copy link
Member

@aturon aturon commented Oct 16, 2017

cc @rust-lang/lang, adding this to our list of issues that need to be triaged.

@joshtriplett

This comment has been minimized.

Copy link
Member

@joshtriplett joshtriplett commented Oct 19, 2017

We discussed this one in the lang team meeting. This does need an RFC; how quickly that progresses will depend on how smoothly and non-bikesheddy the RFC goes. The approach of making symbols public if pub and #[no_mangle] seems reasonable.

Also, we should stop automatically exporting such symbols from a dylib, if possible. That may be difficult without regressions, but it's a serious library hygiene issue; it prevents hiding ABI changes for a dependency inside your own library, without breaking your own ABI.

@petrochenkov

This comment has been minimized.

Copy link
Contributor

@petrochenkov petrochenkov commented Oct 19, 2017

The approach of making symbols public if pub and #[no_mangle] seems reasonable.

Oh, this is great. What is done now (if I remember correctly), e.g. heuristic-based link-time visibility based on sum of factors like crate type, ABI (non-"Rust") and no_mangle even on private items is a mess and kinda breaks the spirit of Rust privacy (even if link-time visibility is a separate issue from compile-time one).

@joshtriplett

This comment has been minimized.

Copy link
Member

@joshtriplett joshtriplett commented Oct 19, 2017

I do think it'd be nice to have a broader RFC that explicitly defines when symbols should be exported, across all use-cases.

@dbrgn

This comment has been minimized.

Copy link

@dbrgn dbrgn commented Feb 1, 2018

I just stumbled over this. Is there any workaround that we can use in the meantime?

In the worst case I could wrapp all 3rd party functions with my own:

extern crate saltyrtc_client_ffi;

pub type salty_event_loop_t = saltyrtc_client_ffi::salty_event_loop_t;

#[no_mangle]
pub extern "C" fn salty_event_loop_new() -> *mut salty_event_loop_t {
    saltyrtc_client_ffi::salty_event_loop_new()
}

...but I hope there's a better way than this.

If not, this is really a stumbling block when trying to reuse code from FFI compatible crates...

@Michael-F-Bryan

This comment has been minimized.

Copy link

@Michael-F-Bryan Michael-F-Bryan commented Feb 7, 2018

I just encountered this issue at work. We've got a ffi_utils crate which contains general things like error handling, null pointer checks, and exception safety and I was hoping the original symbols would be exported from a cdylib which pulls in the ffi_utils crate.

@dbrgn for now I've found a really crummy workaround.... I'm creating a macro which will manually re-export the symbols. So you "only" need to call export_error_handling functions!() to make sure the symbols are included in the final DLL.

It feels horrible, but it works...


#[doc(hidden)]
#[macro_export]
macro_rules! export_c_symbol {
    (fn $name:ident($( $arg:ident : $type:ty ),*) -> $ret:ty) => {
        #[no_mangle]
        pub unsafe extern "C" fn $name($( $arg : $type),*) -> $ret {
            $crate::error_handling::$name($( $arg ),*)
        }
    };
    (fn $name:ident($( $arg:ident : $type:ty ),*)) => {
        export_c_symbol!(fn $name($( $arg : $type),*) -> ());
    }
}

/// As a workaround for rust-lang/rust#6342, you can use this macro to make sure
/// the symbols for `ffi_utils`'s error handling are correctly exported in your
/// `cdylib`.
#[macro_export]
macro_rules! export_error_handling_functions {
    () => {
    export_c_symbol!(fn clear_last_error());
    export_c_symbol!(fn last_error_length() -> ::libc::c_int);
    export_c_symbol!(fn last_error_length_utf16() -> ::libc::c_int);
    export_c_symbol!(fn error_message_utf8(buf: *mut ::libc::c_char, length: ::libc::c_int) -> ::libc::c_int);
    export_c_symbol!(fn error_message_utf16(buf: *mut u16, length: ::libc::c_int) -> ::libc::c_int);
    };
}
@dbrgn

This comment has been minimized.

Copy link

@dbrgn dbrgn commented Feb 7, 2018

@Michael-F-Bryan thanks. yeah, there's probably currently no way around wrapping these functions.

by the way, you should try to avoid two #[no_mangle] functions with the same name, even across crate boundaries: https://users.rust-lang.org/t/including-third-party-no-mangle-functions-in-a-cdylib/15388/2

@Michael-F-Bryan

This comment has been minimized.

Copy link

@Michael-F-Bryan Michael-F-Bryan commented Feb 7, 2018

by the way, you should try to avoid two #[no_mangle] functions with the same name, even across crate boundaries

Yep. The first time I ran the test suite with that macro I got lots of linker errors due to duplicate symbols. I "fixed" that by removing the #[no_mangle] from the original functions so the compiler would still mangle them.

Interestingly, I never got the "duplicate symbols" error when compiling the DLL in release mode. I'm assuming rustc/LLVM inlines the original functions and then strips out their now "unused" function symbols before they get to the linking stage.

@dbrgn

This comment has been minimized.

Copy link

@dbrgn dbrgn commented Feb 7, 2018

It doesn't happen when linking together two libraries with rustc, but it happened to me when trying to link two independent shared libraries (compiled from Rust) into an iOS app. There might be a way to extract the Rust stdlib into a separate shared library (similar to the way it's done with libc, if I understand this correctly), but I haven't found a way to do this so far.

But that's probably offtopic here :)

@agalakhov

This comment has been minimized.

Copy link

@agalakhov agalakhov commented Sep 5, 2018

Stepped into this issue as well. Unfortunately the solution with function wrapper does not work for me as I have to reexport arrays.

@tanriol

This comment has been minimized.

Copy link

@tanriol tanriol commented Sep 22, 2018

The workaround does not work for me either as I need to (re)export variadic functions which cannot be defined in Rust.

@joshtriplett

This comment has been minimized.

Copy link
Member

@joshtriplett joshtriplett commented Sep 22, 2018

@tanriol While it doesn't address your immediate issue, in case you haven't already seen it, #2137 and rust-lang/rust#44930 may help you in the future.

acfoltzer-fastly referenced this issue in bytecodealliance/lucet Feb 22, 2019
This required some shenanigans with the probestack definition to get it exported in the final
`lucet_runtime.so`. I left a TODO in the code to revert it once [this Rust
issue](https://github.com/rust-lang/rust/issues/36342) is fixed.
acfoltzer-fastly referenced this issue in bytecodealliance/lucet Feb 23, 2019
Moving the stack probe into the compiled Lucet modules lets us dodge Rust's current inability to
reexport dynamic symbols (see <https://github.com/rust-lang/rust/issues/36342>). It loses a small
amount of fidelity that we got with stack overflow traps previously, as we can't distinguish a stack
overflow originating in the stack probe from anywhere else.

The C test suites are now parameterized over region, much like the Rust test suites.
@marmeladema

This comment has been minimized.

Copy link

@marmeladema marmeladema commented Apr 10, 2019

I encountered this issue on Linux using rust 1.33.0. The trick to set lto to true and incremental to false works fine if the crate type is only cdylib, if you add rlib for example crate-type = ["cdylib", "rlib"] then no imported symbols are re-exported.

@DianaNites

This comment has been minimized.

Copy link

@DianaNites DianaNites commented Jan 16, 2020

I just ran into, and spent hours, debugging this. :(

In my case I can trivially wrap it but it'd be nice if I could tell Rust not to throw away my symbol :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.