Re-exporting C symbols for cdylib

[qinsoon]

In an extern block, C symbols can be imported to Rust. When building Rust library into dylib, the symbols stay visible and can be used from C. However, when building into newly introduced cdylib, the imported C symbols are no longer visible. I am not aware of a mechanism to re-export the C symbols.

[petrochenkov]

Does making them pub help? (More precisely, making them nameable from outside of the crate.)

[qinsoon]

@petrochenkov Declaring pub doesn't help. I assume it is because pub does not mean they are accessible from native, thus their symbols can still be safely removed.

[retep998]

Can we please have some focus brought to this. For many applications which combine both C and Rust and are trying to create a cdylib, the lack of this feature is a blocker.

[michaelwoerister]

It seems clear that we need this functionality in one form or other. Ideally we'd have an RFC to determine who this should be implemented exactly. @aturon, how do we handle this, it being the impl period now? Or maybe we can decide on a straightforward way how to express this without an RFC.

[michaelwoerister]

I'll leave this nominated for now so it doesn't fall off the radar. Hopefully we'll have a clearer picture on how to proceed before the next dev-tools meeting.

[aturon]

I think we could approach the procedural issue here in a few ways:

1. We could land the functionality under a feature gate, and have an RFC prior to stabilization
2. We could have a public discussion on thread, involving lang team, with fcp signoff
3. We could just do an RFC :)

It's not like RFCs are prohibited per se, more that we're trying to focus the teams on executing already-planned work. This is small enough that it's probably not a big distraction.

I do think that in general this area of Rust could use more design focus, FWIW.

[retep998]

Currently the recommended way to export a function that you define yourself from a cdylib or staticlib is a #[no_mangle] pub extern fn foo(...) { ... } at the crate root. So I'd like to propose a solution that is as close to the existing forms as possible with minimal changes to the language:

• Declare the external functions directly at the crate root and make them public: extern { pub fn foo(...); }
• If the external function is declared elsewhere then publicly re-export into the crate root: pub use foo::bar;

The crate root being the root of the cdylib or staticlib crate itself.

[aturon]

cc @rust-lang/lang, adding this to our list of issues that need to be triaged.

[joshtriplett]

We discussed this one in the lang team meeting. This does need an RFC; how quickly that progresses will depend on how smoothly and non-bikesheddy the RFC goes. The approach of making symbols public if pub and #[no_mangle] seems reasonable.

Also, we should stop automatically exporting such symbols from a dylib, if possible. That may be difficult without regressions, but it's a serious library hygiene issue; it prevents hiding ABI changes for a dependency inside your own library, without breaking your own ABI.

[petrochenkov]

The approach of making symbols public if pub and #[no_mangle] seems reasonable.

Oh, this is great. What is done now (if I remember correctly), e.g. heuristic-based link-time visibility based on sum of factors like crate type, ABI (non-"Rust") and no_mangle even on private items is a mess and kinda breaks the spirit of Rust privacy (even if link-time visibility is a separate issue from compile-time one).

[joshtriplett]

I do think it'd be nice to have a broader RFC that explicitly defines when symbols should be exported, across all use-cases.

[dbrgn]

I just stumbled over this. Is there any workaround that we can use in the meantime?

In the worst case I could wrapp all 3rd party functions with my own:

extern crate saltyrtc_client_ffi;

pub type salty_event_loop_t = saltyrtc_client_ffi::salty_event_loop_t;

#[no_mangle]
pub extern "C" fn salty_event_loop_new() -> *mut salty_event_loop_t {
    saltyrtc_client_ffi::salty_event_loop_new()
}

...but I hope there's a better way than this.

If not, this is really a stumbling block when trying to reuse code from FFI compatible crates...

[Michael-F-Bryan]

I just encountered this issue at work. We've got a ffi_utils crate which contains general things like error handling, null pointer checks, and exception safety and I was hoping the original symbols would be exported from a cdylib which pulls in the ffi_utils crate.

@dbrgn for now I've found a really crummy workaround.... I'm creating a macro which will manually re-export the symbols. So you "only" need to call export_error_handling functions!() to make sure the symbols are included in the final DLL.

It feels horrible, but it works...

---

#[doc(hidden)]
#[macro_export]
macro_rules! export_c_symbol {
    (fn $name:ident($( $arg:ident : $type:ty ),*) -> $ret:ty) => {
        #[no_mangle]
        pub unsafe extern "C" fn $name($( $arg : $type),*) -> $ret {
            $crate::error_handling::$name($( $arg ),*)
        }
    };
    (fn $name:ident($( $arg:ident : $type:ty ),*)) => {
        export_c_symbol!(fn $name($( $arg : $type),*) -> ());
    }
}

/// As a workaround for rust-lang/rust#6342, you can use this macro to make sure
/// the symbols for `ffi_utils`'s error handling are correctly exported in your
/// `cdylib`.
#[macro_export]
macro_rules! export_error_handling_functions {
    () => {
    export_c_symbol!(fn clear_last_error());
    export_c_symbol!(fn last_error_length() -> ::libc::c_int);
    export_c_symbol!(fn last_error_length_utf16() -> ::libc::c_int);
    export_c_symbol!(fn error_message_utf8(buf: *mut ::libc::c_char, length: ::libc::c_int) -> ::libc::c_int);
    export_c_symbol!(fn error_message_utf16(buf: *mut u16, length: ::libc::c_int) -> ::libc::c_int);
    };
}

[dbrgn]

@Michael-F-Bryan thanks. yeah, there's probably currently no way around wrapping these functions.

by the way, you should try to avoid two #[no_mangle] functions with the same name, even across crate boundaries: https://users.rust-lang.org/t/including-third-party-no-mangle-functions-in-a-cdylib/15388/2

[Michael-F-Bryan]

by the way, you should try to avoid two #[no_mangle] functions with the same name, even across crate boundaries

Yep. The first time I ran the test suite with that macro I got lots of linker errors due to duplicate symbols. I "fixed" that by removing the #[no_mangle] from the original functions so the compiler would still mangle them.

Interestingly, I never got the "duplicate symbols" error when compiling the DLL in release mode. I'm assuming rustc/LLVM inlines the original functions and then strips out their now "unused" function symbols before they get to the linking stage.

[dbrgn]

It doesn't happen when linking together two libraries with rustc, but it happened to me when trying to link two independent shared libraries (compiled from Rust) into an iOS app. There might be a way to extract the Rust stdlib into a separate shared library (similar to the way it's done with libc, if I understand this correctly), but I haven't found a way to do this so far.

But that's probably offtopic here :)

[agalakhov]

Stepped into this issue as well. Unfortunately the solution with function wrapper does not work for me as I have to reexport arrays.

[tanriol]

The workaround does not work for me either as I need to (re)export variadic functions which cannot be defined in Rust.

[joshtriplett]

@tanriol While it doesn't address your immediate issue, in case you haven't already seen it, #2137 and rust-lang/rust#44930 may help you in the future.

[marmeladema]

I encountered this issue on Linux using rust 1.33.0. The trick to set lto to true and incremental to false works fine if the crate type is only cdylib, if you add rlib for example crate-type = ["cdylib", "rlib"] then no imported symbols are re-exported.

[DianaNites]

I just ran into, and spent hours, debugging this. :(

In my case I can trivially wrap it but it'd be nice if I could tell Rust not to throw away my symbol :(

[devsnek]

So right now it is not possible to re-export, for example, libc::malloc?