Linker error while compiling cryptography-rust

[Tarnum-tst]

I don't know why this issue plagued me and what is the culprit.
I am trying Windows build for latest pyca cryptography 36 dev1 and got linker error while compiling cryptography-rust:

"C:\Python37\cryptography-main\src\rust\target\i686-pc-windows-gnu\release\deps\cryptography_rust.cryptography_rust.8yeboqs1-cgu.12.rcgu.o:cryptography_rust.: undefined reference to _imp___ZN3pem11ASCII_ARMOR17hda6e74d90990760eE
collect2.exe: error: ld returned 1 exit status"

Missing function should obviously be named ZN3pem11ASCII_ARMOR17hda6e74d90990760eE and linked from pem library.
But why it got erroneous imp prefix? No idea.

I'm using 32-bit environment with Python 3.7 / Rust 1.48 (i686-pc-windows-gnu) / PEM 1.0.0

[jcreekmore]

Uhm... I am not sure what is going on there. I don't have a full Python 3.7 cross-build environment for Windows 32-bit (I run macos), so I cannot just build the pyca cryptography-rust library because it fails with pyo3. I tried to strip down the cryptography-rust crate as much as I could and generated a little cdynlib crate that just linked against pem (and anyhow.... for reasons) and tried building with the following:

cargo build --target i686-pc-windows-gnu --release
   Compiling anyhow v1.0.44
   Compiling regex-syntax v0.6.25
   Compiling once_cell v1.8.0
   Compiling base64 v0.13.0
   Compiling regex v1.5.4
   Compiling pem v1.0.0
   Compiling cryptography-rust v0.1.0 (/Users/jonathan/src/cryptography/src/tmp)
warning: unused variable: `no_match_err`
  --> src/lib.rs:14:5
   |
14 |     no_match_err: &'static str,
   |     ^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_no_match_err`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `multiple_match_err`
  --> src/lib.rs:15:5
   |
15 |     multiple_match_err: &'static str,
   |     ^^^^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_multiple_match_err`

warning: 2 warnings emitted

    Finished release [optimized] target(s) in 16.02s

cryptography/src/tmp on  main [?] is 📦 v0.1.0 via 🦀 v1.48.0 took 16s
❯ rustc -V
rustc 1.48.0 (7eac88abb 2020-11-16)

This is the stripped down Cargo.toml I used:

[package]
name = "cryptography-rust"
version = "0.1.0"
authors = ["The cryptography developers <cryptography-dev@python.org>"]
edition = "2018"
publish = false

[dependencies]
anyhow = "1.0"
pem = "1.0"

[lib]
name = "cryptography_rust"
crate-type = ["cdylib"]

[profile.release]
lto = "thin"
overflow-checks = true

and this was the stripped down src/lib.rs:

#![deny(rust_2018_idioms)]

use anyhow::anyhow;

/// parse all sections in a PEM file and return the only matching section.
/// If no or multiple matching sections are found, return an error.
pub fn find_in_pem(
    data: &[u8],
    filter_fn: fn(&pem::Pem) -> bool,
    no_match_err: &'static str,
    multiple_match_err: &'static str,
) -> Result<pem::Pem, anyhow::Error> {
    let all_sections = pem::parse_many(data)?;
    if all_sections.is_empty() {
        return Err(pem::PemError::MalformedFraming.into());
    }
    let matching_sections: Vec<pem::Pem> = all_sections.into_iter().filter(filter_fn).collect();
    matching_sections
        .into_iter()
        .next()
        .ok_or_else(|| anyhow!("error"))
}

All of this seemed to link just fine, but I am obviously doing a cross-build from macos to Windows here.

From what I found, it looks like the _imp__ prefix could be put into place if there is a disconnect between 32-bit and 64-bit linking possibly? https://stackoverflow.com/a/5894798

[Tarnum-tst]

Yes, I'm building 32-bit binaries on 64-bit Windows and this maybe a culprit. My MinGW-gcc 6.1.0 have both 32- and 64-bit components. Will try to dig into it.

Will definitely write on my success/unsuccess in a few days. Thank You!

[Tarnum-tst]

So, after a detailed research I can state that gcc has nothing to do with my issue.
These obj files (cryptography_rust.cryptography_rust.8yeboqs1-cgu.0.rcgu.o-cryptography_rust.cryptography_rust.8yeboqs1-cgu.15.rcgu.o) are definitely generated by the Rust compiler (rustc).
And I only have a 32-bit Rust compiler in my system (i686-pc-windows-gnu) so conflict with 64-bit compiler is not the case.
Perhaps it is a Rust compiler itself causing my issue specific to 32-bit rustc working on 64-bit Windows?
Anyway, I only have a single missing function with imp prefix ('__imp___ZN3pem11ASCII_ARMOR17hda6e74d90990760eE') in a single obj file (currently cryptography_rust.cryptography_rust.8yeboqs1-cgu.13.rcgu.o: .text+0x22b for 0fb5dbe38af1caff128cb038b3ee5bee39e38d1a commit of pyca cryptograpy) preventing linking cryptography_rust.dll at the final stage.
The problem could be solved via hex-editing, by replacing '__imp___ZN3pem11ASCII_ARMOR17hda6e74d90990760eE' with '__ZN3pem11ASCII_ARMOR17hda6e74d90990760eE' and putting six zero bytes (0x00) at the end for keeping file size (cryptography_rust.cryptography_rust.8yeboqs1-cgu.13.rcgu.o) intact.
Voila: linking completed successfully, _rust.pyd (cryptography_rust.dll) and _openssl.pyd extensions generated.

[jcreekmore]

While I still think this is some sort of a toolchain issue, I may have a workaround if you can test the refactor-ascii-armor branch by overriding the dependency as in the cryptography crate Cargo.toml file as described in: https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html

This should be what needs to be added to the Cargo.toml file to test it out.

[patch.crates-io]
pem = { git = 'https://github.com/jcreekmore/pem-rs', branch = 'refactor-ascii-armor'  }

[Tarnum-tst]

@jcreekmore
Thank You very much! Your solution works perfectly, everything compiled just fine without any errors! Tested with latest cryptography commit 3f515a1524109ee8d4a814c4cd65066198cf52ea compiled against OpenSSL 3.0.0 - all tests passed!

[jcreekmore]

Alright, I will merge that in and make a 1.0.1 release with that change. I still don't understand why the ASCII_ARMOR symbol was being confused that way, but I imagine it was some interaction between once_cell and the 32-bit windows cross-compile and linking process.