New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

under latest MinGW, cannot link with C code using stdout #47048

Open
tspiteri opened this Issue Dec 28, 2017 · 20 comments

Comments

Projects
None yet
9 participants
@tspiteri
Contributor

tspiteri commented Dec 28, 2017

I'm having an issue under MinGW and I'm not sure if it's a MinGW issue or a rustc issue. On the one hand the bug crops up after an update to two MinGW packages, but on the other hand I cannot reproduce with only C, and I don't know if rustc is using MinGW in some undocumented unsupported way.

say_hi.c:

#include <stdio.h>
void say_hi(void) {
    fprintf(stdout, "hi!\n");
}

c_main.c

void say_hi(void);
int main(void) {
    say_hi();
    return 0;
}

r_main.rs:

extern "C" {
    fn say_hi();
}
fn main() {
    unsafe {
        say_hi();
    }
}

Under the latest MinGW, specifically with packages

  • mingw-w64-x86_64-crt-git-6.0.0.5066.61efe559-1 and
  • mingw-w64-x86_64-headers-git-6.0.0.5066.61efe559-1,

using only gcc like this works:

$ gcc -c say_hi.c
$ ar cr libsay_hi.a say_hi.o
$ gcc c_main.c -L. -lsay_hi -o c_main.exe && ./c_main.exe
hi!

But it doesn't work with rustc:

$ gcc -c say_hi.c
$ ar cr libsay_hi.a say_hi.o
$ rustc r_main.rs -L. -lsay_hi -o r_main.exe && ./r_main.exe
error: linking with `gcc` failed: exit code: 1
  |
  = note: "gcc" "-Wl,--enable-long-section-names" "-fno-use-linker-plugin" "-Wl,--nxcompat" "-nostdlib" "-m64" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\crt2.o" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsbegin.o" "-L" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "r_main.r_main0.rust-cgu.o" "-o" "r_main.exe" "r_main.crate.allocator.rust-cgu.o" "-Wl,--gc-sections" "-nodefaultlibs" "-L" "." "-L" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-l" "say_hi" "-Wl,-Bstatic" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd-60d4a252bdda9250.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc_system-a05404e5e059417a.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librand-81cbf6f1d5a19206.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libpanic_unwind-0b63e20460046692.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc-f824a78e715239af.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd_unicode-2abaea46f194a6e1.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libunwind-7daa32005d584d8a.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liblibc-49ff25caaa6e945f.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcore-f2bd2d524c6ace24.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcompiler_builtins-c6139fac5e899293.rlib" "-Wl,-Bdynamic" "-l" "advapi32" "-l" "ws2_32" "-l" "userenv" "-l" "shell32" "-Wl,-Bstatic" "-l" "gcc_eh" "-l" "pthread" "-Wl,-Bdynamic" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-luser32" "-lkernel32" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsend.o"
  = note: ./libsay_hi.a(say_hi.o):say_hi.c:(.text+0x10): undefined reference to `__imp___acrt_iob_func'
          

error: aborting due to previous error

If I downgrade those two packages to

  • mingw-w64-x86_64-crt-git-5.0.0.5002.34a7c1c0-1 and
  • mingw-w64-x86_64-headers-git-5.0.0.5002.34a7c1c0-1,

it works with rustc too:

$ gcc -c say_hi.c
$ ar cr libsay_hi.a say_hi.o
$ rustc r_main.rs -L. -lsay_hi -o r_main.exe && ./r_main.exe
hi!

One last thing, if I compile the library using the old packages, and then upgrade the packages, both gcc c_main.c ... and rustc r_main.rs ... work. If I compile the library using the new packages, and then downgrade the packages, both gcc c_main.c ... and rustc r_main.rs ... fail.

@mati865

This comment has been minimized.

Contributor

mati865 commented Dec 29, 2017

Mingw-w64 seen some huge changes lately, if recompiling Rust with latest mingw-w64 doesn't work or still gives errors like that then it's probably a regression they introduced.

@cretz

This comment has been minimized.

cretz commented Feb 4, 2018

Got anyone else arriving here across the web w/ new MinGW breaks, in addition to:

pacman -U /var/cache/pacman/pkg/mingw-w64-x86_64-crt-git-5.0.0.4745.d2384c2-1-any.pkg.tar.xz
pacman -U /var/cache/pacman/pkg/mingw-w64-x86_64-headers-git-5.0.0.4747.0f8f626-1-any.pkg.tar.xz

You might also need to downgrade winpthreads (granted your version numbers may be different):

pacman -U /var/cache/pacman/pkg/mingw-w64-x86_64-winpthreads-git-5.0.0.4741.2c8939a-1-any.pkg.tar.xz /var/cache/pacman/pkg/mingw-w64-x86_64-libwinpthread-git-5.0.0.4741.2c8939a-1-any.pkg.tar.xz

Had to do both at once due to codeps. I ran into this Rust+MinGW issue when statically building latest Tor and deps w/ latest MinGW.

@tspiteri

This comment has been minimized.

Contributor

tspiteri commented Feb 5, 2018

The old packages are also available at http://repo.msys2.org/mingw/x86_64/, where you can download the latest versions of the packages in question that include git-5.0.0 in their version rather than the new incompatible git-6.0.0. This could be required for example on a new MinGW installation where there are no git-5.0.0 packages in /var/cache/pacman/pkg.

@lins05

This comment has been minimized.

lins05 commented Feb 9, 2018

Reached here by googling.

@cretz thanks for the tip, it works

@csstaub csstaub referenced this issue Feb 16, 2018

Merged

gvt update -all #145

@mati865

This comment has been minimized.

Contributor

mati865 commented Mar 4, 2018

Should be fixed with crt/headers packages in version 6.0.0.5114.b9400293. They are already available in the repository.

@tspiteri

This comment has been minimized.

Contributor

tspiteri commented Mar 5, 2018

I upgraded to version 6.0.0.5114 and reproduced the issue. Downgrading to 5.0.0.5002 still works.

(Keep in mind that if the library is built using 5.0.0.5002, linking will work using 6.0.0.*.)

@mati865

This comment has been minimized.

Contributor

mati865 commented Mar 5, 2018

Similar issue was fixed recently Alexpux/MINGW-packages#3469.

Maybe rust internals have to be recompiled.

@NovemberZulu

This comment has been minimized.

Contributor

NovemberZulu commented Mar 31, 2018

Still happens for with rustc 1.25.0 (package in question is cargo-update). May be Rust build machine has older MinGW?

@mati865

This comment has been minimized.

Contributor

mati865 commented Mar 31, 2018

Yeah, I just forgot that Rust is using mingw-w64-builds toolchain which is still based on 5.x mingw-w64 releases.
That basically makes official Rust unusable within MSYS2.

@NovemberZulu

This comment has been minimized.

Contributor

NovemberZulu commented Mar 31, 2018

The build instructions for for windows-gnu specifically suggest using MSYS2 :( . Could you please elaborate a bit about that mingw-w64-builds toolchain?

@mati865

This comment has been minimized.

Contributor

mati865 commented Mar 31, 2018

Bundling /mingw64/x86_64-w64-mingw32/lib/libcrtdll.a from MSYS2 toolchain could help but somebody would have to test it properly.

Could you please elaborate a bit about that mingw-w64-builds toolchain?

https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/

@NovemberZulu

This comment has been minimized.

Contributor

NovemberZulu commented Mar 31, 2018

I looked around the link, and it seems that master already has __acrt_iob_func, but v5.x does not. I wasnot able to find any kind of roadmap or ETA for v6.x release.

@NovemberZulu

This comment has been minimized.

Contributor

NovemberZulu commented Apr 1, 2018

I built current Rust master using MSYS2, as described in README, and now I am able to build cargo-update without tinkering with include files. May be switching the official builds to MSYS2 is an option?

@petrochenkov

This comment has been minimized.

Contributor

petrochenkov commented Apr 1, 2018

May be switching the official builds to MSYS2 is an option?

What MSYS2 exactly?
The mingw-w64 toolchain in MSYS2 is constantly updated and these updates regularly break something.
That's why Rust uses a stable predictable standalone version from Mingw-builds.
I'd personally recommend to do the same thing and use a standalone mingw toolchain instead of MSYS2 package (at least that's what I do).

It may make sense to bump the version used by Rust from 6.3.0 to something up-to-date though.

@NovemberZulu

This comment has been minimized.

Contributor

NovemberZulu commented Apr 1, 2018

What MSYS2 exactly?

Some reasonably stable reasonably recent version :) I know this is self-contradictory.

That's why Rust uses a stable predictable standalone version from Mingw-builds.

I totally see the point, the issue is the build environment is not very well documented. Updating README with proper setup instructions for Windows GNU will help a lot.

@mati865

This comment has been minimized.

Contributor

mati865 commented Apr 1, 2018

Using mingw-builds toolchain with MSYS2 is not perfect either.
If you want to use libs from MSYS2 with different toolchain you can also run into this error, like here.

@tspiteri

This comment has been minimized.

Contributor

tspiteri commented May 30, 2018

This build script can work around the issue. It is a bit hacky; basically it tests for the bug using this issue's test case, and if it detects the bug it creates and links libworkaround_47048.a which contains the __imp___acrt_iob_func symbol as defined in newer MinGW, which is apparently not included when linking using rustc. This only works around the original issue, and does not handle the further issues for winpthreads raised in #47048 (comment).

use std::env;
use std::fs::{self, File};
use std::io::Write;
use std::path::{Path, PathBuf};
use std::process::Command;

fn main() {
    let target = env::var_os("TARGET")
        .expect("TARGET")
        .into_string()
        .expect("TARGET");
    if target.contains("-windows-gnu") {
        mingw_check_47048();
    }
}

fn mingw_check_47048() {
    let rustc = env::var_os("RUSTC").expect("RUSTC");
    let out_dir = PathBuf::from(env::var_os("OUT_DIR").expect("OUT_DIR"));
    let try_dir = out_dir.join("try_47048");
    let mut cmd;

    fs::create_dir_all(&try_dir).expect("create directory");
    create_file(&try_dir.join("say_hi.c"), SAY_HI_C);
    create_file(&try_dir.join("c_main.c"), C_MAIN_C);
    create_file(&try_dir.join("r_main.rs"), R_MAIN_RS);
    create_file(&try_dir.join("workaround.c"), WORKAROUND_C);

    cmd = Command::new("gcc");
    cmd.current_dir(&try_dir).args(&["-fPIC", "-c", "say_hi.c"]);
    execute(cmd);

    cmd = Command::new("ar");
    cmd.current_dir(&try_dir)
        .args(&["cr", "libsay_hi.a", "say_hi.o"]);
    execute(cmd);

    cmd = Command::new("gcc");
    cmd.current_dir(&try_dir)
        .args(&["c_main.c", "-L.", "-lsay_hi", "-o", "c_main.exe"]);
    execute(cmd);

    // try simple rustc command that should work, so that failure
    // really is the bug being checked for
    cmd = Command::new(&rustc);
    cmd.arg("--version");
    execute(cmd);

    cmd = Command::new(&rustc);
    cmd.current_dir(&try_dir)
        .args(&["r_main.rs", "-L.", "-lsay_hi", "-o", "r_main.exe"]);
    let status = cmd
        .status()
        .unwrap_or_else(|_| panic!("Unable to execute: {:?}", cmd));
    let need_workaround = !status.success();

    // build and test libworkaround_47048.a
    if need_workaround {
        cmd = Command::new("gcc");
        cmd.current_dir(&try_dir).args(&["-fPIC", "-O2", "-c", "workaround.c"]);
        execute(cmd);

        cmd = Command::new("ar");
        cmd.current_dir(&try_dir)
            .args(&["cr", "libworkaround_47048.a", "workaround.o"]);
        execute(cmd);

        cmd = Command::new(&rustc);
        cmd.current_dir(&try_dir).args(&[
            "r_main.rs",
            "-L.",
            "-lsay_hi",
            "-lworkaround_47048",
            "-o",
            "r_main.exe",
        ]);
        execute(cmd);

        let src = try_dir.join("libworkaround_47048.a");
        let lib_dir = out_dir.join("lib");
        fs::create_dir_all(&lib_dir).expect("create directory");
        let dst = lib_dir.join("libworkaround_47048.a");
        fs::rename(src, dst).expect("move file");

        let lib_dir_str = lib_dir.to_str().expect("unsupported characters");
        println!("cargo:rustc-link-search=native={}", lib_dir_str);
        println!("cargo:rustc-link-lib=static=workaround_47048");
    }

    fs::remove_dir_all(try_dir).expect("remove directory");
}

fn create_file(filename: &Path, contents: &str) {
    let mut file = File::create(filename)
        .unwrap_or_else(|_| panic!("Unable to create file: {:?}", filename));
    file.write_all(contents.as_bytes())
        .unwrap_or_else(|_| panic!("Unable to write to file: {:?}", filename));
}

fn execute(mut command: Command) {
    let status = command
        .status()
        .unwrap_or_else(|_| panic!("Unable to execute: {:?}", command));
    if !status.success() {
        if let Some(code) = status.code() {
            panic!("Program failed with code {}: {:?}", code, command);
        } else {
            panic!("Program failed: {:?}", command);
        }
    }
}

const SAY_HI_C: &'static str = r#"/* say_hi.c */
#include <stdio.h>
void say_hi(void) {
    fprintf(stdout, "hi!\n");
}
"#;

const C_MAIN_C: &'static str = r#"/* c_main.c */
void say_hi(void);
int main(void) {
    say_hi();
    return 0;
}
"#;

const R_MAIN_RS: &'static str = r#"// r_main.rs
extern "C" {
    fn say_hi();
}
fn main() {
    unsafe {
        say_hi();
    }
}
"#;

const WORKAROUND_C: &'static str = r#"/* workaround.c */
#define _CRTBLD
#include <stdio.h>

FILE *__cdecl __acrt_iob_func(unsigned index)
{
    return &(__iob_func()[index]);
}

typedef FILE *__cdecl (*_f__acrt_iob_func)(unsigned index);
_f__acrt_iob_func __MINGW_IMP_SYMBOL(__acrt_iob_func) = __acrt_iob_func;
"#;
@mati865

This comment has been minimized.

Contributor

mati865 commented Jun 1, 2018

Recent mingw-w64 GCC 8.1.0 build contains __acrt_iob_func so it can be hit by this issue as well.

bors added a commit that referenced this issue Jul 6, 2018

Auto merge of #51989 - mati865:gcc8, r=Mark-Simulacrum
Build Rust with MinGW GCC 8.1.0

I've tested it locally on Windows 10 and Wine and it seems to fix #47048.
GDB also should be fixed: #40184 (comment)

This commit is not meant to be merged as is but to run tests with new toolchain.
If it succeeds it can be considered to push toolchain packages to AWS and begin using them.

bors added a commit that referenced this issue Jul 11, 2018

Auto merge of #51989 - mati865:gcc8, r=alexcrichton
Build Rust with MinGW GCC 8.1.0

I've tested it locally on Windows 10 and Wine and it seems to fix #47048.
GDB also should be fixed: #40184 (comment)

This commit is not meant to be merged as is but to run tests with new toolchain.
If it succeeds it can be considered to push toolchain packages to AWS and begin using them.

bors added a commit that referenced this issue Jul 24, 2018

Auto merge of #51989 - mati865:gcc8, r=alexcrichton
Build Rust with MinGW GCC 8.1.0

I've tested it locally on Windows 10 and Wine and it seems to fix #47048.
GDB also should be fixed: #40184 (comment)

This commit is not meant to be merged as is but to run tests with new toolchain.
If it succeeds it can be considered to push toolchain packages to AWS and begin using them.

bors added a commit that referenced this issue Jul 25, 2018

Auto merge of #51989 - mati865:gcc8, r=alexcrichton
Build Rust with MinGW GCC 8.1.0

I've tested it locally on Windows 10 and Wine and it seems to fix #47048.
GDB also should be fixed: #40184 (comment)

This commit is not meant to be merged as is but to run tests with new toolchain.
If it succeeds it can be considered to push toolchain packages to AWS and begin using them.

bors added a commit that referenced this issue Jul 25, 2018

Auto merge of #51989 - mati865:gcc8, r=alexcrichton
Build Rust with MinGW GCC 8.1.0

I've tested it locally on Windows 10 and Wine and it seems to fix #47048.
GDB also should be fixed: #40184 (comment)

This commit is not meant to be merged as is but to run tests with new toolchain.
If it succeeds it can be considered to push toolchain packages to AWS and begin using them.

mykmelez added a commit to mykmelez/lmdb-rs that referenced this issue Jul 27, 2018

danburkert added a commit to danburkert/lmdb-rs that referenced this issue Aug 4, 2018

Update .appveyor.yml
Linking lmdb with MinGW has begun failing due to [1], so this commit
disables it.

[1]: rust-lang/rust#47048

danburkert added a commit to danburkert/lmdb-rs that referenced this issue Aug 4, 2018

Update .appveyor.yml
Linking lmdb with MinGW has begun failing due to [1], so this commit
disables it.

[1]: rust-lang/rust#47048

danburkert added a commit to danburkert/lmdb-rs that referenced this issue Aug 4, 2018

Update .appveyor.yml
Linking lmdb with MinGW has begun failing due to [1], so this commit
disables it.

[1]: rust-lang/rust#47048

pkgw added a commit to pkgw/tectonic that referenced this issue Oct 8, 2018

.appveyor.yml: try CI'ing on the MSVC toolchain rather than GNU
Based on [rust#47048](rust-lang/rust#47048), it
seems that the combination of Rust and the GNU toolchain is a bit dicey to
work with. This PR works on my personal Windows VM which uses the MSVC
toolchain, and the conda-forge packages also use the MSVC toolchain, so let's
try switching to that. If that works, we'll decide that Windows+GNU isn't
supported ... although it was just working recently, so hopefully it wouldn't
be *too* hard to get it into shape again.

pkgw added a commit to pkgw/tectonic that referenced this issue Oct 8, 2018

.appveyor.yml: try CI'ing on the MSVC toolchain rather than GNU
Based on [rust#47048](rust-lang/rust#47048), it
seems that the combination of Rust and the GNU toolchain is a bit dicey to
work with. This PR works on my personal Windows VM which uses the MSVC
toolchain, and the conda-forge packages also use the MSVC toolchain, so let's
try switching to that. If that works, we'll decide that Windows+GNU isn't
supported ... although it was just working recently, so hopefully it wouldn't
be *too* hard to get it into shape again.

pkgw added a commit to pkgw/tectonic that referenced this issue Oct 8, 2018

.appveyor.yml: try CI'ing on the MSVC toolchain rather than GNU
Based on [rust#47048](rust-lang/rust#47048), it
seems that the combination of Rust and the GNU toolchain is a bit dicey to
work with. This PR works on my personal Windows VM which uses the MSVC
toolchain, and the conda-forge packages also use the MSVC toolchain, so let's
try switching to that. If that works, we'll decide that Windows+GNU isn't
supported ... although it was just working recently, so hopefully it wouldn't
be *too* hard to get it into shape again.

pkgw added a commit to pkgw/tectonic that referenced this issue Oct 8, 2018

.appveyor.yml: try CI'ing on the MSVC toolchain rather than GNU
Based on [rust#47048](rust-lang/rust#47048), it
seems that the combination of Rust and the GNU toolchain is a bit dicey to
work with. This PR works on my personal Windows VM which uses the MSVC
toolchain, and the conda-forge packages also use the MSVC toolchain, so let's
try switching to that. If that works, we'll decide that Windows+GNU isn't
supported ... although it was just working recently, so hopefully it wouldn't
be *too* hard to get it into shape again.

pkgw added a commit to pkgw/tectonic that referenced this issue Oct 8, 2018

.appveyor.yml: try CI'ing on the MSVC toolchain rather than GNU
Based on [rust#47048](rust-lang/rust#47048), it
seems that the combination of Rust and the GNU toolchain is a bit dicey to
work with. This PR works on my personal Windows VM which uses the MSVC
toolchain, and the conda-forge packages also use the MSVC toolchain, so let's
try switching to that. If that works, we'll decide that Windows+GNU isn't
supported ... although it was just working recently, so hopefully it wouldn't
be *too* hard to get it into shape again.
@mati865

This comment has been minimized.

Contributor

mati865 commented Nov 9, 2018

I'm not sure who is responsible for Windows GNU, @retep998?

I assume libs from #47359 are necessary for people who don't have GCC in their PATH. However they are an issue for people with GCC/MinGW versions different from Rust ones.
They are built in specific environment and the fact they work in (some) other environments is just a coincidence that shouldn't be relied upon. Yet Windows GNU target prioritises them over host libraries and relies on host headers causing conflicts like undefined reference to __imp___acrt_iob_func'` from this issue.

It can be solved by:

  • using host libraries over bundled if they exists
  • bundling headers (66 MiB) and using them instead of host
@retep998

This comment has been minimized.

Member

retep998 commented Nov 9, 2018

Rustup does not currently offer a way to opt out of the bundled MinGW, but there is a way to work around that. If you use the x86_64-pc-windows-msvc toolchain and add the 86_64-pc-windows-gnu-target you will be able to cross compile to x86_64-pc-windows-gnu without any bundled MinGW and instead relying entirely on your locally installed MinGW.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment