Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Depending on the gcc version, doing echo .. | nim c -r - gives "can't open nm output" error #14060

Closed
kaushalmodi opened this issue Apr 21, 2020 · 18 comments

Comments

@kaushalmodi
Copy link
Contributor

kaushalmodi commented Apr 21, 2020

Hello,

At work, we are using gcc 6.1.0 for stability reasons. With that gcc, passing nim code to nim via the stdin results in an error.

That issue does not happen on gcc 9.1.0.

Example

echo echo NimVersion | nim c -r -  

Current Output

I get collect2: fatal error: can't open nm output: Success when I have gcc 6.1.0 loaded in my env.

Hint: used config file '/home/kmodi/usr_local/apps/6/nim/devel/config/nim.cfg' [Conf]
Hint: used config file '/home/kmodi/usr_local/apps/6/nim/devel/config/config.nims' [Conf]
Hint: used config file '/home/kmodi/.config/nim/config.nims' [Conf]
Hint:  [Link]
collect2: fatal error: can't open nm output: Success
compilation terminated.
Error: execution of an external program failed: 'gcc   -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl'

Here's the same output with more debug info:

echo echo NimVersion | nim c -r --listcmd --hint:cc:on --hint:link:on -
Hint: used config file '/home/kmodi/usr_local/apps/6/nim/devel/config/nim.cfg' [Conf]
Hint: used config file '/home/kmodi/usr_local/apps/6/nim/devel/config/config.nims' [Conf]
Hint: used config file '/home/kmodi/.config/nim/config.nims' [Conf]
CC: stdlib_system.nim: gcc -c  -w   -I/home/kmodi/usr_local/apps/6/nim/devel/lib -I/cad/adi/apps/nim -o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c
Hint: gcc   -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl [Link]
collect2: fatal error: can't open nm output: Success
compilation terminated.
Error: execution of an external program failed: 'gcc   -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl'

Expected Output

I get this when using gcc 9.1.0 (but I cannot have that gcc version loaded in my work env).

Hint: used config file '/home/kmodi/usr_local/apps/6/nim/devel/config/nim.cfg' [Conf]
Hint: used config file '/home/kmodi/usr_local/apps/6/nim/devel/config/config.nims' [Conf]
Hint: used config file '/home/kmodi/.config/nim/config.nims' [Conf]
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: ../../../../../stdinfile.nim
Hint:  [Link]
Hint: 47906 LOC; 4.360 sec; 61.984MiB peakmem; Debug build; proj: stdinfile; out: /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile [SuccessX]
Hint: /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  [Exec]
1.3.1

Additional Information

  • While echo echo NimVersion | nim c -r - gives this error, if I run the autogenerated /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile executable, I get the expected echo output.

  • I have seen this issue before too on older nim versions, but I did not figure out then that that issue had to do with gcc version: Passing code via stdin to Nim stopped working [regression Nim 0.19+] #11294 (comment)

  • The same issue is seen with a bit newer gcc 7.2.0. The next gcc version I have after that is 9.1.0 (and I don't see that issue with gcc 9.1.0)

  • I get the same error on gcc 6.1.0 when running echo echo NimVersion | nim c - (without -r)

$ nim -vNim Compiler Version 1.3.1 [Linux: amd64]
Compiled at 2020-04-21
Copyright (c) 2006-2020 by Andreas Rumpf

git hash: 5608a4e57b521f2a31b35b4c7f905a87b6e7b0aa
active boot switches: -d:release
  • OS : RHEL 6.8
@alaviss
Copy link
Collaborator

alaviss commented Apr 22, 2020

It appears to me a gcc bug? Can you try --genScript then use the script to compile? If that doesn't work as well I think we'll have to attribute this to gcc.

@kaushalmodi
Copy link
Contributor Author

I tried:

echo echo NimVersion | nim c --genScript -

That created:

gcc -c  -w   -I/home/kmodi/usr_local/apps/6/nim/devel/lib -I/home/kmodi/sandbox -o stdlib_io.nim.c.o stdlib_io.nim.c
gcc -c  -w   -I/home/kmodi/usr_local/apps/6/nim/devel/lib -I/home/kmodi/sandbox -o stdlib_system.nim.c.o stdlib_system.nim.c
gcc -c  -w   -I/home/kmodi/usr_local/apps/6/nim/devel/lib -I/home/kmodi/sandbox -o @m..@s..@s..@s..@sstdinfile.nim.c.o @m..@s..@s..@s..@sstdinfile.nim.c
gcc   -o stdinfile  stdlib_io.nim.c.o stdlib_system.nim.c.o @m..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl

When I run the script, it creates stdinfile executable which runs fine.

Note that even when echo echo NimVersion | nim c -r - fails with that error, it still generates that stdinfile executable that runs perfectly fine if I run it after that error. Above command generates:

km²/tmp:../.cache/nim/stdinfile_d> lta
.rw-r--r-- 4.0k kmodi 21 Apr 21:57 @m..@s..@s..@s..@s..@sstdinfile.nim.c
.rw-r--r-- 240k kmodi 21 Apr 21:57 stdlib_system.nim.c
.rw-r--r-- 4.6k kmodi 21 Apr 21:57 stdlib_io.nim.c
.rw-r--r-- 3.7k kmodi 21 Apr 21:57 stdlib_io.nim.c.o
.rw-r--r-- 122k kmodi 21 Apr 21:57 stdlib_system.nim.c.o
.rw-r--r-- 3.8k kmodi 21 Apr 21:57 @m..@s..@s..@s..@s..@sstdinfile.nim.c.o
.rwxr-xr-x  91k kmodi 21 Apr 21:57 stdinfile*

@kaushalmodi
Copy link
Contributor Author

It's almost as if older gcc is returning this collect2: fatal error: can't open nm output which is perceived as a fatal error by nim compiler internally, but gcc still does things fine inspite of that.

@alaviss
Copy link
Collaborator

alaviss commented Apr 22, 2020

It's almost as if older gcc is returning this collect2: fatal error: can't open nm output which is perceived as a fatal error by nim compiler internally, but gcc still does things fine inspite of that.

We don't parse the output of those commands, they are non standardized anyway, instead we check the exit code:

Nim/compiler/extccomp.nim

Lines 862 to 864 in bff742c

if exitCode != 0:
rawMessage(conf, errGenerated, "execution of an external compiler program '" &
cmds[idx] & "' failed with exit code: " & $exitCode & "\n\n")

If your gcc doesn't return a clean exit code, then we can only assume that something is wrong.

@alaviss
Copy link
Collaborator

alaviss commented Apr 22, 2020

Does the same error message happen in the script? You can add set -e to the beginning of the script to make it fails if any of the commands return an unclean error code.

If the same happens with the script, then there's something wrong with your gcc.

@kaushalmodi
Copy link
Contributor Author

I changed the script to:

#!/usr/bin/env bash

set -e

gcc -c  -w   -I/home/kmodi/usr_local/apps/6/nim/devel/lib -I/tmp/kmodi/.cache/nim/stdinfile_d -o stdlib_io.nim.c.o stdlib_io.nim.c
gcc -c  -w   -I/home/kmodi/usr_local/apps/6/nim/devel/lib -I/tmp/kmodi/.cache/nim/stdinfile_d -o stdlib_system.nim.c.o stdlib_system.nim.c
gcc -c  -w   -I/home/kmodi/usr_local/apps/6/nim/devel/lib -I/tmp/kmodi/.cache/nim/stdinfile_d -o @m..@s..@s..@s..@s..@s..@sstdinfile.nim.c.o @m..@s..@s..@s..@s..@s..@sstdinfile.nim.c
gcc   -o stdinfile  stdlib_io.nim.c.o stdlib_system.nim.c.o @m..@s..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl

It ran and finished fine and generated the stdinfile executable.

@kaushalmodi
Copy link
Contributor Author

kaushalmodi commented Apr 22, 2020

If your gcc doesn't return a clean exit code, then we can only assume that something is wrong.

The surprising thing is that the executable is getting generated fine inspite of collect2: fatal error: can't open nm output

image

@kaushalmodi
Copy link
Contributor Author

If I run echo echo NimVersion | nim c -r --verbosity:3 -, I get:

/home/kmodi/usr_local/apps/6/nim/devel/lib/system/io.nim(818, 1) Hint: 15925015 [Processing]
/home/kmodi/usr_local/apps/6/nim/devel/lib/system/io.nim(821, 1) Hint: 15925019 [Processing]
/home/kmodi/usr_local/apps/6/nim/devel/lib/system/io.nim(842, 1) Hint: 15925040 [Processing]
/home/kmodi/usr_local/apps/6/nim/devel/lib/system.nim(3018, 1) Hint: 15926014 [Processing]
/home/kmodi/usr_local/apps/6/nim/devel/lib/system.nim(3020, 1) Hint: 15926027 [Processing]
???(0, 0) Hint: 15926614 [Processing]
stdinfile.nim(1, 6) Hint: 15927003 [Processing]

       ^
???(0, 0) Hint: 15927026 [Processing]

Hint: gcc   -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl [Link]
collect2: fatal error: can't open nm output: Success
compilation terminated.
Error: execution of an external program failed: 'gcc   -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl'

Does that give any clue?

@kaushalmodi
Copy link
Contributor Author

kaushalmodi commented Apr 22, 2020

If I switch to gcc 9.1.0, I get that same ??(0, 0) Hint .. when running with verbosity 3:

/home/kmodi/usr_local/apps/6/nim/devel/lib/system/io.nim(842, 1) Hint: 15925040 [Processing]
/home/kmodi/usr_local/apps/6/nim/devel/lib/system.nim(3018, 1) Hint: 15926014 [Processing]
/home/kmodi/usr_local/apps/6/nim/devel/lib/system.nim(3020, 1) Hint: 15926027 [Processing]
???(0, 0) Hint: 15926614 [Processing]
stdinfile.nim(1, 6) Hint: 15927003 [Processing]

       ^
???(0, 0) Hint: 15927026 [Processing]

Hint: gcc   -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm   -ldl [Link]
Hint: 47906 LOC; 0.717 sec; 61.445MiB peakmem; Debug build; proj: stdinfile; out: /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile [SuccessX]
[GC] total memory: 64430080
[GC] occupied memory: 56455760
[GC] stack scans: 3973
[GC] stack cells: 2929
[GC] cycle collections: 0
[GC] max threshold: 0
[GC] zct capacity: 2304
[GC] max cycle table size: 0
[GC] max pause time [ms]: 0
[GC] max stack size: 58088

Hint: /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  [Exec]
1.3.1

@alaviss
Copy link
Collaborator

alaviss commented Apr 22, 2020

If your gcc doesn't return a clean exit code, then we can only assume that something is wrong.

The surprising thing is that the executable is getting generated fine inspite of collect2: fatal error: can't open nm output

Well the clean exit code is the universal contract :P

I think what error out was gcc attempting to inspect the generated executable, but I'm not sure.

It ran and finished fine and generated the stdinfile executable.

Hmm, gcc might not be playing well with how we spawn the processes. This might be a bug that got fixed in the recent gcc versions.

@alaviss
Copy link
Collaborator

alaviss commented Apr 22, 2020

Can you try --parallelBuild:1?

@kaushalmodi
Copy link
Contributor Author

--parallelBuild:1 also gave the same error.

@alaviss
Copy link
Collaborator

alaviss commented Apr 22, 2020

Try this then: --parallelBuild:1 --passC:-v --passL:-v

It should make gcc print out exactly what it's doing.

At this point I'm not sure if it's a Nim problem. With --parallelBuild:1 we are using system() to call gcc, which should be similar to how the shell calls gcc :/

@kaushalmodi
Copy link
Contributor Author

Try this then: --parallelBuild:1 --passC:-v --passL:-v

So what ever that collect2 is doing is failing:

GNU C11 (GCC) version 6.1.0 (x86_64-pc-linux-gnu)
        compiled by GNU C version 6.1.0, GMP version 6.0.0, MPFR version 3.1.2, MPC version 1.0.2, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 259643b47d2569a7cf09e8b7a67091e7
COLLECT_GCC_OPTIONS='-c' '-w' '-v' '-I' '/home/kmodi/usr_local/apps/6/nim/devel/lib' '-I' '/home/kmodi/sandbox/nim' '-o' '/tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o' '-mtune=generic' '-march=x86-64'
 as -v -W -I /home/kmodi/usr_local/apps/6/nim/devel/lib -I /home/kmodi/sandbox/nim --64 -o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o /tmp/ccvkFqvG.s
GNU assembler version 2.20.51.0.2 (x86_64-redhat-linux) using BFD version version 2.20.51.0.2-5.44.el6 20100205
COMPILER_PATH=/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/
LIBRARY_PATH=/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/../lib64/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-c' '-w' '-v' '-I' '/home/kmodi/usr_local/apps/6/nim/devel/lib' '-I' '/home/kmodi/sandbox/nim' '-o' '/tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o' '-mtune=generic' '-march=x86-64'
Hint:  [Link]
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/6.1.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../configure --prefix=/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0 --with-mpfr=/cad/adi/apps/gnu/linux/x86_64/6 --with-mpc=/cad/adi/apps/gnu/linux/x86_64/6
Thread model: posix
gcc version 6.1.0 (GCC)
COMPILER_PATH=/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/
LIBRARY_PATH=/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/../lib64/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/:/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-o' '/tmp/kmodi/.cache/nim/stdinfile_d/stdinfile' '-v' '-mtune=generic' '-march=x86-64'
 /cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/libexec/gcc/x86_64-pc-linux-gnu/6.1.0/collect2 --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile /usr/lib/../lib64/crt1.o /usr/lib/../lib64/crti.o /cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/crtbegin.o -L/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/../lib64 -L/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0 -L/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib -L/cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/../../.. /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o -lm -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /cad/adi/apps/gnu/linux/x86_64/6/local/gcc/6.1.0/lib/gcc/x86_64-pc-linux-gnu/6.1.0/crtend.o /usr/lib/../lib64/crtn.o
collect2: fatal error: can't open nm output: Success
compilation terminated.
Error: execution of an external program failed: 'gcc   -o /tmp/kmodi/.cache/nim/stdinfile_d/stdinfile  /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_io.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/stdlib_system.nim.c.o /tmp/kmodi/.cache/nim/stdinfile_d/@m..@s..@s..@s..@s..@sstdinfile.nim.c.o  -lm  -v  -ldl'

@alaviss
Copy link
Collaborator

alaviss commented Apr 22, 2020

Well I have absolutely no idea why it's failing, maybe you can try a slightly newer gcc? Like gcc 6.5 (latest release of gcc 6)?

@kaushalmodi
Copy link
Contributor Author

I can switch among gcc 6.1.0, 7.2.0 and 9.1.0, but 6.1.0 needs to be the default due to a tool dependency. I see this issue in both 6.1.0 and 7.2.0, but not 9.1.0.

I really thank you for your time!

At the very least, this issue might serve to just document this issue with old gcc versions.

@kaushalmodi
Copy link
Contributor Author

kaushalmodi commented Apr 22, 2020

I have reported this issue to the gcc mailing list: https://gcc.gnu.org/pipermail/gcc-help/2020-April/000184.html

I hope we get a solution from there.

@kaushalmodi
Copy link
Contributor Author

kaushalmodi commented Jun 10, 2021

Closing this ticket this issue seems to be fixed in gcc 6.3.0.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants