Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building koch from nim devel fails when config.nims importing `os` present #10030

Closed
kaushalmodi opened this issue Dec 17, 2018 · 15 comments

Comments

Projects
None yet
5 participants
@kaushalmodi
Copy link
Contributor

commented Dec 17, 2018

I have been able to reproduce this issue on the latest Nim devel as of now ( 0409f23 ).

Steps to reproduce the issue

  1. Build koch using the csources-built nim binary as you would normally do when building Nim from devel.
  2. Create a ~/.config/nim/config.nims with just import os line.
  3. In your Nim build dir, where you build koch, do ./koch boot -d:release.

You will get something like:

km²~/downloads/:git/Nim> ./koch boot -d:release                 12/17 2:46pm
iteration: 1
compiler/nim0 c -d:release --nimcache:nimcache/r_linux_amd64 compiler/nim.nim
Hint: used config file '/home/kmodi/downloads/git/Nim/config/nim.cfg' [Conf]
Hint: used config file '/home/kmodi/downloads/git/Nim/compiler/nim.cfg' [Conf]
Hint: used config file '/home/kmodi/.config/nim/config.nims' [Conf]
../lib/system.nim(376, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
../lib/pure/os.nim(705, 14) Error: undeclared identifier: 'Stat'
FAILURE

An ugly workaround I use to allow Nim devel to build successfully on Travis is to temporarily remove the config.nims from Nim's search path: kaushalmodi/hello_musl@ed97b4b

/cc @nc-x, @Araq

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented Dec 17, 2018

(moved to #10031)

@kaushalmodi kaushalmodi changed the title Building koch from nim devel fails with config.nims present Building koch from nim devel fails when config.nims containing certain `os` module procs present Dec 17, 2018

@timotheecour

This comment has been minimized.

Copy link
Contributor

commented Dec 17, 2018

@kaushalmodi remember to use permalinks instead of links to master, which are mutable, eg: https://github.com/kaushalmodi/nim_config/blob/e38fb2d1c5d70b1ee2fe47683b706d1059f352a2/config.nims#L1

@timotheecour

This comment has been minimized.

Copy link
Contributor

commented Dec 17, 2018

Note that even though the --skipUserCfg option is available for the csources-build nim binary, it doesn't work..
See below that it still is fetching /home/kmodi/.config/nim/config.nims:

seems like a separate bug... maybe file another bug for that?

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented Dec 17, 2018

remember to use permalinks instead of links to master

Thanks for pointing that out. I have now updated that link.

I always do that! Just forgot this time.


Update: I have now removed even that link.. you don't need that whole config.nims to reproduce this issue. Simply adding import os to config.nims is enough.

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented Dec 17, 2018

seems like a separate bug... maybe file another bug for that?

Done! #10031

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented Dec 17, 2018

Even if I skip my user config in the koch building step by doing ./bin/nim --skipUserCfg c koch, I get the same failure when koch is used to build the final nim binary:

km²~/downloads/:git/Nim> ./koch boot -d:release                 12/17 2:29pm
iteration: 1
compiler/nim0 c -d:release --nimcache:nimcache/r_linux_amd64 compiler/nim.nim
Hint: used config file '/home/kmodi/downloads/git/Nim/config/nim.cfg' [Conf]
Hint: used config file '/home/kmodi/downloads/git/Nim/compiler/nim.cfg' [Conf]
Hint: used config file '/home/kmodi/.config/nim/config.nims' [Conf]
../lib/system.nim(376, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
../lib/pure/os.nim(705, 14) Error: undeclared identifier: 'Stat'
FAILURE

kaushalmodi added a commit to kaushalmodi/hello_musl that referenced this issue Dec 17, 2018

kaushalmodi added a commit to kaushalmodi/nim_config that referenced this issue Dec 17, 2018

@kaushalmodi kaushalmodi changed the title Building koch from nim devel fails when config.nims containing certain `os` module procs present Building koch from nim devel fails when config.nims importing `os` present Dec 17, 2018

@timotheecour

This comment has been minimized.

Copy link
Contributor

commented Jan 9, 2019

confirmed the bug

timotheecour added a commit to timotheecour/Nim that referenced this issue Jan 9, 2019

@Araq Araq closed this in #10244 Jan 10, 2019

Araq added a commit that referenced this issue Jan 10, 2019

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented Jan 10, 2019

@timotheecour @Araq Can you please re-open this issue? This is still not fixed.

See https://travis-ci.org/kaushalmodi/hello_musl/builds/478090554#L672-L678:

Hint: used config file '/home/travis/build/kaushalmodi/hello_musl/nim/ab425d793af1a5c0b221bfbeab382b4781479c63/config/nim.cfg' [Conf]
Hint: used config file 'koch.nim.cfg' [Conf]
Hint: used config file '/home/travis/build/kaushalmodi/hello_musl/nim/ab425d793af1a5c0b221bfbeab382b4781479c63/config/config.nims' [Conf]
lib/system.nim(392, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
lib/system.nim(392, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
lib/pure/os.nim(718, 14) Error: undeclared identifier: 'Stat'
/home/travis/.travis/job_stages: line 115: ./koch: No such file or directory

Notice here that as koch fails to build, we don't get the actual nim binary built using koch, and so the Nim version reflects the version built using csources:

Nim Compiler Version 0.19.0 [Linux: amd64]
Compiled at 2018-09-26
Copyright (c) 2006-2018 by Andreas Rumpf

I can recreate the failure locally too. You need to delete the earlier koch generated nim binary to reproduce this error locally.

Update (2019/01/15): I cannot recreate this error locally any more (I probably forgot to add --skipUserCfg switch last time). But I can still recreate this error on Travis.

kaushalmodi added a commit to kaushalmodi/nim_config that referenced this issue Jan 10, 2019

Revert "Revert "Hack: Attempt to make Nim building to work by moving …
…config.nims out""

This reverts commit 4bfb6f9.

Nope, still not fixed: nim-lang/Nim#10030 (comment)

kaushalmodi added a commit to kaushalmodi/hello_musl that referenced this issue Jan 10, 2019

Revert "Revert "Hack: Attempt to make Nim building to work by moving …
…config.nims out""

This reverts commit 06b5ca0.

Nope, still not fixed:
nim-lang/Nim#10030 (comment)

@narimiran narimiran reopened this Jan 10, 2019

@timotheecour

This comment has been minimized.

Copy link
Contributor

commented Jan 12, 2019

I can recreate the failure locally too

@kaushalmodi that's odd, my patch #10244 worked for me, and still does;

can you please run these commands and let me know what you're getting?

cat ~/.config/nim/config.nims
import os

git clone https://github.com/nim-lang/Nim
cd Nim

git rev-parse HEAD
647066e378bdbc85646dd88ea90eb15eddbbbc50

sh build_all.sh
./bin/nim --version
Nim Compiler Version 0.19.9 [MacOSX: amd64]
Compiled at 2019-01-12
Copyright (c) 2006-2018 by Andreas Rumpf

git hash: 647066e378bdbc85646dd88ea90eb15eddbbbc50
active boot switches: -d:release
@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented Jan 15, 2019

@timotheecour

Yes, I confirm that your snippet above works fine on my machine (RHEL 6.8). But it still fails on Travis. Probably there are some nuances about the 64-bit Linux OS on Travis that's causing the problem?

See the https://travis-ci.org/kaushalmodi/hello_musl/builds/478090554#L669 log of a Travis run that I just reran .. it is still failing.

Relevant bits from that log:

int: used config file '/home/travis/build/kaushalmodi/hello_musl/nim/795e5e11ef53260a0a15e25eba36dddbd83900c1/config/nim.cfg' [Conf]
Hint: used config file 'koch.nim.cfg' [Conf]
Hint: used config file '/home/travis/build/kaushalmodi/hello_musl/nim/795e5e11ef53260a0a15e25eba36dddbd83900c1/config/config.nims' [Conf]
lib/system.nim(392, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
lib/system.nim(392, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
lib/pure/os.nim(718, 14) Error: undeclared identifier: 'Stat'
/home/travis/.travis/job_stages: line 115: ./koch: No such file or directory

.. and from that same part in the Nim build locally, I see:

bin/nim c --skipUserCfg koch
Hint: used config file '/tmp/kmodi/nim_i10030/Nim/config/nim.cfg' [Conf]
Hint: used config file '/tmp/kmodi/nim_i10030/Nim/koch.nim.cfg' [Conf]
Hint: used config file '/tmp/kmodi/nim_i10030/Nim/config/config.nims' [Conf]
lib/system.nim(392, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
Hint: system [Processing]
lib/system.nim(392, 50) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
Hint: koch [Processing]
Hint: os [Processing]

So somehow, os.nim fails to compile on Travis with import os in config.nims. I have verified this with the latest Nim devel commit as of now: 795e5e1.

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented Jan 15, 2019

Here's a minimum reproducible example: https://github.com/kaushalmodi/hello_musl/tree/nim-issue-10030

and its failure on Travis: https://travis-ci.org/kaushalmodi/hello_musl/builds/480134192#L622-L628.

Discussion on gitter: https://gitter.im/nim-lang/Nim?at=5c3e5ed31cb70a372ae4de0b

(I wanted to link to IRC logs but https://irclogs.nim-lang.org/16-01-2019.html gave a 404 error at the time of typing this.)

timotheecour added a commit to timotheecour/Nim that referenced this issue Jan 15, 2019

@timotheecour timotheecour self-assigned this Jan 16, 2019

timotheecour added a commit to timotheecour/Nim that referenced this issue Jan 16, 2019

@Araq Araq closed this in #10324 Jan 16, 2019

Araq added a commit that referenced this issue Jan 16, 2019

kaushalmodi added a commit to kaushalmodi/hello_musl that referenced this issue Jan 16, 2019

Revert "Revert "Revert "Hack: Attempt to make Nim building to work by…
… moving config.nims out"""

This reverts commit 8ebba25.

nim-lang/Nim#10030 is finally fixed!

@narimiran narimiran added the Tools label Feb 7, 2019

@juancarlospaco

This comment has been minimized.

Copy link
Contributor

commented May 29, 2019

This still reproduces as of today as of exactly described above:

$ choosenim update self
   Updating choosenim
      Info: Already up to date at version 0.4.0

$ choosenim update devel
   Updating devel
Downloading Nim devel from GitHub
[##################################################] 100.0% 0kb/s
 Extracting devel.tar.gz
   Building Nim #devel
   Warning: Building from latest C sources. They may not be compatible with the Nim version you have chosen to install.
Downloading Nim C sources from GitHub
[##################################################] 100.0% 0kb/s
 Extracting master.tar.gz
   Building C sources
   Building koch
 Exception: Execution failed with exit code 1
        ... Command: ./bin/nim c koch
        ... Output: Hint: used config file '/home/juan/.choosenim/toolchains/nim-#devel/config/nim.cfg' [Conf]
        ... Hint: used config file 'koch.nim.cfg' [Conf]
        ... Hint: used config file '/home/juan/.choosenim/toolchains/nim-#devel/config/config.nims' [Conf]
        ... lib/system.nim(469, 48) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
        ... lib/system.nim(469, 48) Warning: unknown magic 'Destroy' might crash the compiler [UnknownMagic]
        ... lib/pure/os.nim(964, 14) Error: undeclared identifier: 'Stat'
   Cleaning failed build
     Error: Build failed


$ rm --verbose $XDG_CONFIG_HOME/nim/config.nims
removed '/home/juan/.config/nim/config.nims'

$ choosenim update devel
   Updating devel
Downloading Nim devel from GitHub
[##################################################] 100.0% 0kb/s
 Extracting devel.tar.gz
   Building Nim #devel
   Warning: Building from latest C sources. They may not be compatible with the Nim version you have chosen to install.
Downloading Nim C sources from GitHub
[##################################################] 100.0% 0kb/s
 Extracting master.tar.gz
   Building C sources
   Building koch
   Building Nim
   Building tools (nimble, nimgrep, nimpretty, nimsuggest)
    Updated to #devel
      Info: Version #devel already selected

$ nim --version
Nim Compiler Version 0.19.9 [Linux: amd64]
Compiled at 2019-05-29
Copyright (c) 2006-2019 by Andreas Rumpf
active boot switches: -d:release

GitHub wont let me Re-Open Issue whatsoever.

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented May 29, 2019

It builds fine for me. May be choosenim needs to get fixed to use the appropriate switches when building koch.

@juancarlospaco

This comment has been minimized.

Copy link
Contributor

commented May 29, 2019

Ok, maybe I need to reinstall the choosenim 👍

@kaushalmodi

This comment has been minimized.

Copy link
Contributor Author

commented May 29, 2019

Or I could be misremembering.. the fix was inside the build.sh. So as long as you are using that, it should be good: 0bb76dd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.