Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix-shell failure #3041

Open
paulyoung opened this issue Jan 7, 2022 · 23 comments
Open

nix-shell failure #3041

paulyoung opened this issue Jan 7, 2022 · 23 comments

Comments

@paulyoung
Copy link
Contributor

I'm encountering the following when trying to enter a nix-shell.

macOS Monterey, Version 12.0.1 (21A559)

$ nix-shell                                                                         ~/projects/dfinity/motoko/main
these 28 derivations will be built:
...
+ ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -o src/uunf_data.cmx src/uunf_data.ml
ocamlopt.opt got signal and exited
Command exited with code 2.
pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
     '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
     'README.md' 'src/uunf.a' 'src/uunf.cmxs' 'src/uunf.cmxa' 'src/uunf.cma'
     'src/uunf.cmx' 'src/uunf.cmi' 'src/uunf.mli' 'src/uunf_data.cmx'
     'src/uunf_tmapbyte.cmx' 'src/uunf_tmapbool.cmx' 'src/uunf_tmap.cmx'
     'src/uunf_fmt.cmx' 'src/uunf_string.a' 'src/uunf_string.cmxs'
     'src/uunf_string.cmxa' 'src/uunf_string.cma' 'src/uunf_string.cmx'
     'src/uunf_string.cmi' 'src/uunf_string.mli' 'test/unftrip.native'
     'doc/index.mld' 'DEVEL.md' 'test/examples.ml']: exited with 10
error: builder for '/nix/store/bz3mcx2d3k6vakgy6y80vlmrn696xbyp-ocaml4.12.0-uunf-14.0.0.drv' failed with exit code 1;
       last 10 log lines:
       > Command exited with code 2.
       > pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
       >      '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
       >      'README.md' 'src/uunf.a' 'src/uunf.cmxs' 'src/uunf.cmxa' 'src/uunf.cma'
       >      'src/uunf.cmx' 'src/uunf.cmi' 'src/uunf.mli' 'src/uunf_data.cmx'
       >      'src/uunf_tmapbyte.cmx' 'src/uunf_tmapbool.cmx' 'src/uunf_tmap.cmx'
       >      'src/uunf_fmt.cmx' 'src/uunf_string.a' 'src/uunf_string.cmxs'
       >      'src/uunf_string.cmxa' 'src/uunf_string.cma' 'src/uunf_string.cmx'
       >      'src/uunf_string.cmi' 'src/uunf_string.mli' 'test/unftrip.native'
       >      'doc/index.mld' 'DEVEL.md' 'test/examples.ml']: exited with 10
       For full logs, run 'nix log /nix/store/bz3mcx2d3k6vakgy6y80vlmrn696xbyp-ocaml4.12.0-uunf-14.0.0.drv'.
error: 1 dependencies of derivation '/nix/store/634a6ri5zdyl58xk8ryg7x20dzv3z4i8-ocaml4.12.0-uucp-14.0.0.drv' failed to build
error: build of '/nix/store/1l8f8sccnxrpslzxpr8vbcfzx7y1cjin-ocaml4.12.0-ppx_tools_versioned-5.4.0.drv', '/nix/store/2kfn5zp27n79kil34yp0q2isvigq1dvr-ocaml4.12.0-ocamlformat-0.19.0.drv', '/nix/store/48gwh4k52881xi07v2w7fxzx5pddiwlq-drun.drv', '/nix/store/58xqsnaf68v35akbfvc707h9f1mf5k84-ocaml4.12.0-merlin-4.4-412.drv', '/nix/store/634a6ri5zdyl58xk8ryg7x20dzv3z4i8-ocaml4.12.0-uucp-14.0.0.drv', '/nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv', '/nix/store/ccr88y8wkar8qfpij4i4vkq1lha7slcp-ic-ref-run.drv', '/nix/store/f5mrcazzd4ax7ih63a1gh90zfcvhwqxp-niv-0.2.19.drv', '/nix/store/h8srnrkcni5xjkyamh5w6a9f2ysh834p-qc-motoko-1.drv', '/nix/store/jv1fc3z0rki4z1d6as3pipqcwfyfgfw0-rust-1.59.0-nightly-2021-12-01-48a5999fc.drv', '/nix/store/jzfh2szjc3pbracm3ryzwd21r8zhbdk9-lsp-int-0.drv', '/nix/store/qqack1dx9sl7a55ql9cxxbmic3fy0mjx-ocaml4.12.0-ocaml-migrate-parsetree-1.8.0.drv', '/nix/store/wfjvsz5viwbmxnx2b9frr4iyrdqhcka0-wasm-profiler-instrument.drv' failed
@paulyoung
Copy link
Contributor Author

Perhaps NixOS/nixpkgs#153686 will help. I'll give it a try.

@paulyoung
Copy link
Contributor Author

That worked but then I ran into this:

  = note: Undefined symbols for architecture arm64:
            "_readdir$INODE64", referenced from:
                yanix::sys::bsd::dir::iter_impl::h2f2c32a20989b80a in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.12.rcgu.o)
            "_fstat$INODE64", referenced from:
                yanix::file::fstat::h2c7cb40e007c0839 in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.0.rcgu.o)
            "_fstatat$INODE64", referenced from:
                yanix::file::fstatat::had9f7a12369ea7f3 in libwasi_common-9bd597a51adbffdc.rlib(wasi_common-9bd597a51adbffdc.wasi_common.a51b261f-cgu.6.rcgu.o)
          ld: symbol(s) not found for architecture arm64
          clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
          

   Vendoring socket2 v0.4.0 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/socket2-0.4.0) to drun-vendor.tar.gz/socket2
   Vendoring spin v0.5.2 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/spin-0.5.2) to drun-vendor.tar.gz/spin
Lookup Uucp.Func.is_math for each uchar (10x)
   Vendoring spki v0.4.1 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/spki-0.4.1) to drun-vendor.tar.gz/spki
error: could not compile `wasmtime-cli` due to previous error
   Vendoring ssh2 v0.9.3 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/ssh2-0.9.3) to drun-vendor.tar.gz/ssh2
error: builder for '/nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv' failed with exit code 101;
       last 10 log lines:
       >                 yanix::sys::bsd::dir::iter_impl::h2f2c32a20989b80a in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.12.rcgu.o)
       >             "_fstat$INODE64", referenced from:
       >                 yanix::file::fstat::h2c7cb40e007c0839 in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.0.rcgu.o)
       >             "_fstatat$INODE64", referenced from:
       >                 yanix::file::fstatat::had9f7a12369ea7f3 in libwasi_common-9bd597a51adbffdc.rlib(wasi_common-9bd597a51adbffdc.wasi_common.a51b261f-cgu.6.rcgu.o)
       >           ld: symbol(s) not found for architecture arm64
       >           clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
       >           
       >
       > error: could not compile `wasmtime-cli` due to previous error
       For full logs, run 'nix log /nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv'.
error: build of '/nix/store/04wl6h47i7xmn4l8l195xlpnjv0a718w-ocaml4.12.0-uucp-14.0.0.drv', '/nix/store/3vli2xzc16vc7m4fpx968nhy9dvmpadn-ocaml4.12.0-ocamlformat-0.19.0.drv', '/nix/store/48gwh4k52881xi07v2w7fxzx5pddiwlq-drun.drv', '/nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv', '/nix/store/ccr88y8wkar8qfpij4i4vkq1lha7slcp-ic-ref-run.drv', '/nix/store/f5mrcazzd4ax7ih63a1gh90zfcvhwqxp-niv-0.2.19.drv', '/nix/store/h8srnrkcni5xjkyamh5w6a9f2ysh834p-qc-motoko-1.drv', '/nix/store/jv1fc3z0rki4z1d6as3pipqcwfyfgfw0-rust-1.59.0-nightly-2021-12-01-48a5999fc.drv', '/nix/store/wfjvsz5viwbmxnx2b9frr4iyrdqhcka0-wasm-profiler-instrument.drv' failed

@paulyoung
Copy link
Contributor Author

I suspect a newer version of wasmtime will work.

@ggreif, it looks like you may have been trying to do that via NixOS/nixpkgs#109571. Do you have any suggestions?

@paulyoung
Copy link
Contributor Author

That version of wasmtime does appear to help although I'm seeing some test failures for it. Trying to address those as well.

@paulyoung
Copy link
Contributor Author

It looks like I can work around those with the export HOME=$TMP; trick.

Now running into nmattia/niv#332

@paulyoung
Copy link
Contributor Author

With the following changes I was successfully able to enter a nix-shell:

diff --git a/nix/default.nix b/nix/default.nix
index 70f10a5a..187928ff 100644
--- a/nix/default.nix
+++ b/nix/default.nix
@@ -88,6 +88,46 @@ let
           };
         })
 
+        (self: super: {
+          # https://github.com/nmattia/niv/issues/332#issuecomment-958449218
+          niv = self.haskell.lib.compose.overrideCabal (drv: { enableSeparateBinOutput = false; }) super.haskellPackages.niv;
+
+          # https://github.com/NixOS/nixpkgs/pull/109571
+          wasmtime = with self; rustPlatform.buildRustPackage rec {
+            pname = "wasmtime";
+            version = "0.32.0";
+
+            src = fetchFromGitHub {
+              owner = "bytecodealliance";
+              repo = pname;
+              rev = "v${version}";
+              sha256 = "sha256-iko2G2cUIQYv7Sia8fLtb7d6XCbpOKz31ST62eE19B0";
+              fetchSubmodules = true;
+            };
+
+            cargoSha256 = "sha256-z8x004BbRWi9cRf2I27uiFuu2Jnr1Z3Ey992S5hdyNs";
+
+            nativeBuildInputs = [ python cmake clang ];
+            buildInputs = [ llvmPackages.libclang ] ++
+            lib.optionals stdenv.isDarwin [ darwin.apple_sdk.frameworks.Security ];
+            LIBCLANG_PATH = "${llvmPackages.libclang.lib}/lib";
+
+            configurePhase = ''
+              export HOME=$TMP;
+            '';
+
+            doCheck = true;
+
+            meta = with lib; {
+              description = "Standalone JIT-style runtime for WebAssembly, using Cranelift";
+              homepage = "https://github.com/bytecodealliance/wasmtime";
+              license = licenses.asl20;
+              maintainers = [ maintainers.matthewbauer ];
+              platforms = platforms.unix;
+            };
+          };
+        })
+
       ];
     };
 in
diff --git a/nix/sources.json b/nix/sources.json
index 0eec0718..31f4acbc 100644
--- a/nix/sources.json
+++ b/nix/sources.json
@@ -97,16 +97,16 @@
         "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
     },
     "nixpkgs": {
-        "branch": "release-21.11",
+        "branch": "backport-153630-to-release-21.11",
         "builtin": true,
         "description": "Nixpkgs/NixOS branches that track the Nixpkgs/NixOS channels",
         "homepage": null,
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "d1e59cfc49961e121583abe32e2f3db1550fbcff",
-        "sha256": "03ldf1dlxqf3g8qh9x5vp6vd9zvvr481fyjds111imll69y60wpm",
+        "rev": "17de6590083e80981c4459354530ca6411de485b",
+        "sha256": "05fvgqh4j01wfriqzrhfsaf3sb4knv14gmj1682f88xjfbl4qwgi",
         "type": "tarball",
-        "url": "https://github.com/NixOS/nixpkgs/archive/d1e59cfc49961e121583abe32e2f3db1550fbcff.tar.gz",
+        "url": "https://github.com/NixOS/nixpkgs/archive/17de6590083e80981c4459354530ca6411de485b.tar.gz",
         "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
     },
     "nixpkgs-mozilla": {

@ggreif
Copy link
Contributor

ggreif commented Jan 7, 2022

@paulyoung Great that you can use nix-shell on M1! And also thanks for the detective work you have invested!
Indeed the Wasmtime package needs some love. It has several problems:

  • utimensat is needed by the build, but not available by Darwin < 10.13
  • your mentioned test failure (I'll integrate your workaround)
  • test wants to access the internet (do you have pure = false in your config?)

So don't hold your breath just yet :-)

ggreif added a commit to ggreif/nixpkgs that referenced this issue Jan 7, 2022
this is supposed to improve testing

frobbed from dfinity/motoko#3041
@paulyoung
Copy link
Contributor Author

For what it’s worth, I learned the HOME=$TMP trick a while ago from here to work around the issue reported there: NixOS/nixpkgs#16144 (comment)

I don’t think I have pure = false in my config. It’s possible I ran nix-shell --impure but I don’t remember doing that.

@paulyoung
Copy link
Contributor Author

I've come back to this after a while and now I'm somehow seeing a different wasmtime error:

error: could not compile `wasmtime-cli` due to previous error
error: builder for '/nix/store/3qkdh55ydxi4d97gq6qgrrc4s92hwj28-wasmtime-0.32.0.drv' failed with exit code 101;
       last 10 log lines:
       >   = note: Undefined symbols for architecture x86_64:
       >             "_futimens", referenced from:
       >                 rustix::imp::libc::fs::syscalls::futimens::h40adb1a14bbd1f4a in librustix-6124166338298bee.rlib(rustix-6124166338298bee.rustix.56f6ede2-cgu.5.rcgu.o)
       >             "_utimensat", referenced from:
       >                 rustix::imp::libc::fs::syscalls::utimensat::h67ad5e8dfbb603b0 in librustix-6124166338298bee.rlib(rustix-6124166338298bee.rustix.56f6ede2-cgu.5.rcgu.o)
       >           ld: symbol(s) not found for architecture x86_64
       >           clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
       >
       >
       > error: could not compile `wasmtime-cli` due to previous error
       For full logs, run 'nix log /nix/store/3qkdh55ydxi4d97gq6qgrrc4s92hwj28-wasmtime-0.32.0.drv'.
error: build of '/nix/store/3qkdh55ydxi4d97gq6qgrrc4s92hwj28-wasmtime-0.32.0.drv', '/nix/store/4phzhm3klqygs0zgvdm8jb4z4abk17z7-ic-ref-run.drv', '/nix/store/9n49s3szs960fn7mrgfl7zzgw2zmlb4f-drun.drv', '/nix/store/kb759lqwh0h8rbjfwczb9xciszmxl2gg-niv-0.2.19.drv', '/nix/store/pxbdp5gz8hjjdyzf6mywxpc5mj074rph-rust-1.59.0-nightly-2021-12-01-48a5999fc.drv' failed

@domwoe
Copy link
Member

domwoe commented Apr 13, 2022

I'm getting the same error trying to install moc with nix-env -i -f . -A moc on a M1.

@sunfishcode
Copy link

With bytecodealliance/rustix#275, rustix now works on macOS versions that lack utimensat and futimens. bytecodealliance/wasmtime#4022 is a PR to update Wasmtime.

@paulyoung
Copy link
Contributor Author

Related: NixOS/nixpkgs#166965

@ninegua
Copy link
Member

ninegua commented May 13, 2022

I was able to build moc binaries on aarch64 with a simpler nix file. It has been tested with nix 2.8.0 and unstable channel on a 2021 m1 mbp running OS X 12.3.1.

ninegua@ba72f95

In particular, I think the nix configuration should let users choose their own nixpkgs to avoid building "the world".

@nomeata
Copy link
Collaborator

nomeata commented May 13, 2022

In particular, I think the nix configuration should let users choose their own nixpkgs to avoid building "the world".

As an option, certainly. But by default we do want to pin nixpkgs, right?

@ninegua
Copy link
Member

ninegua commented May 13, 2022

But by default we do want to pin nixpkg

Yes of course

@ninegua
Copy link
Member

ninegua commented May 13, 2022

Let me elaborate a bit. On CI I think it is always good to pin a nixpkg version, do static build, cross compiling, etc etc. On a dev's machine, having the exact setup as CI should be optional, because it is rarely needed, especially when native platform differs.

I spent some time yesterday on this because I wanted to have a fully working nix-shell of https://github.com/dfinity/ic on my new work laptop (m1 MBP). It was a lot simpler to use rustup for that purpose, but for testing, I need to pull in dependencies which include the motoko repo. So I spent some time to get nix-build working on m1. I might spend a bit more time to clean up a few things and upstream my patches (and also to get dfx compiling too) but no guarantees...

@nomeata
Copy link
Collaborator

nomeata commented May 13, 2022

On a dev's machine, having the exact setup as CI should be optional, because it is rarely needed, especially when native platform differs.

I’d say, in that case CI should also run that particular native platform :-)

Especially with things like using a specific nightly or even patching this I think the default should be that nix-shell uses exactly what’s on CI, and only if you know what you are doing (and why) you should be given a way to override that choice. At least so far the Motoko team seemed reasonably happy with that patronizing approach :-)

@ninegua
Copy link
Member

ninegua commented May 13, 2022

Normally I'd agree with you, but buildRustPackage is such a pain... unless somebody solves the incremental build problem for using rust in nix, I'd prefer a very impure dev environment like rustup

@nomeata
Copy link
Collaborator

nomeata commented May 13, 2022

Note that we do have a separate (make- and/or cargo based) incremental build setup where nix only provides the tools. We are not using nix-build for incremental work.

@paulyoung
Copy link
Contributor Author

@ninegua I tried your branch and ran into this:

building '/nix/store/jggfr9py5nscx5y9843y80kzp0krmys5-git-ls-files.drv'...
error: Package ‘libcxx-13.0.0’ in /nix/store/jsciqap989svc3a0bf2gjnsh38k4shw9-nixpkgs-22.05pre335126.391f93a83c3/nixpkgs/pkgs/development/compilers/llvm/13/libcxx/default.nix:42 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)

I tried again with NIXPKGS_ALLOW_BROKEN=1 but got what appears to be a related failure.

@paulyoung
Copy link
Contributor Author

Which version of nixpkgs are you using?

@ninegua
Copy link
Member

ninegua commented May 16, 2022

@paulyoung For m1 OS X, you have to switch to use unstable channel, which has wasmtime 0.36.0. For m1 Linux, you need to stay with the 21.11 channel, which has wasmtime 0.21.0. It is unfortunate that 0.21.0 panics in wasmtime testing on m1 OS X, while 0.36.0 panics in testing on m1 Linux.

Edit: If your failure if related to NIXPKGS_ALLOW_BROKEN, then you should also add --impure to nix-build. For nix-shell it should not be necessary.

Also, wasmtime is not needed for compilation, so if it gets in the way, you can comment it out from rts/Makefile.

@ninegua
Copy link
Member

ninegua commented May 16, 2022

I was able to build all binaries on aarch64 OS X and Linux. You can try it out https://github.com/ninegua/ic-nix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants