Bump elmPackages by domenkozar · Pull Request #344866 · NixOS/nixpkgs

domenkozar · 2024-09-27T12:20:02Z

Note that elm-verify-examples is still broken.

Fixes #320127 #267454 #304198

PedroHLC · 2024-09-27T12:36:32Z

NOTE: I'll try running it again! EDIT: Same result.

Result of nixpkgs-review pr 344866 run on x86_64-linux 1

2 packages failed to build:

elmPackages.elm-pages
elmPackages.elm-verify-examples

12 packages built:

elmPackages.create-elm-app
elmPackages.elm-analyse
elmPackages.elm-coverage
elmPackages.elm-doc-preview
elmPackages.elm-git-install
elmPackages.elm-graphql
elmPackages.elm-land
elmPackages.elm-language-server
elmPackages.elm-live
elmPackages.elm-review
elmPackages.elm-spa
elmPackages.elm-upgrade

@domenkozar

authored mostly by @domenkozar

domenkozar · 2024-09-27T13:24:59Z

I pushed a fix for elm-pages.

PedroHLC · 2024-09-27T13:37:42Z

Result of nixpkgs-review pr 344866 run on x86_64-linux 1

1 package failed to build:

elmPackages.elm-verify-examples

13 packages built:

elmPackages.create-elm-app
elmPackages.elm-analyse
elmPackages.elm-coverage
elmPackages.elm-doc-preview
elmPackages.elm-git-install
elmPackages.elm-graphql
elmPackages.elm-land
elmPackages.elm-language-server
elmPackages.elm-live
elmPackages.elm-pages
elmPackages.elm-review
elmPackages.elm-spa
elmPackages.elm-upgrade

PedroHLC

Diff LGTM

PedroHLC · 2024-09-27T13:38:49Z

Actually, elm-pages doesn't run after build:

╰─λ ./results/elmPackages.elm-pages/bin/elm-pages --help
/nix/store/11kh8s407rr06vqb47adra26wmgcgjlh-elm-pages-3.0.16/lib/node_modules/elm-pages/node_modules/rollup/dist/native.js:59
                throw new Error(
                      ^

Error: Cannot find module @rollup/rollup-linux-x64-gnu. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.
    at requireWithFriendlyError (/nix/store/11kh8s407rr06vqb47adra26wmgcgjlh-elm-pages-3.0.16/lib/node_modules/elm-pages/node_modules/rollup/dist/native.js:59:9)
    at Object.<anonymous> (/nix/store/11kh8s407rr06vqb47adra26wmgcgjlh-elm-pages-3.0.16/lib/node_modules/elm-pages/node_modules/rollup/dist/native.js:68:76)
    ... 3 lines matching cause stack trace ...
    at Module._load (node:internal/modules/cjs/loader:1104:12)
    at cjsLoader (node:internal/modules/esm/translators:346:17)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:286:7)
    at ModuleJob.run (node:internal/modules/esm/module_job:234:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:473:24) {
  [cause]: Error: Cannot find module '@rollup/rollup-linux-x64-gnu'
  Require stack:
  - /nix/store/11kh8s407rr06vqb47adra26wmgcgjlh-elm-pages-3.0.16/lib/node_modules/elm-pages/node_modules/rollup/dist/native.js
      at Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
      at Module._load (node:internal/modules/cjs/loader:1051:27)
      at Module.require (node:internal/modules/cjs/loader:1311:19)
      at require (node:internal/modules/helpers:179:18)
      at requireWithFriendlyError (/nix/store/11kh8s407rr06vqb47adra26wmgcgjlh-elm-pages-3.0.16/lib/node_modules/elm-pages/node_modules/rollup/dist/native.js:41:10)
      at Object.<anonymous> (/nix/store/11kh8s407rr06vqb47adra26wmgcgjlh-elm-pages-3.0.16/lib/node_modules/elm-pages/node_modules/rollup/dist/native.js:68:76)
      at Module._compile (node:internal/modules/cjs/loader:1469:14)
      at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
      at Module.load (node:internal/modules/cjs/loader:1288:32)
      at Module._load (node:internal/modules/cjs/loader:1104:12) {
    code: 'MODULE_NOT_FOUND',
    requireStack: [
      '/nix/store/11kh8s407rr06vqb47adra26wmgcgjlh-elm-pages-3.0.16/lib/node_modules/elm-pages/node_modules/rollup/dist/native.js'
    ]
  }
}

domenkozar · 2024-09-27T16:20:33Z

Seems to be #294183

SuperSandro2000 · 2024-09-27T22:51:53Z

pkgs/development/compilers/elm/packages/node/elm-pages/default.nix

    nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ makeWrapper old.nodejs.pkgs.node-gyp-build ];

+    preRebuild = ''
+      sed -i 's/"esbuild": "0\.19\.12"/"esbuild": "0.21.5"/' package.json


This should use substituteInPlace with --replace-fail

SuperSandro2000 · 2024-09-27T22:52:10Z

pkgs/development/compilers/elm/packages/node/generate-node-packages.sh

    -c node-composition.nix \
    --no-copy-node-env -e ../../../../node-packages/node-env.nix
+# well, elm-pages requires two different version of esbuild so we twist it's wrist to only use one
+sed -i 's/sources."esbuild-0.19.12"/sources."esbuild-0.21.5"/' node-packages.nix


Not sure how to do this, since this is a shell script and it doesn't have substituteInPlace.

SuperSandro2000 · 2024-09-27T22:52:51Z

pkgs/development/compilers/elm/packages/node/node-composition.nix

    inherit (pkgs) stdenv lib python2 runCommand writeTextFile writeShellScript;
    inherit pkgs nodejs;
-    libtool = if pkgs.stdenv.hostPlatform.isDarwin then pkgs.cctools or pkgs.darwin.cctools else null;
+    libtool = if pkgs.stdenv.isDarwin then pkgs.cctools or pkgs.darwin.cctools else null;


Why was this changed back?

Maybe because we used my mac to run the generation script. Could probably be reverted?

SuperSandro2000

elm-packages should also eventually stop using node2nix but that's something for another PR

turboMaCk · 2024-09-28T12:32:04Z

pkgs/development/compilers/elm/packages/node/default.nix

+  ESBUILD_BINARY_PATH = lib.getExe (
+      pkgs.esbuild.override {
+        buildGoModule = args: pkgs.buildGoModule (args // rec {
+          version = "0.20.2";
+          src = pkgs.fetchFromGitHub {
+            owner = "evanw";
+            repo = "esbuild";
+            rev = "v${version}";
+            hash = "sha256-h/Vqwax4B4nehRP9TaYbdixAZdb1hx373dNxNHvDrtY=";
+          };
+          vendorHash = "sha256-+BfxCyg0KkDQpHt/wycy/8CTG6YBA/VJvJFhhzUnSiQ=";
+        });
+      }
+    );


I wonder - did you try to just use esbuild that is already available in nixpkgs? I think this might not be that version sensitive.

It seemed very version sensitive when we tried. Had to provide the exact version.

uh that will be a bit painful to maintain. Maybe it would make sense to port these away from node2nix to buildNpmPackage right away to prevent potential regression with future bulk updates.

Agree on it being painful. A slight condolence is the fact that Elm packages are usually quite slow to release new updates.

I tried porting to buildNpmPackage but got lost. I'd need some hand-holding to get it done.

sure not a blocker. Thanks for giving it a try.

Tried again today during NixCon Hack Day: #351767

turboMaCk · 2024-09-28T12:34:11Z

pkgs/development/compilers/elm/packages/node/elm-pages/default.nix

+  ESBUILD_BINARY_PATH = lib.getExe (
+      pkgs.esbuild.override {
+        buildGoModule = args: pkgs.buildGoModule (args // rec {
+          version = "0.21.5";


so I see we even need 2 different version of esbuild. I guess there was an issue with version of ES build when you tried to share the same one?

turboMaCk

LGTM

domenkozar · 2024-10-02T08:40:28Z

Note that the binary still fails to run because we need to also patch up rollup:

❯ /nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/bin/elm-land 
/nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/lib/node_modules/elm-land/node_modules/rollup/dist/native.js:59
		throw new Error(
		      ^

Error: Cannot find module @rollup/rollup-linux-x64-gnu. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.
    at requireWithFriendlyError (/nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/lib/node_modules/elm-land/node_modules/rollup/dist/native.js:59:9)
    at Object.<anonymous> (/nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/lib/node_modules/elm-land/node_modules/rollup/dist/native.js:68:76)
    ... 3 lines matching cause stack trace ...
    at Module._load (node:internal/modules/cjs/loader:1104:12)
    at cjsLoader (node:internal/modules/esm/translators:346:17)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:286:7)
    at ModuleJob.run (node:internal/modules/esm/module_job:234:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:473:24) {
  [cause]: Error: Cannot find module '@rollup/rollup-linux-x64-gnu'
  Require stack:
  - /nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/lib/node_modules/elm-land/node_modules/rollup/dist/native.js
      at Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
      at Module._load (node:internal/modules/cjs/loader:1051:27)
      at Module.require (node:internal/modules/cjs/loader:1311:19)
      at require (node:internal/modules/helpers:179:18)
      at requireWithFriendlyError (/nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/lib/node_modules/elm-land/node_modules/rollup/dist/native.js:41:10)
      at Object.<anonymous> (/nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/lib/node_modules/elm-land/node_modules/rollup/dist/native.js:68:76)
      at Module._compile (node:internal/modules/cjs/loader:1469:14)
      at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
      at Module.load (node:internal/modules/cjs/loader:1288:32)
      at Module._load (node:internal/modules/cjs/loader:1104:12) {
    code: 'MODULE_NOT_FOUND',
    requireStack: [
      '/nix/store/mbbvs7bg3n90gj0fva1zxx5fp70fc9lq-elm-land-0.20.1/lib/node_modules/elm-land/node_modules/rollup/dist/native.js'
    ]
  }
}

Node.js v20.17.0

turboMaCk · 2024-10-02T13:35:50Z

ah right. I also ran into this issue in my quick spike - that's where I gave up. Forgot to test it in this PR.

Anyway lets try fix these issues one by one now when this is merged. I also think we can start decoupling these problematic packages away from node2nix to buildNpmPackage.

domenkozar · 2024-11-19T18:05:56Z

I tried using buildNpmPackage, but build from source is quite complicated as it needs to build parts using elm. Going to try to patch @rollup/wasm-node to be used in place of rollup.

Current elm-land in master is actually broken. It does build, but it doesn't run. This commit rewrites it from node2nix to buildNpmPackage and adds a test to confirm it runs. Previus work: * NixOS#344866 * NixOS#304198 * NixOS#267725 Refs Thaigersprint/thaigersprint-2025#1

zupo · 2025-02-15T15:34:30Z

I have gotten a bit further with elm-land, it now builds and runs on x86-linux and aarch64-darwin: #382339

domenkozar mentioned this pull request Sep 27, 2024

elmPackages.elm-i18next-gen: init at 1.1.0 #320127

Closed

13 tasks

domenkozar force-pushed the bump-elm-packages branch from 0358bb2 to 1df6123 Compare September 27, 2024 12:20

ofborg bot requested review from PedroHLC, ilyakooo0, jali-clarke, turboMaCk and zupo September 27, 2024 13:07

ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Sep 27, 2024

fix elm-pages and elm-land builds

47c0dbd

authored mostly by @domenkozar

domenkozar force-pushed the bump-elm-packages branch from 1df6123 to 47c0dbd Compare September 27, 2024 13:24

PedroHLC approved these changes Sep 27, 2024

View reviewed changes

ofborg bot requested a review from PedroHLC September 27, 2024 14:28

SuperSandro2000 reviewed Sep 27, 2024

View reviewed changes

turboMaCk reviewed Sep 28, 2024

View reviewed changes

wegank added 12.approvals: 1 This PR was reviewed and approved by one person. 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages. labels Sep 29, 2024

turboMaCk approved these changes Oct 1, 2024

View reviewed changes

turboMaCk merged commit 5080634 into NixOS:master Oct 1, 2024

zupo mentioned this pull request Feb 15, 2025

elm-land: fix elm-land build #382339

Closed

13 tasks

royneary mentioned this pull request Mar 16, 2025

elmPackages.elm-open-api: init at 0.7.0 #390504

Open

13 tasks

Uh oh!

Conversation

domenkozar commented Sep 27, 2024

Uh oh!

PedroHLC commented Sep 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

domenkozar commented Sep 27, 2024

Uh oh!

PedroHLC commented Sep 27, 2024

Uh oh!

PedroHLC left a comment

Choose a reason for hiding this comment

Uh oh!

PedroHLC commented Sep 27, 2024

Uh oh!

domenkozar commented Sep 27, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

turboMaCk left a comment

Choose a reason for hiding this comment

Uh oh!

domenkozar commented Oct 2, 2024

Uh oh!

turboMaCk commented Oct 2, 2024

Uh oh!

domenkozar commented Nov 19, 2024

Uh oh!

zupo commented Feb 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

PedroHLC commented Sep 27, 2024 •

edited

Loading