Upgrade to MozillaBuild 4.0 #326

Redfire75369 · 2022-11-25T13:39:23Z

Added Support for MozillaBuild 4.0 to build SpiderMonkey on Windows

Redfire75369 · 2022-11-25T13:40:35Z

The makefile seems to require being called from the mozilla-build terminal though (start-shell.bat), so I'm not sure how to fix this error.

   Compiling mozjs_sys v0.68.2 (C:\Users\Redfire\software\github\mozjs\mozjs)
error: failed to run custom build command for `mozjs_sys v0.68.2 (C:\Users\Redfire\software\github\mozjs\mozjs)`

Caused by:
  process didn't exit successfully: `C:\Users\Redfire\software\github\mozjs\target\debug\build\mozjs_sys-7183b162fa1fdf26\build-script-build` (exit code: 101)
  --- stdout
  cargo:outdir=C:\Users\Redfire\software\github\mozjs\target\debug\build\mozjs_sys-004ea0ad6a849248\out\build
  [[ 'C:\Users\Redfire\software\github\mozjs\mozjs\mozjs'/js/src/configure -ot 'C:\Users\Redfire\software\github\mozjs\mozjs\mozjs'/js/src/configure.in ]] && touch 'C:\Users\Redfire\software\github\mozjs\mozjs\mozjs'/js/src/configure || true

  --- stderr
  "C:\\mozilla-build\\bin\\mozmake\\mozmake.exe"
  '[[' is not recognized as an internal or external command,
  operable program or batch file.
  'true' is not recognized as an internal or external command,
  operable program or batch file.
  mozmake: *** [C:\Users\Redfire\software\github\mozjs\mozjs\makefile.cargo:191: maybe-configure] Error 1
  thread 'main' panicked at 'assertion failed: result.success()', mozjs\build.rs:232:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Redfire75369 · 2022-11-25T13:54:21Z

That's weird, it's a different error in CI than what I'm getting locally.

Redfire75369 · 2022-11-29T13:40:46Z

It seems to work if you start the mozbuild shell before running cargo build.
That CI isn't... correct....

Added mozmake to PATH

Redfire75369 · 2022-11-29T15:32:21Z

  Reticulating splines...
  Finished reading 62 moz.build files in 0.14s
  Read 0 gyp files in parallel contributing 0.00s to total wall time
  Processed into 249 build config descriptors in 0.18s
  RecursiveMake backend executed in 0.28s
    173 total backend files; 173 created; 0 updated; 0 unchanged; 0 deleted; 6 -> 39 Makefile
  Total wall time: 0.68s; CPU time: 0.58s; Efficiency: 86%; Untracked: 0.07s
  config/autoconf.mk:117: *** missing separator.  Stop.
  mozmake: *** [D:\a\mozjs\mozjs\mozjs\makefile.cargo:184: all] Error 2
  thread 'main' panicked at 'assertion failed: result.success()', mozjs\build.rs:225:5

Something's wrong with autoconf/make somewhere and I have no idea where.

Redfire75369 · 2022-11-29T17:05:48Z

It seems that this is being generated in autoconf.mk, with two separate lines, which is what breaks it.

PATH = C:\mozilla-build\msys2\usr\bin;C:\mozilla-build\bin;D:\a\mozjs\mozjs\target\debug\deps;D:\a\mozjs\mozjs\target\debug;C:\Users\runneradmin\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib;C:\Users\runneradmin\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin;C:\Program
Files\Git\usr\bin;C:\mozilla-build\bin;C:\mozilla-build\kdiff3;C:\mozilla-build\python3;C:\mozilla-build\python3\Scripts;C:\mozilla-build\msys2\usr\local\bin;C:\mozilla-build\msys2\usr\bin;C:\mozilla-build\msys2\usr\bin;C:\mozilla-build\msys2\opt\bin;C:\ProgramData\scoop\shims;C:\Users\runneradmin\scoop\shims;C:\Program Files\MongoDB\Server\5.0\bin;C:\aliyun-cli;C:\vcpkg;C:\Program Files (x86)\NSIS;C:\tools\zstd;C:\Program Files\Mercurial;C:\hostedtoolcache\windows\stack\2.9.1\x64;C:\cabal\bin;C:\ghcup\bin;C:\tools\ghc-9.4.2\bin;C:\Program Files\dotnet;C:\mysql\bin;C:\Program Files\R\R-4.2.2\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\Program Files (x86)\sbt\bin;C:\Program Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)\pipx_bin;C:\npm\prefix;C:\hostedtoolcache\windows\go\1.17.13\x64\bin;C:\hostedtoolcache\windows\Python\3.9.13\x64\Scripts;C:\hostedtoolcache\windows\Python\3.9.13\x64;C:\hostedtoolcache\windows\Ruby\3.0.4\x64\bin;C:\tools\kotlinc\bin;C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.352-8\x64\bin;C:\Program Files\ImageMagick-7.1.0-Q16-HDRI;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\ProgramData\kind;C:\Program Files\Microsoft\jdk-11.0.12.7-hotspot\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\ProgramData\Chocolatey\bin;C:\Program Files\PowerShell\7;C:\Program Files\Microsoft\Web Platform Installer;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn;C:\Program Files\Microsoft SQL Server\150\Tools\Binn;C:\Program Files\OpenSSL\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\Program Files\TortoiseSVN\bin;C:\Program Files\CMake\bin;C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.6\bin;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\nodejs;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files\GitHub CLI;C:\tools\php;C:\Program Files (x86)\sbt\bin;C:\SeleniumWebDrivers\ChromeDriver;C:\SeleniumWebDrivers\EdgeDriver;C:\Program Files\Amazon\AWSCLIV2;C:\Program Files\Amazon\SessionManagerPlugin\bin;C:\Program Files\Amazon\AWSSAMCLI\bin;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files\LLVM\bin;C:\Users\runneradmin\.dotnet\tools;C:\Users\runneradmin\.cargo\bin;C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps;C:\mozilla-build\msys2\usr\bin\site_perl;C:\mozilla-build\msys2\usr\bin\vendor_perl;C:\mozilla-build\msys2\usr\bin\core_perl;C:\mozilla-build\bin\mozmake;C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64

Redfire75369 · 2022-11-30T02:02:24Z

In the end, it was caused by the most mundane thing.
This line in start-shell.bat found an ssh binary in C:\Program Files\Git\usr\bin.

FOR /F "tokens=* USEBACKQ" %%F IN (`where ssh 2^>NUL`) DO (
    SET EXTERNAL_TO_MOZILLABUILD_SSH_DIR=%%~dpF
)

Then in https://searchfox.org/mozilla-build/source/msys-config/profile-mozilla.sh#70, cygpath would return a multiline string.

$ cygpath -u C:\\Program Files\\Git\\usr\\bin\\
/c/Program
Files/Git/usr/bin/

Redfire75369 · 2022-11-30T02:57:38Z

This requires a fix on the mozilla-build side so I've filed a bug. (https://bugzilla.mozilla.org/show_bug.cgi?id=1803269)

Redfire75369 · 2022-12-02T12:20:57Z

Update: The bug has now been addressed and a fix has been merged. Once a new update is released, this can probably be merged.

My only worry is the fact that I'm using latest to download mozmake from taskcluster, which might not be the smartest idea.

jdm · 2022-12-02T12:41:46Z

Great work on finding that issue and getting it fixed!

Redfire75369 · 2022-12-02T13:44:49Z

Thanks! Requiring the build to be run in the MozillaBuild shell isn't ideal, but this works for now at least.
This is important though, since building with MozillaBuild 3.0 isn't explicitly supported. Who knows how long it'll last for.

Do you have any suggestions regarding the Taskcluster issue?

Redfire75369 · 2022-12-02T13:49:31Z

I guess we could pin it to the latest one at the time of the commit, which is d1109e59eddb6045f048bd31110ea80da4679f5a6637b44165e9c2523b58d28d

jdm · 2022-12-02T14:33:22Z

Hmm, help me understand what's required here: is mozmake not part of mozbuild? Why do we need to install it separately?

Redfire75369 · 2022-12-02T15:34:48Z

As of MozillaBuild 4.0, mozmake isn't automatically installed. It's bootstrapped by bootstrap.py.

Automatically-bootstrapped tools have been removed (NSIS, mozmake, yasm).

jdm · 2022-12-02T15:35:49Z

Will it build successfully without mozmake?

Redfire75369 · 2022-12-02T16:28:11Z

Given we use a makefile.cargo, probably not 🙃.

In all seriousness, it doesn't work without some form of make, either make, gmake or mozmake.

jdm · 2022-12-02T16:48:09Z

Oh, if there's no make included in MozillaBuild at all then that makes sense. I didn't think that was the case.

Redfire75369 · 2022-12-03T15:29:37Z

After asking the devs a bit, it seems the next release will be in around a month or two, so this will remain a draft until then.
I'll probably spend the time so it creates a script with a single command and automatically starts the MozillaBuild shell with running that script.

Redfire75369 · 2022-12-04T19:20:50Z

I'm not sure about where this UWP error is coming from, anyone have any ideas?

error: failed to add native library D:\a\mozjs\mozjs\target\aarch64-uwp-windows-msvc\debug\build\mozjs_sys-f75d1830bda12164\out\build/js/src/build\js_static.lib: Unsupported archive identifier

Redfire75369 · 2022-12-05T05:38:38Z

Seems to be an unrelated issue. https://github.com/Redfire75369/mozjs/actions/runs/3617051242/jobs/6095562856

jdm · 2022-12-05T13:19:38Z

I've opened #327 about that.

Redfire75369 · 2023-01-06T04:01:37Z

Should the tar and ruzstd build dependencies only be required on windows? I think that's probably a smart idea.
mozmake and MozillaBuild should only be needed on windows.

jdm · 2023-01-06T04:08:20Z

Yep, that sounds reasonable.

Redfire75369 · 2023-01-06T04:55:01Z

If you don't mind, I'd like to change find_make too since it actually doesn't work currently. It needs to run make --version. It'll also let me prevent installing mozmake if you already have make, gmake or mozmake installed.

(It doesn't work because it returns an exit code of 2 if there's no Makefile)

Changed mozmake Installation to occur only on Windows Changed MozillaBuild detection and integration to occur only on Windows

Added Checking for make and gmake on All Platforms Added Checking for mozmake on Windows

bors-servo · 2023-02-05T03:22:12Z

☔ The latest upstream changes (presumably #342) made this pull request unmergeable. Please resolve the merge conflicts.

Switched to cygpath on MSVC Reformatted Makefile

README.md

Fixed Wording in ReadME

bors-servo · 2023-03-25T00:07:05Z

☔ The latest upstream changes (presumably #350) made this pull request unmergeable. Please resolve the merge conflicts.

sagudev · 2023-09-11T08:52:55Z

This was resolved with #392.

Added Automatic mozmake Installation

72abaf3

Updated Mozilla Build in CI

fdef3fb

Updated Build Environment in CI

cf99255

Redfire75369 added 4 commits November 29, 2022 22:12

Started Mozilla Build Shell in CI

9632e9c

Added mozmake to PATH

Fixed Start Shell Script Path

e864bd7

Moved Start Shell to Shell Parameter on CI Step

7b33932

Removed AUTOCONF Environment Variable

d486e06

Redfire75369 added 2 commits December 5, 2022 02:56

Added Temporary Fix for MozillaBuild in Windows CI

396475e

Automatically Start MozillaBuild Shell

418e8e4

Fixed Windows CI Environment

5caf1fd

Redfire75369 force-pushed the mozbuild-4.0 branch from 59813fb to 5caf1fd Compare December 4, 2022 19:32

Removed Build Dependency on ruzstd and tar on Non-Windows Platforms

0e67a76

Changed mozmake Installation to occur only on Windows Changed MozillaBuild detection and integration to occur only on Windows

Redfire75369 force-pushed the mozbuild-4.0 branch from b7a42ab to 6ec783a Compare January 6, 2023 11:15

Fixed find_make

43708c4

Added Checking for make and gmake on All Platforms Added Checking for mozmake on Windows

Redfire75369 force-pushed the mozbuild-4.0 branch 2 times, most recently from 8b72081 to 43708c4 Compare January 6, 2023 13:06

Redfire75369 mentioned this pull request Jan 25, 2023

Upgrade to SM 109 #340

Closed

Redfire75369 added 4 commits February 20, 2023 13:44

Resolved Merge Conflicts

40649ac

Merge branch 'master' into mozbuild-4.0

fd683bf

Removed Python 2 Support in Makefile

42a7dfa

Switched to cygpath on MSVC Reformatted Makefile

Updated Windows Build Instructions

81ed487

sagudev reviewed Mar 7, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

Added Initialisation for MOZILLABUILD Environment Variable

d4a691f

Fixed Wording in ReadME

sagudev mentioned this pull request May 23, 2023

Error building mozjs_sys servo/servo#29142

Closed

mrobinson mentioned this pull request Jun 2, 2023

Upgrade to MozBuild 4.0 #356

Closed

sagudev mentioned this pull request Aug 17, 2023

Use moztools instead of mozilla-build #368

Merged

sagudev closed this Sep 11, 2023

Redfire75369 deleted the mozbuild-4.0 branch December 8, 2023 12:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to MozillaBuild 4.0 #326

Upgrade to MozillaBuild 4.0 #326

Redfire75369 commented Nov 25, 2022 •

edited

Redfire75369 commented Nov 25, 2022

Redfire75369 commented Nov 25, 2022

Redfire75369 commented Nov 29, 2022 •

edited

Redfire75369 commented Nov 29, 2022 •

edited

Redfire75369 commented Nov 29, 2022

Redfire75369 commented Nov 30, 2022

Redfire75369 commented Nov 30, 2022

Redfire75369 commented Dec 2, 2022 •

edited

jdm commented Dec 2, 2022

Redfire75369 commented Dec 2, 2022 •

edited

Redfire75369 commented Dec 2, 2022

jdm commented Dec 2, 2022

Redfire75369 commented Dec 2, 2022

jdm commented Dec 2, 2022

Redfire75369 commented Dec 2, 2022

jdm commented Dec 2, 2022

Redfire75369 commented Dec 3, 2022 •

edited

Redfire75369 commented Dec 4, 2022 •

edited

Redfire75369 commented Dec 5, 2022

jdm commented Dec 5, 2022

Redfire75369 commented Jan 6, 2023 •

edited

jdm commented Jan 6, 2023

Redfire75369 commented Jan 6, 2023 •

edited

bors-servo commented Feb 5, 2023

bors-servo commented Mar 25, 2023

sagudev commented Sep 11, 2023

Upgrade to MozillaBuild 4.0 #326

Upgrade to MozillaBuild 4.0 #326

Conversation

Redfire75369 commented Nov 25, 2022 • edited

Redfire75369 commented Nov 25, 2022

Redfire75369 commented Nov 25, 2022

Redfire75369 commented Nov 29, 2022 • edited

Redfire75369 commented Nov 29, 2022 • edited

Redfire75369 commented Nov 29, 2022

Redfire75369 commented Nov 30, 2022

Redfire75369 commented Nov 30, 2022

Redfire75369 commented Dec 2, 2022 • edited

jdm commented Dec 2, 2022

Redfire75369 commented Dec 2, 2022 • edited

Redfire75369 commented Dec 2, 2022

jdm commented Dec 2, 2022

Redfire75369 commented Dec 2, 2022

jdm commented Dec 2, 2022

Redfire75369 commented Dec 2, 2022

jdm commented Dec 2, 2022

Redfire75369 commented Dec 3, 2022 • edited

Redfire75369 commented Dec 4, 2022 • edited

Redfire75369 commented Dec 5, 2022

jdm commented Dec 5, 2022

Redfire75369 commented Jan 6, 2023 • edited

jdm commented Jan 6, 2023

Redfire75369 commented Jan 6, 2023 • edited

bors-servo commented Feb 5, 2023

bors-servo commented Mar 25, 2023

sagudev commented Sep 11, 2023

Redfire75369 commented Nov 25, 2022 •

edited

Redfire75369 commented Nov 29, 2022 •

edited

Redfire75369 commented Nov 29, 2022 •

edited

Redfire75369 commented Dec 2, 2022 •

edited

Redfire75369 commented Dec 2, 2022 •

edited

Redfire75369 commented Dec 3, 2022 •

edited

Redfire75369 commented Dec 4, 2022 •

edited

Redfire75369 commented Jan 6, 2023 •

edited

Redfire75369 commented Jan 6, 2023 •

edited