Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NWJS on Apple Silicon #7620

Closed
gpetrov opened this issue Nov 14, 2020 · 120 comments
Closed

NWJS on Apple Silicon #7620

gpetrov opened this issue Nov 14, 2020 · 120 comments

Comments

@gpetrov
Copy link

gpetrov commented Nov 14, 2020

As the new Apple Silicon M1 based Macs are shipping now, we would really like to have an NWJS version that runs natively on it.

Seems Electron already have a full working version for some time now:

https://www.electronjs.org/blog/apple-silicon

@rogerwang could you please make also an Apple Silicon binary available for the Mac?

@gpetrov
Copy link
Author

gpetrov commented Nov 14, 2020

Or maybe offer a "fat" build running on both intel and apple silicon, just as google is doing:
https://bugs.chromium.org/p/chromium/issues/detail?id=1142017

node-gyp might also need some improvements to be able to build fat native modules

@gpetrov
Copy link
Author

gpetrov commented Nov 14, 2020

Here are also the build instructions for Chrome on Apple Silicon @rogerwang so it should be easy to incorporate:

https://chromium.googlesource.com/chromium/src.git/+/master/docs/mac_arm64.md

@rogerwang
Copy link
Member

Thanks for the information. Will support it soon.

@gpetrov
Copy link
Author

gpetrov commented Nov 19, 2020

Chrome just released official native version for the Apple Silicon M1 chip
Seems to be running fast and smooth.
They do packed it as Universal "fat" build - including both intel and apple silicon executables.

Only the crash handler is still on intel - don't know why.

NodeJS should compile also fine already.

So @rogerwang hope we get NWJS also there soon. Everybody is waiting.

@hthetiot
Copy link

@sbashyal
Copy link

Thanks for the information. Will support it soon.

What kind of timeline would this be?

@rogerwang
Copy link
Member

@gpetrov
Copy link
Author

gpetrov commented Nov 25, 2020

great news @rogerwang ! The base seems to be working but we have trouble compiling the native modules with nw-gyp
even if specifying the new architecture, still got some headers mixture, not sure if nw-gyp needs an update as well? Or are we not using the std=c++14 somehow.

/Users/george/.nw-gyp/0.50.0/deps/v8/include/v8-internal.h:418:38: error: no
      template named 'remove_cv_t' in namespace 'std'; did you mean 'remove_cv'?
            !std::is_same<Data, std::remove_cv_t<T>>::value>::Perform(data);
                                ~~~~~^~~~~~~~~~~
                                     remove_cv
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/type_traits:697:50: note: 
      'remove_cv' declared here
template <class _Tp> struct _LIBCPP_TEMPLATE_VIS remove_cv/Users/george/.nw-gyp/0.50.0/deps/v8/include/v8-internal.h:418:38: error: no
      template named 'remove_cv_t' in namespace 'std'; did you mean 'remove_cv'?
            !std::is_same<Data, std::remove_cv_t<T>>::value>::Perform(data);
                                ~~~~~^~~~~~~~~~~
                                     remove_cv
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/type_traits:697:50: note: 
      'remove_cv' declared here
template <class _Tp> struct _LIBCPP_TEMPLATE_VIS remove_cv

@rogerwang
Copy link
Member

It works for me with this test: https://github.com/nwjs/nw.js/tree/nw50/test/sanity/native-module-buffer

@gpetrov
Copy link
Author

gpetrov commented Nov 25, 2020

well I tried nodegit and node-pty and both got the same error, see also:
nwjs/nw-gyp#145

@gpetrov
Copy link
Author

gpetrov commented Nov 25, 2020

@rogerwang well managed to compile both nodegit and node-pty for the arm64 now, but on require in NWJS I get a hard crash with:

[1125/113332.123338:WARNING:process_memory_mac.cc(93)] mach_vm_read(0x16b384000, 0x8000): (os/kern) protection failure (2)

They seems to be compiled now well for the arm64:

file node_modules/node-pty/build/Release/pty.node   
node_modules/node-pty/build/Release/pty.node: Mach-O 64-bit bundle arm64

Any ideas?

@rogerwang
Copy link
Member

try patch common.gypi in ~/.nw-gyp as:

diff --git a/common.gypi b/common.gypi
index 2fdb2c5b36..1241ddb45f 100644
--- a/common.gypi
+++ b/common.gypi
@@ -119,7 +119,7 @@
         'obj_dir%': '<(PRODUCT_DIR)/obj.target',
         #'v8_base': '<(PRODUCT_DIR)/obj.target/tools/v8_gypfiles/libv8_snapshot.a',
       }],
-      ['target_arch=="x64"', {
+      ['target_arch=="x64" or target_arch=="arm64"', {
         'v8_enable_pointer_compression': 1,
         'v8_enable_31bit_smis_on_64bit_arch': 1,
       }],
 

@gpetrov
Copy link
Author

gpetrov commented Nov 25, 2020

Thanks will try that! For some miraculous reason the new compiled node modules loaded just fine in the subsequent
launch of NWJS (without recompiling with the patch)

I'm trying to compile more modules like sharp - but seem they all have to be adjusted for bad xcode_settings in binding.gyp

all modules have something like this:

"xcode_settings": {
            "GCC_ENABLE_CPP_EXCEPTIONS": "YES",
            "MACOSX_DEPLOYMENT_TARGET": "10.9",
            'CLANG_CXX_LIBRARY': 'libc++',
            'CLANG_CXX_LANGUAGE_STANDARD': 'c++11',

            "WARNING_CFLAGS": [
              "-Wno-unused-variable",
              "-Wint-conversions",
              "-Wmissing-field-initializers",
              "-Wno-c++11-extensions"
            ]
          }

while they should be using c++14 and SDK 11 for arm64 on the Mac with arm64
Can you fix that in nw-gyp? so it is done automatically? I think the newer node-gyp does that.

@gpetrov
Copy link
Author

gpetrov commented Nov 25, 2020

@rogerwang next to your patch I added also in common.gypi at about line 291 new settings for Xcode_settings:

'xcode_settings': {
          'GCC_OPTIMIZATION_LEVEL': '3', # stop gyp from defaulting to -Os
          'conditions': [
            ['target_arch=="arm64"', {
              'CLANG_CXX_LANGUAGE_STANDARD': 'c++14',
              "OTHER_CPLUSPLUSFLAGS": [
                "-std=c++14",
                "-stdlib=libc++"
              ],
            }]
          ],
        },

I'm happy to report all native modules compile now fine on Mac M1 arm64! Congratulations!

@gpetrov
Copy link
Author

gpetrov commented Nov 25, 2020

@rogerwang still some modules like node-pty are crashing NWJS with:
[1125/173514.526302:WARNING:process_memory_mac.cc(93)] mach_vm_read(0x16fa5c000, 0x8000): (os/kern) protection failure (2)

while all the patches are applied in common.gypi
here are some dump files:

Archive.zip

Hope you can solve it and release the NWJS Mac arm64 build as official.
Please do also include all debug symbols in the official Mac version as requested in: #7340

@gpetrov
Copy link
Author

gpetrov commented Nov 25, 2020

hmm seems this is more related to the requirements from Apple to have all native modules code signed ....
after running:

/usr/bin/xcrun codesign --force --sign - --preserve-metadata=identifier,entitlements node_modules/node-pty/build/Release/pty.node

no more crashes occur for node-pty!

@rogerwang Maybe we should integrate default code signing in each nw-gyp native node module compilation, otherwise if not a crash - a nasty popup appears that the node module is not verified and can only be moved to the trash can. This is very annoying during development/

Of course when we package the final app we sign every binary file officially again.

@gpetrov
Copy link
Author

gpetrov commented Nov 27, 2020

As for the code signing @rogerwang , see golang/go#38485 (comment) and the linked topic golang/go#42684 (comment) for more info, internals and bypass. Might be useful for native node modules compilation flow.

@gpetrov
Copy link
Author

gpetrov commented Dec 3, 2020

any progress on the official build @rogerwang ? We are good to go!

@gpetrov
Copy link
Author

gpetrov commented Dec 28, 2020

Any news @rogerwang - we really like to get an official build, everything to run smoothly on the new M1.

The MacOS Homebrew is already up and running fine natively on M1 with almost all software available asp rebuild binaries "bottles"

So we really hope to see the official NWJS build for the M1 soon!

@corwin-of-amber
Copy link
Contributor

Cheers @rogerwang — your build runs smoothly on my MacBook Air M1 and I was even able to build native modules with plain node-gyp.

I am trying to help with pushing this release forward, so far I have been struggling with the build instructions; I started with a Linux build, which I hoped would be more manageable, but I got stuck with an unmet dependency out/nw/libvulkan.so when generating the ninja files. Are the build instructions up-to-date?

@corwin-of-amber

This comment has been minimized.

@corwin-of-amber
Copy link
Contributor

Now I am facing some missing symbols in v8.

I rebuilt everything and now it links — HOLY SNACKS!
This was just the Linux build, sorry for spamming this issue thread too much, I will have to get my hands on a fast enough Intel Mac connected to a fast enough network (according to the Chromium docs, building for ARM64 is only supported on Intel Macs, I wonder why that is).

@rogerwang
Copy link
Member

rogerwang commented Jan 3, 2021 via email

@rogerwang
Copy link
Member

rogerwang commented Jan 3, 2021 via email

@corwin-of-amber
Copy link
Contributor

Re: the experimental 0.50.1 you posted earlier — I discovered that it crashes when trying to load a WebAssembly. This happened when the binary is streamed (i.e. loaded via a URL, rather than given as Uint8Array). I haven't had a chance to test other scenarios yet.

@gpetrov
Copy link
Author

gpetrov commented Jan 5, 2021

@rogerwang you do know that you can build mac arm on intel, so infrastructure change is not really needed. Google Chromium describes it well: https://chromium.googlesource.com/chromium/src/+/master/docs/mac_arm64.md
or is your mac build worker just outdated and slow?

@zkrige
Copy link

zkrige commented Feb 24, 2023

Is there any update? @dpage kindly provided a Mac to build on more than 3 months ago, and there has been no feedback since then? We have a number of clients begging for apple silicon version of our app and we're sitting with egg on our face because we're unable to help them.

I'd be more than happy to help if I'm able to - please just let me know what you need to have done

@corwin-of-amber
Copy link
Contributor

corwin-of-amber commented Feb 24, 2023

@zkrige If you have a Mac available you can use these tips to build NWjs yourself. It is based on the official build instructions with my tips.

Also here are the exact actions I use in my quick-and-dirty CI (in JSON format).

I will resume my builds but right now I do not have access to my build machine due to a network outage 😱

@ayushmanchhabra
Copy link
Contributor

ayushmanchhabra commented Feb 25, 2023

I will resume my builds

I've added MacOS ARM support on nw-builder using the @corwin-of-amber's builds! It is available from v4.1.0-beta.3 onward. I have a very basic demo set up. I don't have access to a Mac and am looking for feedback. Hope this helps!

@zkrige
Copy link

zkrige commented Feb 25, 2023

@tharatau i have a Mac that I work on. I've been looking to convert our build process from nwjs-builder to your more modern builder anyways. I'll gladly test it and give you feedback

@ayushmanchhabra
Copy link
Contributor

ayushmanchhabra commented Feb 25, 2023

@zkrige Feel free to contact me via the nwutils Discord server or Gitter chatroom, the links are posted on the website:
https://nwutils.io/

@JeFawk
Copy link

JeFawk commented Apr 11, 2023

Any news on this & a stable release featuring it?

@corwin-of-amber
Copy link
Contributor

I have been trying to re-instate my unofficial M1 builds since there does not seem to be sufficient development. Over the last few weeks I was stumbling because of a resurfacing issue causing the non-SDK build to fail:
#6579
It seems that chrome/browser/nwjs_resources.grd is somehow out of sync. I am trying this workaround (thanks @llamasoft) and will report whether the result works.

If anyone has a clue on what's going on there, help is extremely welcome.

I am also not able to build the payload executable anymore, but it seems that it has been removed from the official SDK as well? Perhaps @nebular can confirm that it is no longer needed.

@nebular
Copy link

nebular commented May 13, 2023

Hi @corwin-of-amber ! Sorry i have been absent quite some time - been using your version 0.66.0 that has been running flawlessly for me for this long time. Glad to hear you are reinstating the unofficial builds. Thanks so much for that, I appreciate the big effort you put on it.

I can confirm I am not using "payload", thanks for asking.

Have upgraded my software "Obscene Studio" (https://obscene.studio) to your latest version 0.75 - also to test it in a new machine with a M2 PRO + Ventura 13.3.1. Encountered some irritating issues but at the end it is more a Ventura / Chromium thing .-- if you are curious it is about a chrome bug and other related one that causes getUserMedia calls to take a very long time to resolve, or even freeze until you change cameras ... a disaster :(

@rogerwang
Copy link
Member

Please try this testing build: https://dl.nwjs.io/live-build/nw76/20230516-150708/a88c13a6c/v0.76.1/

I'm setting up the buildbot and it is scheduled to be released with 0.77 in June.

@corwin-of-amber
Copy link
Contributor

Amazing! Thank you so much @rogerwang!
And thanks to all the wonderful M1 activists in this thread :D

@dpage
Copy link

dpage commented May 16, 2023

@rogerwang Thanks! However, it won't open at all on my machine as it doesn't appear to be signed. It probably will open on the build machine (as I assume that's where you tested it), as it knows you built it on that box.

@JeFawk
Copy link

JeFawk commented May 16, 2023

Thank you so much!!! Looking forward to a stable build so that web apps like Scirra (construct.net) can use it so and so we can publish HTML5 games using nwjs on apple native ^^

🥳🎉🎊

@rogerwang
Copy link
Member

@rogerwang Thanks! However, it won't open at all on my machine as it doesn't appear to be signed. It probably will open on the build machine (as I assume that's where you tested it), as it knows you built it on that box.

After removing the quarantine bit it works for me in another system. Will fix it in the next build.

sudo xattr -r -d com.apple.quarantine /path/to/MyApp.app

@nebular
Copy link

nebular commented May 16, 2023

omg omg omg its xmas !! thanks so much @rogerwang
btw, could you elaborate a little if you have fixed specific things ? being for long a user of @corwin-of-amber builds that work pretty well - but in M2 + Ventura getUserMedia is quite unstable

@dpage
Copy link

dpage commented May 17, 2023

@rogerwang Thanks! However, it won't open at all on my machine as it doesn't appear to be signed. It probably will open on the build machine (as I assume that's where you tested it), as it knows you built it on that box.

After removing the quarantine bit it works for me in another system. Will fix it in the next build.

sudo xattr -r -d com.apple.quarantine /path/to/MyApp.app

Oh, neat - didn't realise you could do that. The basic app runs now - I'll try it in a build of pgAdmin and report any issues.

Thanks!

@TongDaDa
Copy link

TongDaDa commented Aug 4, 2023

emm, does nwjs support the m1 chip now? @rogerwang

@JeFawk
Copy link

JeFawk commented Aug 4, 2023

emm, does nwjs support the m1 chip now? @rogerwang

Yes, for about 2 months now, v 0.77 i think

@TongDaDa
Copy link

TongDaDa commented Aug 7, 2023

@rogerwang @JeFawk

Hi, Is it possible to provide a universal nwjs sdk? other than x64 & arm64

ref: https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary

A few later... I use the lipo -create -output command to combine all the binary of nwjs-sdk.app into a single universal binary. BUT, the v8_context_snapshot file seems to be incompatible with this command,

throwing error with this command: fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: can't figure out the architecture type of: /Users/xxx/Desktop/build/nwjs_arm64.app/Contents/Frameworks/nwjs Framework.framework/Versions/115.0.5790.114/Resources/v8_context_snapshot.arm64.bin

lipo -create -output ./universal64_snapshot /Users/xxx/Desktop/build/nwjs_arm64.app/Contents/Frameworks/nwjs\ Framework.framework/Versions/115.0.5790.114/Resources/v8_context_snapshot.arm64.bin /Users/xxx/Desktop/build/nwjs_x64.app/Contents/Frameworks/nwjs\ Framework.framework/Versions/115.0.5790.114/Resources/v8_context_snapshot.x86_64.bin

@bluthen
Copy link
Contributor

bluthen commented Aug 8, 2023

What if you just leave that as two separate files? I think the executable nwjs Framework will load each file specifically by name (including the arch), not a general library name without arch like the others.

@TongDaDa
Copy link

TongDaDa commented Aug 8, 2023

What if you just leave that as two separate files? I think the executable nwjs Framework will load each file specifically by name (including the arch), not a general library name without Archs like the others.

@bluthen You can download the two arch bundles in the URL: https://dl.nwjs.io/v0.77.0/. x64 & arm64. most files can combine a single file from two binaries, but the Contents/Frameworks/nwjs Framework.framework/Versions/115.0.5790.114/Resources/v8_context_snapshot.arm64.bin file cannot be unverified

F4rG41NS3h

@bluthen
Copy link
Contributor

bluthen commented Aug 8, 2023

Sorry if I'm wasting your time. I thought those files are just snapshots of v8 heap when it first loads and not a real binary, so I don't know why you need to combine them.

@TongDaDa
Copy link

TongDaDa commented Aug 8, 2023

Sorry if I'm wasting your time. I thought those files are just snapshots of v8 heap when it first loads and not a real binary, so I don't know why you need to combine them.

@bluthen Soooorry, It does not need to combine them after looking over the chrome fat release, and placing the two arches in the directory.

截屏2023-08-08 15 17 36

Totally, the purpose I did that is to build a UNIVERSAL nwjs build by lipo macos command, this means running an app on the x64 & arm64 platforms.

It is realized by electron, expected for nwjs.

https://stackoverflow.com/questions/73569950/how-to-build-a-universal-bundle-for-macos-on-apple-silicon-electron-builder

@MikalDev
Copy link

MikalDev commented Apr 20, 2024

FYI for the Construct 3 platform which exports to two different Mac versions separately (ARM + Intel), I created a CLI tool to make a universal nwjs app from the two versions. Very early release, testing ongoing, works w/ small C3 demo apps exports and also later C3 Greengrinds addon (and probably C3 greenworks too)

https://github.com/MikalDev/fattennwjs open to PRs

Basic idea is to go through the ARM dir find all executables and fatten them from the intel dir, some special cases needed which complicates the code a little. Also does the v8_context copy from intel to arm.

@MikalDev
Copy link

I noticed that permissions did not work with the fattened version of the nw.js, unless the app was signed again, so added an optional --sign parameter to the fattennwjs app to do a codesign --force --deep -s "${keyName}" "${appPath}" - which works well enough for adhoc. For app store release, etc. there may be more work involved and you may not want to do the --force --deep, instead walk through the executables and only sign what is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests