[ci] Create hermes-engine-darwin npm package #297

alloy · 2020-07-16T17:53:44Z

⚠️ This includes the changes of #296, so I’ll have to rebase this PR once that lands. The changes that pertain to this PR are in 9da61f7

Summary

Adds hermes-engine-darwin to the npm CI job.

Notably the symbolic links need to be stripped from the framework bundle, as npm by design does not support symbolic links to exist in packages. However, seeing as we already package distinct versions of the framework in distinct npm packages, there isn't any need to support multiple versions of Hermes in a single framework bundle; so this should be fine.

Test Plan

TODO: I’ll need to backport all changes to Hermes v0.4.1 in order to pull it into RN macOS v0.62. In the meantime I want to see if CI is correctly cutting this package.

alloy · 2020-07-16T18:21:20Z

~~Note to self: need to fix the rpath now that the binary no longer exists in a nested version dir.~~ Done

Squashed version of facebook#285 & facebook#290 & facebook#296 & facebook#297

alloy · 2020-07-16T21:05:40Z

@willholen Seeing as I don’t have the ability to SSH into the CI jobs, could you perhaps do so and debug the prepack script? It works fine locally, but fails on CI like so https://app.circleci.com/pipelines/github/facebook/hermes/1064/workflows/a5cc7cf1-7726-4fa3-aba2-82a98d917f62/jobs/7208/steps

Notably the symbolic links need to be stripped from the framework bundle, as npm by design does not support symbolic links to exist in packages. However, seeing as we already package distinct versions of the framework in distinct npm packages, there isn't any need to support multiple versions of Hermes in a single framework bundle; so this should be fine.

willholen · 2020-07-16T23:46:19Z

I downloaded the artifact from https://7273-154201259-gh.circle-artifacts.com/0/tmp/hermes/output/hermes-engine-darwin-v0.5.0.tgz and it looks complete as far as I can tell. Very nice!

@alloy The title still says "[DO NOT MERGE]". Is there still testing and tweaking remaining?

alloy · 2020-07-20T12:53:43Z

@willholen Great! Yeah I first want to test integration into RN macOS, but have to do so with the other build of the 0.4 branch. Will update once I’ve done that.

alloy · 2020-07-20T16:55:01Z

Ok, the framework works, but unfortunately removing the symbolic link meant lldb no longer automatically can find the dSYM bundle. So looks like I’ll have to revert the change of removing the symlink and instead rebuild the symlinks in a postinstall script of the packages package.json.

willholen · 2020-07-20T17:46:07Z

@alloy Oh, that's frustrating :(

I don't know how CocoaPods use symlinks, but some generic thoughts are:

If it's complicated by the version numbers, the symlink could just be hard coded to "current" or "0.0.0" or something since the package is already versioned through NPM
Files could simply be duplicated (e.g. via rsync -L). It would take up a bit more space, but it would presumably cause fewer issues if a project is cloned on or copied via Windows
The framework could be added to the NPM as a .tar.gz instead of as its unpacked contents. That way, MacOS can unpack it complete with symlinks, while Windows can ignore it (since it can't use it anyways) and have a smaller on-disk footprint.

alloy · 2020-07-20T20:50:10Z

The framework could be added to the NPM as a .tar.gz instead of as its unpacked contents. That way, MacOS can unpack it complete with symlinks, while Windows can ignore it (since it can't use it anyways) and have a smaller on-disk footprint.

Oh yeah, interesting thought—I like it 👍

Instead packge the binaries in an extra layer of tarball, which also saves some bytes on-disk on platforms that don't need it.

alloy · 2020-07-21T19:32:16Z

Ok, fixed the issue by adding an extra layer of tarball, as suggested. This should be good to go now 👍

facebook-github-bot

@willholen has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

CMakeLists.txt

npm/hermes-engine-darwin/unpack-tarball.js

facebook-github-bot · 2020-07-24T15:48:05Z

@alloy has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2020-07-27T12:04:05Z

@alloy has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot

@willholen has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2020-07-28T00:38:03Z

@willholen merged this pull request in d3394bc.

alloy · 2020-07-28T09:51:25Z

@willholen @mhorowitz Thanks for your help getting this over the finish line! 🙏

Summary: {emoji:26a0} _This includes the changes of #296, so I’ll have to rebase this PR once that lands. The changes that pertain to this PR are in 9da61f7a9da01b5bd49e95dca0b98a5204210790_ Adds hermes-engine-darwin to the npm CI job. Notably the symbolic links need to be stripped from the framework bundle, as npm by design does not support symbolic links to exist in packages. However, seeing as we already package distinct versions of the framework in distinct npm packages, there isn't any need to support multiple versions of Hermes in a single framework bundle; so this should be fine. Pull Request resolved: #297 Test Plan: TODO: I’ll need to backport all changes to Hermes v0.4.1 in order to pull it into RN macOS v0.62. In the meantime I want to see if CI is correctly cutting this package. Reviewed By: mhorowitz Differential Revision: D22700399 Pulled By: willholen fbshipit-source-id: f72aefa3fd98a85c8da3e9a38dfa6be9723a0270

Summary: {emoji:26a0} _This includes the changes of facebook#296, so I’ll have to rebase this PR once that lands. The changes that pertain to this PR are in facebook@9da61f7a9da01b5bd49e95dca0b98a5204210790_ Adds hermes-engine-darwin to the npm CI job. Notably the symbolic links need to be stripped from the framework bundle, as npm by design does not support symbolic links to exist in packages. However, seeing as we already package distinct versions of the framework in distinct npm packages, there isn't any need to support multiple versions of Hermes in a single framework bundle; so this should be fine. Pull Request resolved: facebook#297 Test Plan: TODO: I’ll need to backport all changes to Hermes v0.4.1 in order to pull it into RN macOS v0.62. In the meantime I want to see if CI is correctly cutting this package. Reviewed By: mhorowitz Differential Revision: D22700399 Pulled By: willholen fbshipit-source-id: f72aefa3fd98a85c8da3e9a38dfa6be9723a0270

* Build macOS framework and add CocoaPods podspec (#285) Summary: Supersedes #239 Currently used in our macOS fork of React Native microsoft/react-native-macos#473. (Although we’re using a build of Hermes v0.4.1, as we’re at RN 0.62.0.) * On Apple platforms build a [dynamic] framework bundle when `HERMES_BUILD_APPLE_FRAMEWORK` is set. When set to `FALSE` it will produce a `dylib`, like previously. Defaults to `TRUE`. * On Apple platforms create a debugging symbols bundle. * Add `HERMES_ENABLE_FUZZING`, which is enabled by default. * Add `HERMES_ENABLE_TEST_SUITE`, which is enabled by default. * Add a CocoaPods podspec that can build from source or use a binary. A standalone macOS app that pulls in Hermes as a CocoaPods pod can be found here https://github.com/alloy/TestHermesMaster. ``` $ ./src/utils/build/configure.py --distribute --cmake-flags='-DCMAKE_INSTALL_PREFIX:PATH=../destroot_release' build $ cd build && ninja install && cd .. $ tree destroot_release/ destroot_release/ ├── Library │ └── Frameworks │ ├── hermes.framework │ │ ├── Headers -> Versions/Current/Headers │ │ ├── Resources -> Versions/Current/Resources │ │ ├── Versions │ │ │ ├── 0 │ │ │ │ ├── Headers │ │ │ │ │ ├── CompileJS.h │ │ │ │ │ ├── DebuggerAPI.h │ │ │ │ │ ├── Public │ │ │ │ │ │ ├── Buffer.h │ │ │ │ │ │ ├── CrashManager.h │ │ │ │ │ │ ├── CtorConfig.h │ │ │ │ │ │ ├── DebuggerTypes.h │ │ │ │ │ │ ├── GCConfig.h │ │ │ │ │ │ ├── GCTripwireContext.h │ │ │ │ │ │ └── RuntimeConfig.h │ │ │ │ │ ├── SynthTrace.h │ │ │ │ │ ├── SynthTraceParser.h │ │ │ │ │ ├── TraceInterpreter.h │ │ │ │ │ ├── TracingRuntime.h │ │ │ │ │ ├── hermes.h │ │ │ │ │ └── hermes_tracing.h │ │ │ │ ├── Resources │ │ │ │ │ └── Info.plist │ │ │ │ └── hermes │ │ │ └── Current -> 0 │ │ └── hermes -> Versions/Current/hermes │ └── hermes.framework.dSYM │ └── Contents │ ├── Info.plist │ └── Resources │ └── DWARF │ └── hermes ├── bin │ ├── hbcdump │ ├── hdb │ ├── hermes │ ├── hermesc │ └── hvm └── include ├── hermes │ ├── CompileJS.h │ ├── DebuggerAPI.h │ ├── Public │ │ ├── Buffer.h │ │ ├── CrashManager.h │ │ ├── CtorConfig.h │ │ ├── DebuggerTypes.h │ │ ├── GCConfig.h │ │ ├── GCTripwireContext.h │ │ └── RuntimeConfig.h │ ├── SynthTrace.h │ ├── SynthTraceParser.h │ ├── TraceInterpreter.h │ ├── TracingRuntime.h │ ├── hermes.h │ └── hermes_tracing.h └── jsi ├── JSIDynamic.h ├── decorator.h ├── instrumentation.h ├── jsi-inl.h ├── jsi.h ├── jsilib.h └── threadsafe.h ``` ``` $ ./src/utils/build/configure.py --distribute --cmake-flags='-DHERMES_BUILD_APPLE_FRAMEWORK:BOOLEAN=false -DCMAKE_INSTALL_PREFIX:PATH=../destroot_release' build $ cd build && ninja install && cd .. $ tree destroot_release/ destroot_release/ ├── bin │ ├── hbcdump │ ├── hdb │ ├── hermes │ ├── hermesc │ └── hvm ├── include │ ├── hermes │ │ ├── CompileJS.h │ │ ├── DebuggerAPI.h │ │ ├── Public │ │ │ ├── Buffer.h │ │ │ ├── CrashManager.h │ │ │ ├── CtorConfig.h │ │ │ ├── DebuggerTypes.h │ │ │ ├── GCConfig.h │ │ │ ├── GCTripwireContext.h │ │ │ └── RuntimeConfig.h │ │ ├── SynthTrace.h │ │ ├── SynthTraceParser.h │ │ ├── TraceInterpreter.h │ │ ├── TracingRuntime.h │ │ ├── hermes.h │ │ └── hermes_tracing.h │ └── jsi │ ├── JSIDynamic.h │ ├── decorator.h │ ├── instrumentation.h │ ├── jsi-inl.h │ ├── jsi.h │ ├── jsilib.h │ └── threadsafe.h └── lib ├── libhermes.dylib └── libhermes.dylib.dSYM └── Contents ├── Info.plist └── Resources └── DWARF └── libhermes.dylib ``` Pull Request resolved: #285 Reviewed By: willholen Differential Revision: D22398354 Pulled By: mhorowitz fbshipit-source-id: 732524275cf273866171fc6e2ac2acb062185fbd * Add job to build macOS CocoaPods binaries (#290) Summary: Adds a CI job to build binaries for macOS to be used via CocoaPods. Pull Request resolved: #290 Test Plan: ``` $ mkdir test-hermes-build $ cd test-hermes-build/ $ curl -L -O https://6909-154201259-gh.circle-artifacts.com/0/tmp/hermes/output/hermes-runtime-darwin-v0.5.0.tar.gz $ tar -zxvf hermes-runtime-darwin-v0.5.0.tar.gz $ tree . . ├── LICENSE ├── README.md ├── destroot │ ├── Library │ │ └── Frameworks │ │ ├── hermes.framework │ │ │ ├── Headers │ │ │ │ ├── CompileJS.h │ │ │ │ ├── DebuggerAPI.h │ │ │ │ ├── Public │ │ │ │ │ ├── Buffer.h │ │ │ │ │ ├── CrashManager.h │ │ │ │ │ ├── CtorConfig.h │ │ │ │ │ ├── DebuggerTypes.h │ │ │ │ │ ├── GCConfig.h │ │ │ │ │ ├── GCTripwireContext.h │ │ │ │ │ └── RuntimeConfig.h │ │ │ │ ├── SynthTrace.h │ │ │ │ ├── SynthTraceParser.h │ │ │ │ ├── TraceInterpreter.h │ │ │ │ ├── TracingRuntime.h │ │ │ │ ├── hermes.h │ │ │ │ └── hermes_tracing.h │ │ │ ├── Resources │ │ │ │ └── Info.plist │ │ │ ├── Versions │ │ │ │ ├── 0 │ │ │ │ │ ├── Headers │ │ │ │ │ │ ├── CompileJS.h │ │ │ │ │ │ ├── DebuggerAPI.h │ │ │ │ │ │ ├── Public │ │ │ │ │ │ │ ├── Buffer.h │ │ │ │ │ │ │ ├── CrashManager.h │ │ │ │ │ │ │ ├── CtorConfig.h │ │ │ │ │ │ │ ├── DebuggerTypes.h │ │ │ │ │ │ │ ├── GCConfig.h │ │ │ │ │ │ │ ├── GCTripwireContext.h │ │ │ │ │ │ │ └── RuntimeConfig.h │ │ │ │ │ │ ├── SynthTrace.h │ │ │ │ │ │ ├── SynthTraceParser.h │ │ │ │ │ │ ├── TraceInterpreter.h │ │ │ │ │ │ ├── TracingRuntime.h │ │ │ │ │ │ ├── hermes.h │ │ │ │ │ │ └── hermes_tracing.h │ │ │ │ │ ├── Resources │ │ │ │ │ │ └── Info.plist │ │ │ │ │ └── hermes │ │ │ │ └── Current │ │ │ │ ├── Headers │ │ │ │ │ ├── CompileJS.h │ │ │ │ │ ├── DebuggerAPI.h │ │ │ │ │ ├── Public │ │ │ │ │ │ ├── Buffer.h │ │ │ │ │ │ ├── CrashManager.h │ │ │ │ │ │ ├── CtorConfig.h │ │ │ │ │ │ ├── DebuggerTypes.h │ │ │ │ │ │ ├── GCConfig.h │ │ │ │ │ │ ├── GCTripwireContext.h │ │ │ │ │ │ └── RuntimeConfig.h │ │ │ │ │ ├── SynthTrace.h │ │ │ │ │ ├── SynthTraceParser.h │ │ │ │ │ ├── TraceInterpreter.h │ │ │ │ │ ├── TracingRuntime.h │ │ │ │ │ ├── hermes.h │ │ │ │ │ └── hermes_tracing.h │ │ │ │ ├── Resources │ │ │ │ │ └── Info.plist │ │ │ │ └── hermes │ │ │ └── hermes │ │ └── hermes.framework.dSYM │ │ └── Contents │ │ ├── Info.plist │ │ └── Resources │ │ └── DWARF │ │ └── hermes │ ├── bin │ │ ├── hbcdump │ │ ├── hdb │ │ ├── hermes │ │ ├── hermesc │ │ └── hvm │ └── include │ ├── hermes │ │ ├── CompileJS.h │ │ ├── DebuggerAPI.h │ │ ├── Public │ │ │ ├── Buffer.h │ │ │ ├── CrashManager.h │ │ │ ├── CtorConfig.h │ │ │ ├── DebuggerTypes.h │ │ │ ├── GCConfig.h │ │ │ ├── GCTripwireContext.h │ │ │ └── RuntimeConfig.h │ │ ├── SynthTrace.h │ │ ├── SynthTraceParser.h │ │ ├── TraceInterpreter.h │ │ ├── TracingRuntime.h │ │ ├── hermes.h │ │ └── hermes_tracing.h │ └── jsi │ ├── JSIDynamic.h │ ├── decorator.h │ ├── instrumentation.h │ ├── jsi-inl.h │ ├── jsi.h │ ├── jsilib.h │ └── threadsafe.h ├── hermes-runtime-darwin-v0.5.0.tar.gz └── hermes.podspec 25 directories, 84 files ``` Reviewed By: mhorowitz Differential Revision: D22510579 Pulled By: willholen fbshipit-source-id: 31a234b908090cdcbf7d1eafe78269ed19583883 * Add explicit option to build Apple dSYM (#296) Summary: Add `HERMES_BUILD_APPLE_DSYM` option to build a dSYM bundle for the libhermes target on Apple platforms. This will work with any of the build types and is off by default. Installing the tools with the install/strip target will ensure all tools and the runtime lib are stripped of debug symbols, but leaving the dSYM bundle in tact. Pull Request resolved: #296 Test Plan: ```bash ./src/utils/build/configure.py --distribute --cmake-flags='-DHERMES_BUILD_APPLE_DSYM:BOOLEAN=true -DCMAKE_INSTALL_PREFIX:PATH=../destroot_release' build_release cmake --build ./build_release ``` ```bash cmake --build ./build_release --target install nm -a destroot_release/bin/hermesc | wc -l 27943 ``` ```bash cmake --build ./build_release --target install/strip nm -a destroot_release/bin/hermesc | wc -l 250 ``` …and dSYM DWARF metadata is maintained: ```bash dwarfdump --statistics destroot_release/Library/Frameworks/hermes.framework.dSYM {"version":3,"file":"destroot_release/Library/Frameworks/hermes.framework.dSYM/Contents/Resources/DWARF/hermes","format":"Mach-O 64-bit x86-64","source functions":30305,"source functions with location":30302,"inlined functions":172725,"inlined funcs with abstract origins":172725,"unique source variables":79276,"source variables":353690,"variables with location":232195,"call site entries":186409,"scope bytes total":19161949,"scope bytes covered":10500176,"total function size":1763513,"total inlined function size":998375,"total formal params":300264,"formal params with source location":166067,"formal params with type":300264,"formal params with binary location":200407,"total vars":38809,"vars with source location":38385,"vars with type":38809,"vars with binary location":22161} ``` Reviewed By: tmikov Differential Revision: D22576263 Pulled By: willholen fbshipit-source-id: 2bda49e638d145ba5d029e77069d6adcc0b1dd8c * Create hermes-engine-darwin npm package (#297) Summary: {emoji:26a0} _This includes the changes of #296, so I’ll have to rebase this PR once that lands. The changes that pertain to this PR are in 9da61f7a9da01b5bd49e95dca0b98a5204210790_ Adds hermes-engine-darwin to the npm CI job. Notably the symbolic links need to be stripped from the framework bundle, as npm by design does not support symbolic links to exist in packages. However, seeing as we already package distinct versions of the framework in distinct npm packages, there isn't any need to support multiple versions of Hermes in a single framework bundle; so this should be fine. Pull Request resolved: #297 Test Plan: TODO: I’ll need to backport all changes to Hermes v0.4.1 in order to pull it into RN macOS v0.62. In the meantime I want to see if CI is correctly cutting this package. Reviewed By: mhorowitz Differential Revision: D22700399 Pulled By: willholen fbshipit-source-id: f72aefa3fd98a85c8da3e9a38dfa6be9723a0270 * Kludge unit test resolution of Handle * Remove special build directory suffixes in configure.py Summary: Right now, `configure.py` generates different directory suffixes depending on the build mode. This can lead to some slightly surprising behaviour when people build for ASAN/UBSAN and can also make automated scripts that use `configure.py` clunkier. This change removes those special suffixes when a build directory is specified. Reviewed By: dulinriley Differential Revision: D21268445 fbshipit-source-id: 90f5497f3930dee35dbea7e8f7ad64d6ac244fca Co-authored-by: Eloy Durán <eloy.de.enige@gmail.com> Co-authored-by: Neil Dhar <neildhar@fb.com>

facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Jul 16, 2020

alloy changed the title ~~[ci] Create hermes-engine-darwin npm package~~ [DO NOT MERGE] [ci] Create hermes-engine-darwin npm package Jul 16, 2020

alloy force-pushed the cut-macos-npm-package branch from 9da61f7 to 2cd9ea2 Compare July 16, 2020 17:55

alloy force-pushed the cut-macos-npm-package branch from 2cd9ea2 to c70ffab Compare July 16, 2020 18:43

alloy added a commit to alloy/hermes that referenced this pull request Jul 16, 2020

Build macOS framework, add CocoaPods podspec, cut npm package

152aa79

Squashed version of facebook#285 & facebook#290 & facebook#296 & facebook#297

alloy force-pushed the cut-macos-npm-package branch from c70ffab to 0e26f58 Compare July 16, 2020 19:31

alloy force-pushed the cut-macos-npm-package branch from 0e26f58 to f7a7bbf Compare July 16, 2020 21:35

alloy force-pushed the cut-macos-npm-package branch from f7a7bbf to d8295cb Compare July 16, 2020 21:37

alloy force-pushed the cut-macos-npm-package branch from 4146c14 to d6afe42 Compare July 21, 2020 19:27

[ci] Fix dSYM loading by preserving symlinks

3e49044

Instead packge the binaries in an extra layer of tarball, which also saves some bytes on-disk on platforms that don't need it.

alloy force-pushed the cut-macos-npm-package branch from d6afe42 to 3e49044 Compare July 21, 2020 19:30

alloy changed the title ~~[DO NOT MERGE] [ci] Create hermes-engine-darwin npm package~~ [ci] Create hermes-engine-darwin npm package Jul 21, 2020

facebook-github-bot reviewed Jul 23, 2020

View reviewed changes

mhorowitz reviewed Jul 24, 2020

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

npm/hermes-engine-darwin/unpack-tarball.js Outdated Show resolved Hide resolved

npm/hermes-engine-darwin/unpack-tarball.js Outdated Show resolved Hide resolved

Address PR feedback.

02e2330

Pickup tar from user env

b436879

facebook-github-bot reviewed Jul 27, 2020

View reviewed changes

facebook-github-bot closed this in d3394bc Jul 27, 2020

facebook-github-bot added the Merged label Jul 28, 2020

alloy deleted the cut-macos-npm-package branch July 28, 2020 09:24

kelset mentioned this pull request Jan 13, 2022

rn-macos's Hermes support: RNTester on 0.64 & 0.66 microsoft/react-native-macos#952

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] Create hermes-engine-darwin npm package #297

[ci] Create hermes-engine-darwin npm package #297

alloy commented Jul 16, 2020

alloy commented Jul 16, 2020 •

edited

alloy commented Jul 16, 2020

willholen commented Jul 16, 2020 •

edited

alloy commented Jul 20, 2020

alloy commented Jul 20, 2020

willholen commented Jul 20, 2020

alloy commented Jul 20, 2020

alloy commented Jul 21, 2020

facebook-github-bot left a comment

facebook-github-bot commented Jul 24, 2020

facebook-github-bot commented Jul 27, 2020

facebook-github-bot left a comment

facebook-github-bot commented Jul 28, 2020

alloy commented Jul 28, 2020

[ci] Create hermes-engine-darwin npm package #297

[ci] Create hermes-engine-darwin npm package #297

Conversation

alloy commented Jul 16, 2020

Summary

Test Plan

alloy commented Jul 16, 2020 • edited

alloy commented Jul 16, 2020

willholen commented Jul 16, 2020 • edited

alloy commented Jul 20, 2020

alloy commented Jul 20, 2020

willholen commented Jul 20, 2020

alloy commented Jul 20, 2020

alloy commented Jul 21, 2020

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot commented Jul 24, 2020

facebook-github-bot commented Jul 27, 2020

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot commented Jul 28, 2020

alloy commented Jul 28, 2020

alloy commented Jul 16, 2020 •

edited

willholen commented Jul 16, 2020 •

edited