New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-CPU compilation (kind of). ARM. Raspberry Pi #145

Open
Pupix opened this Issue Jun 23, 2017 · 16 comments

Comments

4 participants
@Pupix
Copy link

Pupix commented Jun 23, 2017

Hello!

I am trying to automate the building and deploying process from my machine to a PI.

Currently I am developing on Ubuntu, deploying the source code to Raspberry and compiling there, since CPU cross compilation is not available.

Every build takes somewhere between 30 to 60 seconds, even for a single change I have to rebuild everything from the ground up.

Is there any way to make this process faster? Perhaps incremental builds ? Monkey patching on top of an already existing binary?

Thanks!

@igorklopov

This comment has been minimized.

Copy link
Collaborator

igorklopov commented Jun 23, 2017

Since pkg@4 there is one way. But you will need to ship the sources of your application inside executable. I mean machine-code-only feature will not work because Ubuntu-x64 can not generate ARM machine code. So, please wait. I will try to figure out the setup.

@igorklopov

This comment has been minimized.

Copy link
Collaborator

igorklopov commented Jun 26, 2017

There is a way to cross-compile to ARM cpu in pkg@4.1.1. Instructions:

  • Open examples/express. Open package.json
  • Pay attention to license field and it's value. It must be set to open source to make pkg include the source code of all files of the package into executable.
  • Compile the project with pkg package.json -t node8-linux-armv6 --no-bytecode
  • The resulting executable will work on Raspberry Pi.

If there are any compile-time or run-time errors with your project, post them here.

@igorklopov igorklopov changed the title Incremental builds? Cross-CPU compilation (kind of). ARM. Raspberry Pi Jun 26, 2017

@Pupix

This comment has been minimized.

Copy link
Author

Pupix commented Jun 27, 2017

After updating to the latest version, any build results in:

pkg@4.1.1
> Fetching base Node.js binaries to: ~/.pkg-cache
  fetched-v8.0.0-linux-x64     [                    ] 0%> Asset 'v2.3/uploaded-v2.3-node-v8.0.0-linux-x64' not found by direct link
> Error! AssertionError [ERR_ASSERTION]: false == true
    at Log.enableProgress (/home/pupix/.npm-global/lib/node_modules/pkg/node_modules/pkg-fetch/lib-es5/log.js:103:28)
    at GitHub.downloadUrl (/home/pupix/.npm-global/lib/node_modules/pkg/node_modules/pkg-fetch/lib-es5/github.js:173:16)
    at Cloud._callee3$ (/home/pupix/.npm-global/lib/node_modules/pkg/node_modules/pkg-fetch/lib-es5/cloud.js:279:32)
    at tryCatch (/home/pupix/.npm-global/lib/node_modules/pkg/node_modules/regenerator-runtime/runtime.js:65:40)
    at Generator.invoke [as _invoke] (/home/pupix/.npm-global/lib/node_modules/pkg/node_modules/regenerator-runtime/runtime.js:303:22)
    at Generator.prototype.(anonymous function) [as next] (/home/pupix/.npm-global/lib/node_modules/pkg/node_modules/regenerator-runtime/runtime.js:117:21)
    at step (/home/pupix/.npm-global/lib/node_modules/pkg/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /home/pupix/.npm-global/lib/node_modules/pkg/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:169:7)
@Pupix

This comment has been minimized.

Copy link
Author

Pupix commented Jun 27, 2017

It only happens when the binary is not cached and it tries to download it from github.

@Pupix

This comment has been minimized.

Copy link
Author

Pupix commented Jun 27, 2017

Downloaded and renamed the binaries manually.

Now whenever I run pkg package.json -t node8-linux-armv6 --no-bytecode I get the following error:

 Error! --no-bytecode and no source breaks final executable

Here the record only has STORE_BLOB, then it fails 2 lines down when STORE_CONTENT doesn't exist.

@Pupix

This comment has been minimized.

Copy link
Author

Pupix commented Jun 27, 2017

I can confirm that the express example works. The problem might be somewhere else and it silently fails without setting the content.

@igorklopov

This comment has been minimized.

Copy link
Collaborator

igorklopov commented Jun 27, 2017

STORE_BLOB - what file in what package has only STORE_BLOB? Your project or npm dependency?

igorklopov added a commit to zeit/pkg-fetch that referenced this issue Jun 27, 2017

igorklopov added a commit to zeit/pkg-fetch that referenced this issue Jun 27, 2017

igorklopov added a commit to zeit/pkg-fetch that referenced this issue Jun 27, 2017

@Pupix

This comment has been minimized.

Copy link
Author

Pupix commented Jun 28, 2017

It is my bin file.

{ '0': true,
  '3': 
   Stats {
     dev: 64512,
     mode: 33204,
     nlink: 1,
     uid: 1000,
     gid: 1000,
     rdev: 0,
     blksize: 4096,
     ino: 393274,
     size: 7388,
     blocks: 16,
     atimeMs: 1498575401147.5488,
     mtimeMs: 1498575394763.496,
     ctimeMs: 1498575394767.496,
     birthtimeMs: 1498575394767.496,
     atime: 2017-06-27T14:56:41.148Z,
     mtime: 2017-06-27T14:56:34.763Z,
     ctime: 2017-06-27T14:56:34.767Z,
     birthtime: 2017-06-27T14:56:34.767Z },
  file: '/var/www/bff847b8c0f04ba9/server.bin.js',
  marker: 
   { config: 
      { name: 'bff847b8c0f04ba9',
        description: 'experiments',
        private: true,
        license: 'MIT',
        version: '1.0.0',
        bin: 'server.bin.js',
        main: 'server.js',
        dependencies: [Object],
        scripts: [Object],
        pkg: [Object] },
     base: '/var/www/bff847b8c0f04ba9',
     toplevel: true,
     permissive: false,
     activated: true },
  body: '...' }

The difference between server.js and server.bin.js is that one uses __dirname and the other used process.cwd/process.execPath

@Pupix

This comment has been minimized.

Copy link
Author

Pupix commented Jun 28, 2017

After some investigation it looks like some npm modules don't have a license field set and they get permissive: false. I've also got some modules that are set to private: true because I haven't published them to npm yet, that get permissive: false.

This leads me to believe that if a module has permissive: false, the top level also gets permissive: false.

How should such situations be handled? Do I need to fork the project just to ignore those 2 behaviors? Should there be a flag to allow private and unlicensed modules?

@igorklopov

This comment has been minimized.

Copy link
Collaborator

igorklopov commented Jun 28, 2017

@Pupix good job! My plan was to discover packages with non-permissive setup, but you already did that. You can patch pkg and make isPermissive have return true; as it's first line. Then you will probably compile successfully. However, this way you disclose the source of your private/non-published npm packages. Probably it is not what users expect (they can prefer building from RPi host), so i hesitate introducing any such flag.

@Pupix

This comment has been minimized.

Copy link
Author

Pupix commented Jun 28, 2017

I did just that, and it's finally building on ubuntu and running on RPi without any troubles. Thanks!

@jnj16180340

This comment has been minimized.

Copy link

jnj16180340 commented Dec 5, 2017

I've enjoyed some success in cross-compiling kernel modules w/ qemu, following this guide from resin.io. I'm currently looking into what changes were made on qemu since 2015, maybe it could be a route to cross-compilation?

szaffarano added a commit to szaffarano/pkg that referenced this issue Feb 12, 2018

Add permission flag to allow private modules
Add ```--permissive``` flag that allow private and unlicensed modules be
processed when using ```--no-bytecode``` flag.  For more information
please see zeit#145

szaffarano added a commit to szaffarano/pkg that referenced this issue Feb 12, 2018

Add permission flag to allow private modules
Add --permissive flag that allow private and unlicensed modules be
processed when using --no-bytecode flag.  For more information
please see zeit#145
@jflayhart

This comment has been minimized.

Copy link

jflayhart commented Feb 23, 2018

Mine just hangs indefinitely on the make step when trying to compile on Raspberry Pi (Jessie). I don't understand, I have node installed on the pi, why does it need to get the node binaries again? Seems redundant.
screen shot 2018-02-22 at 6 50 12 pm

@jflayhart

This comment has been minimized.

Copy link

jflayhart commented Feb 23, 2018

@igorklopov, in fact, I can't even get your express example to run on the Pi:

> pkg . -o startServer --no-bytecode

> pkg@4.3.0
> Error! No available node version satisfies 'node7'
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! express-example@ dist: `pkg . -o startServer --no-bytecode`
npm ERR! Exit status 2

And then while setting target to node8+ it just hangs on the "compiling node from sources" step. I would REALLY love for pkg to work as it potentially could fulfill a very big need here, but it doesn't work right now. It compiles fine on my Mac using node8-macos target, but that doesn't help as I need an executable for armv6.

UPDATE

So when running on Pi or through Docker QEMU, it turns out the "compiling from sources" step that I thought was hanging was actually working, but it takes 22 hours to complete!! What can I do to make sure I don't have to compile everything from node.js source, since it takes 20+ hours?

@jnj16180340

This comment has been minimized.

Copy link

jnj16180340 commented Feb 28, 2018

@jflayhart For me,
pkg . --public -b -d --targets node8-linux-armv7
appeared to hang but finished ~overnight. arm64/armv8 or whatever it's called here took about the same. Absolutely no idea what the bottleneck is-- Doesn't look like cpu, ram nor disk

@jflayhart

This comment has been minimized.

Copy link

jflayhart commented Mar 1, 2018

@jnj16180340 My assumption is that the bottleneck has to do with downloading and compiling node from source, which takes time "over the wire". What makes no sense to me is, if the same version of node exists locally, why not use that instead of going to node's server to download and compile everything? I'd love to help with a PR, but I don't know where to start with binary packaging.

Is this what the --build flag is for? https://github.com/zeit/pkg#build If so, I don't understand how it should be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment