Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support 64 bit NodeJS executable #158

Open
mhoeger opened this issue Dec 7, 2018 · 14 comments

Comments

Projects
None yet
6 participants
@mhoeger
Copy link
Collaborator

commented Dec 7, 2018

From @AshleyGrant on November 19, 2018 17:15

Switching to the 64 bit runtime does not switch to a 64 bit version of Node. Unfortunately, this causes issues as there are some NPM packages that only support running on 64 bit Node. One in particular is "sharp." This is an image manipulation library that is orders of magnitude faster than alternatives, but only supports running on 64 bit Node due to it calling to DLLs that are compiled to run on x64.

As it currently stands, Node developers have no way to utilize 64 bit Node on a consumption plan. Switching to the 64 bit Node executable when running the function app in 64 bit mode would solve this problem.

Copied from original issue: Azure/Azure-Functions#1041

We cannot deploy 64 bit nodejs and make breaking changes in underlying infrastructure of this behavior, especially because app service customers will be affected. However, we can introduce this change with the introduction of ~ targets for WEBSITE_NODE_DEFAULT_VERSION.

Involves:

  • deploy 64 bit nodejs version for each major version: #201
  • update dwas logic to search for 64-bit version if ~ targeted by WEBSITE_NODE_DEFAULT_VERSION: Azure/Azure-Functions#1137
@alexisbg

This comment has been minimized.

Copy link

commented Feb 17, 2019

It is really boring. I need to find an alternative to Azure Functions. Is there any update about this issue?

@mhoeger

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 19, 2019

Sorry for the delay. We don't fully support a 64 bit Node.js executable. However, we do have a workaround to "bring your own node executable" and then point to it.

Bringing your own node.exe

  • In Kudu, which will be at [sitename].scm.azurewebsites.net, go to Debug Console.
  • Drop the node.exe you want to use in D:\home (can be in root, can be nested)
  • Take note of the path (ex: D:\home\node)

Pointing to a custom node.exe

  • Add the AppSetting languageWorkers:node:defaultExecutablePath and set it to the path (from before) to your node.exe (ex: D:\home\node)

Again, this is a not ideal workaround, but let me know if you are not unblocked by this.

@alexisbg

This comment has been minimized.

Copy link

commented Feb 20, 2019

Thank you for your answer. It is also required to switch Platform to 64-bit in Application Settings for this to work.

@ChrisSmith5

This comment has been minimized.

Copy link

commented Feb 22, 2019

@mhoeger I followed your instructions and loaded from a x64 node.exe. However, I am still getting the following error when loading the Sharp node module:

Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.postProcessMedia ---> Microsoft.Azure.WebJobs.Script.Rpc.RpcException: Result: Failure
Exception: Worker was unable to load function postProcessMedia: 'Error: \?\D:\home\site\wwwroot\postProcessMedia\node_modules\sharp\build\Release\sharp.node is not a valid Win32 application.
\?\D:\home\site\wwwroot\postProcessMedia\node_modules\sharp\build\Release\sharp.node'
Stack: Error: \?\D:\home\site\wwwroot\postProcessMedia\node_modules\sharp\build\Release\sharp.node is not a valid Win32 application.
\?\D:\home\site\wwwroot\postProcessMedia\node_modules\sharp\build\Release\sharp.node
at Object.Module._extensions..node (module.js:681:18)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at bindings (D:\home\site\wwwroot\postProcessMedia\node_modules\bindings\bindings.js:112:48)
at Object. (D:\home\site\wwwroot\postProcessMedia\node_modules\sharp\lib\constructor.js:10:34)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object. (D:\home\site\wwwroot\postProcessMedia\node_modules\sharp\lib\index.js:3:15)
at Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\Description\Rpc\WorkerLanguageInvoker.cs:line 74
at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\Description\FunctionInvokerBase.cs:line 84
at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task1 src) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\Description\FunctionGenerator.cs:line 225 at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker2.InvokeAsync(Object instance, Object[] arguments) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:line 63
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 556
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstance instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 503
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstance instance, ParameterHelper parameterHelper, IFunctionOutputDefinition outputDefinition, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 439
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstance instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 249
--- End of inner exception stack trace ---
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstance instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 293

@alexisbg

This comment has been minimized.

Copy link

commented Feb 22, 2019

\D:\home\site\wwwroot\postProcessMedia\node_modules\sharp\build\Release\sharp.node is not a valid Win32 application.
Are you sure your function runs on the 64-bit node.exe?
If you print a process.arch in your code, do you see 'x64' and not 'ia32'?

@AshleyGrant

This comment has been minimized.

Copy link

commented Feb 28, 2019

I was not able to get past running npm install It doesn't seem to use the 64 bit version of Node when running npm install during deployment. This stops me from successfully deploying.

@alexisbg

This comment has been minimized.

Copy link

commented Mar 1, 2019

Sadly while Git/CI npm install, a deprecated Node.js 6.x is called instead of your 64-bit node.exe.

The only way is to upload a ZIP file containing your compiled prod dependencies to a blob container. This can be achieved with WEBSITE_RUN_FROM_PACKAGE app setting together with "bundledDependencies" and "files" properties in your package.json.

@mhoeger mhoeger closed this Mar 1, 2019

@mhoeger mhoeger reopened this Mar 1, 2019

@mhoeger

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 1, 2019

@ahmelsayed - I'm guessing that choosing the bitness of node.exe will be much easier to support on the Linux offering. Is that right?

@AshleyGrant

This comment has been minimized.

Copy link

commented Mar 5, 2019

I have to say this is extremely frustrating. IMHO, FaaS offerings should do their best to abstract away the underlying infrastructure. This issue lays bare the underlying infrastructure that Azure Functions is built on. Frankly, the underlying OS shouldn't matter to a developer building on a FaaS platform.

I find it hard to understand how Azure Functions, a product that was first announced less than 3 years ago, would have an architecture that makes implementing support for a 64 bit platform so difficult. It's not like x64 support was yet to be implemented in Node for Windows. It has been supported since at least v4.0.0 released in September of 2015.

Why is changing the bitness of the .NET runtime a simple toggle switch in the UI, but changing which NodeJS executable is used a Herculean effort? Is it not possible to have a different VM image to use that has Node x64 installed?

And frankly, I would venture to guess that the vast majority of Windows-based developers are developing on machines that run Node x64, given that it is the default download on the NodeJS website:
image

So for Azure Functions to run the 32 bit version of Node is a real problem for Node developers who run windows and deploy to Azure functions.

I look forward to discussing this issue at the MVP Summit in a few weeks @eduardolaureano.

@mhoeger

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2019

@AshleyGrant - I am sorry about the frustrating experience. I think you are primarily asking for all functions to run on a 64 bit host and a 64 bit node.exe by default. We have changed the default on the Linux offering to run on 64 bit node (Linux consumption apps are currently in preview). It's a bit trickier for the Windows offering. To provide some context, the changes we made to have JavaScript code run directly on a Node.js process instead of through the Edge.js .NET library were pretty recent. With this, we've uncovered some gaps in the underlying infrastructure such as a lack of deployed 64 bit node executables on worker machines. Azure Functions relies on spare compute from App Service, and this is an App Service limitation that we have run into. I've started talking to @paulbatum about finding the appropriate owner to update the App Service machines with 64 bit node versions. It's a more niche scenario so they haven't heard all too much noise about it, but it is a change that's long overdue. Although our goal is abstraction, as you've said, I do hope you understand that getting to that point can require changes that have a broad scope and impacts beyond the immediate situation. The toggle not applying to node.exe is absolutely a gap in expectation / experience, and we appreciate your patience as we fix that.

@AshleyGrant

This comment has been minimized.

Copy link

commented Mar 6, 2019

I'm not necessarily asking for all functions to run on a 64 bit host w/64 bit Node by default. I do think that 64 bit Node should be used when running on a 64 bit host, though.

Thank you for the explanation of the underlying constraints. Hopefully this can be resolved sooner rather than later. Let's please keep this issue open until then, though.

@zavr-1

This comment has been minimized.

Copy link

commented Mar 13, 2019

@AshleyGrant @ChrisSmith5 when using git deployment, you can install sharp by setting npm_config_arch = x64 to allow that old npm to install x64 dependencies, and then use the solution by @mhoeger to run x64 node env.

@AshleyGrant

This comment has been minimized.

Copy link

commented Apr 4, 2019

I've come up with a workaround that allows me to bring a custom Node executable along for the ride when I "build" my application using Azure Devops. I'm mostly using the strategy specified by @mhoeger, but distributing node.exe as part of my application zip file. Details here: https://aurelia.ninja/2019/04/04/a-strategy-for-deploying-a-custom-node-executable-to-azure-functions-on-windows/

@AshleyGrant

This comment has been minimized.

Copy link

commented Apr 4, 2019

@zavr-1 I've confirmed that your solution works, but I figure if I'm going far enough to include my own node executable in my demo code, I might as well use a CI/CD tool (Azure Devops in my case) to orchestrate the build. My solution also eliminates the need for manually copying node.exe. In a pinch, I would use your solution, though!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.