Unable to load external scoped (@) module/package

[derekseymour]

Describe the bug
Function code fails when attempting to load an external scoped module. Non-scoped external modules (such as moment) are loading fine. Not sure if this issue is related to a scoped namespace (the VM2 docs seem to indicate scoped external packages/modules are allowed), but after trying several so far none have worked.

To Reproduce

1. From the n8n server environment, install the following npm modules/packages. Tested in a Docker environment with a single worker and a main process in queue mode. Both main and worker have these modules installed globally.

npm install -g \
	@turf/turf@^6.5.0 \
	moment@^2.14.1

2. Set the following environment variables (allow all modules to ensure this isn't access related):

NODE_FUNCTION_ALLOW_BUILTIN=*
NODE_FUNCTION_ALLOW_EXTERNAL=*

3. Create a Function node with the following code:

const moment = require('moment');
const turf = require('@turf/turf');

let test = [];
test.push({
  json: {test: moment().format()}
});

return test;

Expected behavior
The code should import both modules and run without error. Commenting out the scoped @turf/turf require allows the code to run properly. Since both modules are loaded the same, the assumption is that there's a problem loading scoped modules/namespaces. A few other scoped modules (@azure) have been attempted as well and all gave the same error message.

Actual behavior

ERROR: Cannot find module '@turf/turf'

Stack
VMError: Cannot find module '@turf/turf'
    at _require (/usr/local/lib/node_modules/n8n/node_modules/vm2/lib/sandbox.js:377:25)
    at /usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Function:2:14
    at Object.<anonymous> (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Function:10:2)
    at NodeVM.run (/usr/local/lib/node_modules/n8n/node_modules/vm2/lib/main.js:1233:29)
    at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Function/Function.node.js:81:31)
    at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/src/Workflow.js:524:37)
    at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/src/WorkflowExecute.js:447:62
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Environment (please complete the following information):

• OS: Docker using Ubuntu Linux 20.04
• n8n Version: 0.152.0
• Node.js Version: 16
• Database System: MySQL 5.7
• Operation Mode: queue

Additional context
Also failed with @azure/cosmos, @azure/service-bus, and @logdna/logger. Assumption is a problem with scoped namespaces unless there's something else in common with all of these modules that would prevent n8n from loading.

[rafalkrupinski]

Did you try to explicitly list all packages?

[derekseymour]

@rafalkrupinski Tested with the following but got the same Cannot find module '@turf/turf' error:

NODE_FUNCTION_ALLOW_BUILTIN=*
NODE_FUNCTION_ALLOW_EXTERNAL=@turf/turf,moment

Just to ensure the correct env var was being used, I removed 'moment' and got a ERROR: The module 'moment' is not whitelisted in VM. Thanks for the suggestion though.

[derekseymour]

Got it fixed.

Per discussion in https://community.n8n.io/t/unable-to-load-external-scoped-module-package/9852/2

My worker container is a custom Ubuntu image which is where I'm doing my npm installs. I had assumed that was the only place the modules would be needed since that’s where the execution is happening, but apparently they needed to be installed within the main container (running the base “n8nio/n8n” Docker image) as well.

Solution: If using queue mode, make sure to install external npm modules in both the main and worker containers.