getting ERR_DLOPEN_FAILED after running re2 on alpine docker

[goingaround]

versions:
Node.js v18.17.0
alpine3.18
re2@1.19.1
pnpm@7.30.5

error log:
{"time":"2023-07-19T22:48:36.3633975Z","level":"ERROR","msg":"node:internal/modules/cjs/loader:1340","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364227625Z","level":"ERROR","msg":" return process.dlopen(module, path.toNamespacedPath(filename));","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364270875Z","level":"ERROR","msg":" ^","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364308833Z","level":"ERROR","msg":"Error: /opt/build/scanner/node_modules/.pnpm/re2@1.19.1/node_modules/re2/build/Release/re2.node:","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364337083Z","level":"ERROR","msg":" at Module._extensions..n invalid ELF headerode (node:internal/modules/cjs/loader:1340:18)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.3643545Z","level":"ERROR","msg":" at Module.load (node:internal/modules/cjs/loader:1119:32)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.3643795Z","level":"ERROR","msg":" at Module._load (node:internal/modules/cjs/loader:960:12)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364399708Z","level":"ERROR","msg":" at Module.require (node:internal/modules/cjs/loader:1143:19)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364421542Z","level":"ERROR","msg":" at require (node:internal/modules/cjs/helpers:110:18)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364466958Z","level":"ERROR","msg":" at Object.<anonymous> (/opt/build/scanner/node_modules/.pnpm/re2@1.19.1/node_modules/re2/re2.js:3:13)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364493667Z","level":"ERROR","msg":" at Module._compile (node:internal/modules/cjs/loader:1256:14)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364508667Z","level":"ERROR","msg":" at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364523875Z","level":"ERROR","msg":" at Module.load (node:internal/modules/cjs/loader:1119:32)","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364551292Z","level":"ERROR","msg":" at Module._load (node:internal/modules/cjs/loader:960:12) {","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364568708Z","level":"ERROR","msg":" code: 'ERR_DLOPEN_FAILED'","workflow":"scanner"} {"time":"2023-07-19T22:48:36.364590167Z","level":"ERROR","msg":"}","workflow":"scanner"} {"time":"2023-07-19T22:48:36.3646285Z","level":"ERROR","msg":"Node.js v18.17.0","workflow":"scanner"}

[uhop]

I use Node on Alpine to run RE2: https://github.com/uhop/node-re2/blob/master/.github/actions/linux-alpine-node-18/Dockerfile and it works as anyone can see in the Actions tab.

My guesses would be:

• pnpm could be acting out. You can search Issues for possible resolutions of pnpm-related problems. One way to rule it out is to use npm and see if the problem persists.
• Environments used to build a docker image and to run it are different. The most egregious example would be a build on an ARM-based Mac and running the result on x64-based Linux. But I had a similar problem with C++-compiled projects: an optimizer sniffs the build environment for available CPU instructions and the result fails on a CPU with a slightly different set of CPU instructions.

Please let me know if these suggestions help you in any way.

[DEVTomatoCake]

I just encountered this issue after re-installing all npm packages on Node v20.3.1.

Whenever I try to require("re2"), Node logs this error and stops working:

Error: Error relocating /home/container/node_modules/re2/build/Release/re2.node: _ZN4absl12lts_2023012518debugging_internal17AddressIsReadableEPKv: symbol not found
    at Module._extensions..node (node:internal/modules/cjs/loader:1343:18)
    at Module.load (node:internal/modules/cjs/loader:1115:32)
    at Module._load (node:internal/modules/cjs/loader:962:12)
    at Module.require (node:internal/modules/cjs/loader:1139:19)
    at require (node:internal/modules/helpers:121:18)
    at Object.<anonymous> (/home/container/node_modules/re2/re2.js:3:13)
    at Module._compile (node:internal/modules/cjs/loader:1257:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1311:10)
    at Module.load (node:internal/modules/cjs/loader:1115:32)
    at Module._load (node:internal/modules/cjs/loader:962:12) {
  code: 'ERR_DLOPEN_FAILED'
}

The re2.node file exists in the Release directory, and I've encountered it on re2 1.19.1 and 1.20.1.

[rbitanga-manticore]

Seeing the same error with:

• node v18.17.0
• alpine 3.18.3
• npm 9.6.7
• re2 1.20.1

re2 1.18.0 seems to build fine so something was introduced that's causing the errors between that version and at least 1.19.1

[leandroaguiar-lr]

I'm also seeing this error after upgrading from 1.18.0 to 1.20.1 while using the node:16.20.0-alpine3.17 base image from Docker Hub.

Reproduction is easy, just add this Dockerfile to an empty folder and run docker build . --no-cache.

FROM node:16.20.0-alpine3.17

WORKDIR /app
RUN DEVELOPMENT_SHOW_VERIFICATION_RESULTS=true npm add re2@1.20.1

[uhop]

I was able to build successfully with the following Dockerfile:

FROM node:16.20.0-alpine3.17

RUN apk add --no-cache python3 make gcc g++ linux-headers

WORKDIR /app
RUN DEVELOPMENT_SHOW_VERIFICATION_RESULTS=true npm add re2@1.20.1

I ran it on my ARM-based Mac M2 and there is no image for an ARM-based Linux. It failed to retrieve a proper cached re2.node file and didn't build because the image does not include Python 3. So I added the RUN line like in actions I use to build the binary file.

[uhop]

Searching the web I've noticed a lot of complaints about Alpine doing something from version to version. As a reference for Node 16, I used node:16-alpine, which translates at the moment of build to:

16-alpine, 16-alpine3.18, 16.20-alpine, 16.20-alpine3.18, 16.20.2-alpine, 16.20.2-alpine3.18, gallium-alpine, gallium-alpine3.18

The Dockerfile above uses node:16.20.0-alpine3.17, which is an older version all around. It could be something to play with.

[rbitanga-manticore]

With the hello world example of

require("re2");

const http = require('http');

const hostname = '127.0.0.1';
const port = 3000;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World');
});

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`);
});

and

FROM node:18-alpine
WORKDIR /app
RUN apk add --update --no-cache python3 make gcc g++ linux-headers
COPY . .
RUN npm add re2@1.20.1
EXPOSE 3000
CMD [ "node", "app.js" ]

we get

node:internal/modules/cjs/loader:1340
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: Error relocating /app/node_modules/re2/build/Release/re2.node: _ZN4absl12lts_2023012518debugging_internal17AddressIsReadableEPKv: symbol not found
    at Module._extensions..node (node:internal/modules/cjs/loader:1340:18)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:121:18)
    at Object.<anonymous> (/app/node_modules/re2/re2.js:3:13)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12) {
  code: 'ERR_DLOPEN_FAILED'
}

With npm add re2@1.20.0 instead we get a failure to add re2

npm ERR! In file included from ../lib/./wrapped_re2.h:6,
npm ERR!                  from ../lib/addon.cc:1:
npm ERR! ../vendor/re2/re2/re2.h:217:10: fatal error: absl/base/call_once.h: No such file or directory
npm ERR!   217 | #include "absl/base/call_once.h"
npm ERR!       |          ^~~~~~~~~~~~~~~~~~~~~~~
npm ERR! compilation terminated.
npm ERR! make: *** [re2.target.mk:231: Release/obj.target/re2/lib/addon.o] Error 1
npm ERR! gyp ERR! build error
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/usr/src/app/node_modules/node-gyp/lib/build.js:203:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:514:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Linux 5.15.90.1-microsoft-standard-WSL2
npm ERR! gyp ERR! command "/usr/local/bin/node" "/usr/src/app/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /usr/src/app/node_modules/re2
npm ERR! gyp ERR! node -v v18.17.1
npm ERR! gyp ERR! node-gyp -v v9.4.0
npm ERR! gyp ERR! not ok

With npm add re2@1.19.2 we get the expected

Server running at http://127.0.0.1:3000/

[rbitanga-manticore]

The symbol is obviously from absl and it looks like it's because you pull in vendor/abseil-cpp/absl/debugging/stacktrace.cc which includes stacktrace_x86-inl.inc which in turn includes address_is_readable.h but you don't include address_is_readable.cc as a target source in binding.gyp

[leandroaguiar-lr]

@uhop yeah, installing the native compilation tools gets the Docker image built because of the fallback.

But as @rbitanga-manticore pointed it still fails during runtime with the missing symbol error.

[jcurlier]

+1

[uhop]

The symbol is obviously from absl and it looks like it's because you pull in vendor/abseil-cpp/absl/debugging/stacktrace.cc which includes stacktrace_x86-inl.inc which in turn includes address_is_readable.h but you don't include address_is_readable.cc as a target source in binding.gyp

google/re2 has decided to include dependencies, namely absl. This change landed as an external patch in 1.20.0, greatly expanding this project's compiled files. Apparently, some of them were missed. I appreciate actionable information like this, so I can correct the issue. Thank you.

[uhop]

Released on NPM as 1.20.2. Please test.

[zotanmew]

This is still broken in v1.20.5 on alpine-arm, though the error message has changed.

node:internal/modules/cjs/loader:1340
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^
Error: Error relocating /app/node_modules/re2/build/Release/re2.node: _ZN4absl12lts_2023080218debugging_internal11VDSOSupportC1Ev: symbol not found
    at Module._extensions..node (node:internal/modules/cjs/loader:1340:18)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/app/node_modules/re2/re2.js:3:13)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12) {
  code: 'ERR_DLOPEN_FAILED'
}

Node.js v18.18.2

This is reproducable with the app.js and Dockerfile from #180 (comment), simply by changing the version to 1.20.5

[zotanmew]

I've managed to fix this issue by adding the following targets to binding.gyp:

        "vendor/abseil-cpp/absl/debugging/internal/elf_mem_image.cc",
        "vendor/abseil-cpp/absl/debugging/internal/vdso_support.cc",

I'll submit a PR in a minute.

[zotanmew]

Done, #195