Native C++ module filenames should include the platform in the name #2414

Closed
dobesv opened this Issue Dec 25, 2011 · 8 comments

Projects

None yet

4 participants

@dobesv
dobesv commented Dec 25, 2011

When building native modules from C++, putting the platform in the name would allow the same module folder to be used for multiple platforms.

So instead of just contextify.node the file would be named contextify-win32-ia32 for a windows machine and contextify-linux2-ia32 for a linux machine.

dobesv commented Dec 25, 2011

To avoid increasing the number of file extensions that node searches you could output an intermediate file for the binary modules, such as a .js or .json file, that specifies what actual file to load for a given platform.

See also #2136 (comment)

Mithgol commented Dec 25, 2011

If require('./modulename') meant different filenames on different systems (say, modulename.so on Linux, modulename.dll on Windows, modulename.dylib on Mac OS), then the module's author could simply provide all the three pre-compiled modules in the same directory and consider the job done.

Currently the name is the same (modulename.node) for every system, but the pre-compiled modules still need to be distributed somehow, and thus the task of picking the correct module may be seen as the task for npm (such as issue npm/npm#1891, currently open) or some other package manager.

(For example, as @Benvie has said at node-ffi/node-ffi#30, providing some compiled binaries is the usual mode of things for Windows. However, as @kkaefer has said at mapbox/node-sqlite3#48, there's no clear blessed path to distribute such binaries via npm.)

As I see it, platform-specific names are better because the work is done at the Node.js core and does not make any package manager a necessity. Such names would be very useful in development (when the module is not even packaged anyway). Simple installation schemes could also proliferate (such as: «grab module.zip, unpack it, and everything just works on any system») whenever the module's author does not feel like dealing with npm or any other package registry.

For example, since at least November 2011 node-firebird-libfbclient provides a pre-compiled (and zip-packed) Windows package:

I believe that any such zip archive could already be made somewhat cross-platform if its author did not have to make a separate node_modules/firebird/build/Release/binding.node file for each of the systems. (I mean, if two or three system-specific files could be packed there for each Node.js binary module.)

Mithgol commented Dec 25, 2011

Another interesting example is how extensions for Firefox are made and packaged when they are JavaScript code around binary files.

Unpack ColorZilla's XPI package (which is a renamed ZIP archive), and you'll find five binaries there:

  • platform/Darwin/components/ColorZilla.dylib
  • platform/Linux/components/ColorZilla.so
  • platform/Linux/components/ColorZilla.gcc4.so
  • platform/Linux/components/ColorZilla.x86_64.so
  • platform/WINNT/components/ColorZilla.dll

As you may see, their names are platform-specific.

All five binaries are installed in the extension's folder. (The necessary file is picked by Firefox, not by extension manager.)

It seems that Firefox does already follow the scheme similar to what was suggested for Node.js by @dobesv above.

dobesv commented Dec 25, 2011

My concern about using only the suffix (.dll, .dylib, or .so) is that it
won't distinguish the CPU in use. Linux isn't the only platform using .so
and even within linux the version of glibc, the version of node, and the
CPU may vary as well.

The native extensions may need to have the (major) version of node in the
name as well, in case you have built separate binaries for, say, node 0.4
and node 0.6.

On Sun, Dec 25, 2011 at 6:21 PM, Mithgol <
reply@reply.github.com

wrote:

Another interesting example is how extensions for Firefox are made
and packaged when they are JavaScript code around binary files.

Unpack ColorZilla's XPI package
(which is a renamed ZIP archive), and you'll find five binaries:

  • platform/Darwin/components/ColorZilla.dylib
  • platform/Linux/components/ColorZilla.so
  • platform/Linux/components/ColorZilla.gcc4.so
  • platform/Linux/components/ColorZilla.x86_64.so
  • platform/WINNT/components/ColorZilla.dll

As you may see, their names are platform-specific.

All five binaries are installed in the extension's folder.
(The necessary file is picked by Firefox, not by extension manager.)

It seems that Firefox does already follow the scheme similar to what was
suggested for Node.js by @dobesv above.


Reply to this email directly or view it on GitHub:
#2414 (comment)

Mithgol commented Dec 26, 2011

Good point, the devs should consider that.

Mozilla Firefox (which I already used as an example above with ColorZilla) allow for its extensions to have a special manifest file where the necessary binary variants of the component are enumerated:

binary-component platform/WINNT/components/ColorZilla.dll ABI=WINNT_x86-msvc
binary-component platform/Darwin/components/ColorZilla.dylib ABI=Darwin_x86-gcc3
binary-component platform/Darwin/components/ColorZilla.dylib ABI=Darwin_ppc-gcc3
binary-component platform/Linux/components/ColorZilla.so ABI=Linux_x86-gcc3
binary-component platform/Linux/components/ColorZilla.gcc4.so ABI=Linux_x86-gcc3
binary-component platform/Linux/components/ColorZilla.x86_64.so ABI=Linux_x86_64-gcc3

ABI there means «application binary interface», and they have quite a lot of those: 18 platform identifiers + six separate suffixes for CPU types + six separate suffixes for compilers.

That's not a pure name-based approach: the extension's developer may give any name to the binary as long as that name is listed inside the manifest file.

The list of possible suffixes for Node.js, most likely, won't be anywhere near that long, and thus the module naming could rely purely on suffixes, not on some manifest (JSON) parsing.

Mithgol commented Jan 28, 2012

I believe that lib/bindings.js in node-ffi is a good example of a work around this issue.

Owner
rvagg commented Nov 14, 2014

👎 from me, partly because this would lead to feature-creep with people wanting to put the NODE_MODULE_VERSION, then the NODE_VERSION_STRING, then system information like glibc version (cause this matters) and perhaps even versions of other dependencies. There are too many variables involved here.

Closing this as wontfix for now, for the reasons @rvagg outlined, as well as the relative lack of movement on this issue over the last 3 years.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment