View engines `require` logic can fail under some conditions

[th0r]

I'm talking about this line.

I've created a repo that shows a bug.

Conditions that cause it:

1. Package manager that flattens node_modules structure should be used (npm 3 or yarn).
2. Project and some of it's deps should use unconflicting (semver-compatible) versions of express (in my repo it's webpack-bundle-analyzer)
3. Project (or some of it's deps) and this dep should have incompatible versions of the same template engine (in my repo project uses ejs 0.8.8 and webpack-bundle-analyzer uses ejs ^2.5.2)

How to reproduce a bug:

Clone this repo and run npm i && npm start or yarn && yarn run start (remember to use npm 3).

You'll see an error ENOENT: no such file or directory, open '/Volumes/Work/express-view-engines-bug/node_modules/webpack-bundle-analyzer/views/('style', { filename: 'viewer.css' })'.

This is because webpack-bundle-analyzer expects express to use ejs ^2.5.2 from it's deps as view engine with it's new syntax for include method, but instead it loads ejs 0.8.8 (project's dep) with old include syntax.

Why that happens

After npm install node_modules has the following flattened structure (condition 1):

node_modules
|- express
|- ejs 0.8.8
|- webpack-bundle-analyzer
|- |- node_modules
|- |- |- ejs 2.5.2

Note that webpack-bundle-analyzer doesn't have it's own copy of express under node_modules because it's version matches project's express dep (condition 2).

When webpack-bundle-analyzer starts express app it sets ejs as default view engine.
This line requires ejs and node loads wrong version of it from <project>/node_modules/ejs (condition 3) because express is located in <project>/node_modules/express, but it should load it from <project>/node_modules/webpack-bundle-analyzer/node_modules/ejs.

[dougwilson]

Yes, all this is known. Do you have any thoughts on changes to make to Express? Using that automatic require is optional; you can always require() your package yourself and set the engine definition for strong coupling.

[th0r]

@dougwilson Well, the problem is we should call require of the module that invoked app.set('view engine', 'ejs').

To be honest, I don't know any non-hacky way to do that.

My thoughts were to travel up by module.parents to find first non-express module and use it's require but parent property is static and points to the first module that requested it.

The other hacky option is to use stacktrace but in this case you'll get only filename of invoker, not it's module object. Although, it can be found in require.cache after that... But it looks like a mega-hack for me 😉

[dougwilson]

Sure, but AFAIK there is no good solution. I'm welcome to hear suggestions, but that entire API is just for the quick apps, and mainly only for the top level apps. Any non top level app should be using app.engine to ensure the require() is done in the correct context, which is why Express has that.