serverless-middleware incompatibility with serverless-webpack

[rezaerami]

I've been trying to use middleware with serverless-webpack and encountered some issues that almost make it impossible to combine serverless-middleware, serverless-webpack, and TypeScript.

The problem arises when placing serverless-middleware before serverless-webpack in the plugins list within serverless.yml. This causes the function's entry point to change from /path/to/invoked.handler to .middleware/functionxxx/, a clever behavior. However, a hurdle emerges: the outcome of the middleware contains imports like:

import * as src_middlewares_logger from '../src/middlewares/logger.js';
import * as src_handlers_index from '../src/handlers/index.js';
import * as src_middlewares_errorHandler from '../src/middlewares/errorHandler.js';

Later, when Webpack attempts to bundle these files, it results in an error. The cause of the error is that, unlike serverless-typescript-plugin which transpiles TypeScript to JavaScript and doesn't mind if files exist or not, Webpack expects JS files and throws a MODULE NOT FOUND error.

To address this, I need to either transpile those files using tsc to create JS files, or set up a separate Webpack build to bundle them into JS before invoking serverless-middleware. This way, when the actual serverless-webpack processes the serverless-middleware-wrapped functions, it encounters JS files and avoids the MODULE NOT FOUND error.

Another approach is to reorder the plugins, placing serverless-webpack after serverless-plugin-typescript. This resolves the issue by generating JS files, but it contradicts the initial intent of using serverless-webpack to avoid the overhead introduced by serverless-plugin-typescript.

A potential solution might involve modifying how files are imported in serverless-middleware wrapped functions, and removing the file extension to ensure compatibility with middleware placement. Additionally, if serverless-middleware could be configured to read bundled files by specifying the root, it might offer a cleaner solution.

For now, I've positioned serverless-middleware after serverless-plugin-typescript, though this creates unnecessary overhead due to middleware wrapping, building, and bundling in sequence, which doesn't seem ideal.

also, there should be a way to include middleware in webpack build, by reading middleware directory or providing them from middleware plugin to webpack cause right now my approach to include middleware in webpack is by adding them to entry object in webpack, it's not necessary to do it if serverless-middleware was mentioned after serverless-typescrypt-plugin (which is not a good practice at all)

    entry: {
      ...slsw.lib.entries,
      'src/middlewares/errorHandler': './src/middlewares/errorHandler.js',
      'src/middlewares/logger': './src/middlewares/logger.js',
    },

[juanjoDiaz]

Hi @rezaerami ,

Thanks for the extensive description.

I've used serverless-middleware together with serverless-webpack in the past without a problem.
So, either you have something misconfigured or something has changed in the latest versions breaking things.

As you have noticed, middleware works by create a single handler that imports all the middlewares and chain them into a single handler as expected by serverless. So, serverless-middlware should always be declared before serverless-webpack.

Reading at the issue, the problem here seems to be: why can't webpack find the imported modules?

A potential solution might involve modifying how files are imported in serverless-middleware wrapped functions, and removing the file extension to ensure compatibility with middleware placement.

File extensions were introduced in version 3.2 to enable support for esm modules. Node is flexible with the extensions but EcmaScript requires etensions.
Is the problem that your webpack loader doesn't support file extensions somehow?

Additionally, if serverless-middleware could be configured to read bundled files by specifying the root, it might offer a cleaner solution.

What do you mean?
serverless-middleware reads whatever is in your yaml (handlers & middlewares) and creates the wrapping function.
It doesn't matter if the

Another approach is to reorder the plugins, placing serverless-webpack after serverless-plugin-typescript.

Why are you using both plugins?
Aren't both of them essentially for the same (to transpile TS to JS)?

there should be a way to include middleware in webpack build, by reading middleware directory or providing them from middleware plugin to webpack

I don't understand this.
Webpack should be able to read files for whichever folder they are, isn't it?
Why does it matter if files are in src or .middleware?

If you create a sample project for me to replicate I can try to help more.

[rezaerami]

not sure if you have used them with TS before cause JS works fine.

The reason webpack can't find JS modules is that they are not created yet by webpack itself.
so serverless-middleware creates a file and in the file imports are pointing to JS files
Those JS files later should get created by webpack, so this is the order of workflow that webpack expects

• webpack sees the middleware ts file
• bundles the ts file with its imports together
• gives us a bundled JS file

but since those js files (imports inside wrapped function file in .middleware) are not created by webpack yet, when it reaches to middleware TS file, it tries to import them but it can't so it puts module not found in the bundled middleware file,

Is the problem that your webpack loader doesn't support file extensions somehow?

here is my webpack config

const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const path = require('path');

const webpackConfig = () => {
  console.log('beginning of webpack build', slsw.lib.entries);

  return {
    mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
    entry: {
      ...slsw.lib.entries,
      'src/middlewares/errorHandler': './src/middlewares/errorHandler.js',
      'src/middlewares/logger': './src/middlewares/logger.js',
    },
    target: 'node',
    externals: [nodeExternals()],
    resolve: {
      extensions: ['.ts', '.js'],
      alias: {
        '@': path.resolve(__dirname, 'src/'),
      },
    },
    module: {
      rules: [
        {
          test: /\.ts$/,
          use: 'ts-loader',
        },
        {
          test: /\.js$/,
          use: 'babel-loader',
        },
      ],
    },
  };
};

module.exports = webpackConfig();

What do you mean?
serverless-middleware reads whatever is in your yaml (handlers & middlewares) and creates the wrapping function.
It doesn't matter if the

let's say if I could make middleware create functions on different directories
so now if I try to di it my serverless.yl will look like this

    pre:
        - .webpack/services/path/to/middleware.foo

then the import for middleware in wrapped handler file will also look like this
import {foo} from ".webpack/services/path/to/middleware.js but the directory that wrapped function is already created in is ".webpack/services" so this import path won't be correct anymore
although this is just in case that we could put middleware after serverless-webpack, but since you said it always has to come before webpack this option is out!

I don't understand this.
Webpack should be able to read files for whichever folder they are, isn't it?

this is again in case we could put webpack before middleware which is also not an option anymore

here is the sample project:
https://github.com/rezaerami/serverless-middleware-test

make sure to run yarn
yarn

and then you can test that function with
serverless invoke local --function function1

node 18x is required

[juanjoDiaz]

Alright!

Thanks for the sample repo.

Your problem is, as expected, it related with version 3.2.0 of middleware adding the .js extension to imports and to your webpack config and haw webpack resolves modules.

You can read more about the benefits of adding the js extension to imports in Typescript here: https://www.typescriptlang.org/docs/handbook/esm-node.html
#45 is the issue in this repo that prompted me to add it.

The problem is that ts-loader doesn't natively support having imports with a .js extension.
Some plugins were created as workarounds for this. Fortunately, since webpack v5.74.0, an option called extensionAlias was added to resolve this case.

Your webpack.config.js should look like:

const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const path = require('path');

const webpackConfig = () => {
  console.log('beginning of webpack build', slsw.lib.entries);

  return {
    mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
    entry: slsw.lib.entries,
    target: 'node',
    externals: [nodeExternals()],
    resolve: {
      extensions: ['.ts', '.js'],
      // This is the key. This tells webpack to load files with js extension as ts
      extensionAlias: {
        '.js': ['.js', '.ts'],
      },
      alias: {
        '@': path.resolve(__dirname, 'src/'),
      },
    },
    module: {
      rules: [
        {
          test: /\.ts$/,
          use: 'ts-loader',
        },
        // You don't need this anymore as everything will be loaded using ts-loader.
        // {
        //   test: /\.js$/,
        //   use: 'babel-loader',
        // },
      ],
    },
  };
};

module.exports = webpackConfig();

btw, your project has some incompatible dependencies that you should probably look into:

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR! 
npm ERR! While resolving: aws-node@1.0.0
npm ERR! Found: typescript@5.2.2
npm ERR! node_modules/typescript
npm ERR!   dev typescript@"^5.2.2" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer typescript@"4.6.x || 4.7.x || 4.8.x || 4.9.x || 5.0.x || 5.1.x" from typedoc@0.24.8
npm ERR! node_modules/typedoc
npm ERR!   dev typedoc@"^0.24.8" from the root project
npm ERR!   peer typedoc@"^0.24.8" from @blazed-space/blz-typedoc-theme@0.0.2
npm ERR!   node_modules/@blazed-space/blz-typedoc-theme
npm ERR!     dev @blazed-space/blz-typedoc-theme@"^0.0.2" from the root project

[rezaerami]

cool, by adding webpack extensionAlias it has solved
thanks

but I highly suggest to add these to documentation
1- middleware must be added before webpack
2- in case we're using TS + v3.2.0 we must add alias for webpack

thanks again, I'll close the issue

ps: example repo has been removed (for users seeing this issue in future)