Production build memory consumption

[ntimo]

Hey,
I just wanted to try to use codedoc but for some reason the build always fails for me.

Steps to reproduce:

• docker run --rm -it node:lts bash
• cd /opt/
• npm i -g @codedoc/cli
• codedoc init
• codedoc build

build output:

root@0cf095635bee:/opt# codedoc build

# Building ...
# Fetching project configuration ...
# building ........ docs/md/index.md
# built:: ......... index.html
# building ........ docs/md/docs/cheat-sheet.md
# built:: ......... docs/cheat-sheet.html
# building ........ docs/md/docs/code-features.md
# built:: ......... docs/code-features.html
# building ........ docs/assets/codedoc-styles.css
# built:: ......... docs/assets/codedoc-styles.css
# building ........ docs/assets/codedoc-bundle.js
# Build Failed.

root@0cf095635bee:/home/test# 

[loreanvictor]

Ok so there seems to be an issue with webpack over-consuming memory while building in production mode. I will check CODEDOC's webpack configuration to see if there is room for optimizing anything via configs, but in the meanwhile I would recommend building on local machine instead of inside a container, or increasing memory limits of the container. To verify this, you can simply run codedoc serve instead of codedoc build and check if its working.

[loreanvictor]

So this is the webpack config that results in OOM:

{
  "resolve": {
    "extensions": [
      ".tsx",
      ".ts",
      ".js"
    ],
    "modules": [
      ".codedoc/node_modules"
    ]
  },
  "resolveLoader": {
    "modules": [
      "node_modules",
      ".codedoc/node_modules"
    ]
  },
  "entry": "/opt/docs/assets/codedoc-bundle.entry.js",
  "output": {
    "filename": "codedoc-bundle.js",
    "path": "/opt/docs/assets"
  },
  "module": {
    "rules": [
      {
        "test": {},
        "use": "ts-loader",
        "exclude": {}
      }
    ]
  }
}

The only work-around right now seems to be turning off type checking on ts-loader. Since this is limited to building in production mode in memory-limited environments, I can recommend one of the following:

• Do not build inside a container with memory limits. You can safely serve inside a container, using a Dockerfile like this and a compose file like this, for example.
• Increase memory limit of your container
• If you, for some reason, have no other options but to build inside a memory-limited container, change your build config in .codedoc/build.ts so that it builds in development mode:

import { build } from '@codedoc/core';

import { config } from './config';
import { installTheme$ } from './content/theme';
import { content } from './content';


build(config, content, installTheme$, {
  mode: 'development',        // --> this line is added
  resolve: {
    modules: ['.codedoc/node_modules']
  },
  resolveLoader: {
    modules: ['.codedoc/node_modules']
  }
});

Note that this is really not recommended as it results in bundles that are not minified properly. Generally speaking though, you should not need to build manually except for deployment, where you typically would not have such strict memory limitations.

Closing this issue since it does not seem the webpack / ts-loader team intend to tend to this anytime soon (unfortunately).

[TysonMN]

there seems to be an issue with webpack over-consuming memory while building in production mode.

How did you figure that out? Did you follow the reproduction steps and reproduce the problem yourself?

As a minimal improvement, how about adding output to codedoc build that something (or specifically webpack) ran out of memory?

[loreanvictor]

How did you figure that out? Did you follow the reproduction steps and reproduce the problem yourself?

Naturally, also testing with different webpack configurations.

As a minimal improvement, how about adding output to codedoc build that something (or specifically webpack) ran out of memory?

I'm not sure how feasible this is, as the node process itself basically runs out of memory and crashes (webpack is executed on the same node process).

[TysonMN]

My previous project at work had a memory issue. It was a .NET application (written in C#). When a request for memory is larger than the available memory, an OutOfMemoryException is thrown. I caught this exception and decided to force kill the application by calling Environment.Exit(-1) but not before logging that the application would be terminated due to lack of memory. Additionally, I can keep tabs on how things are going by subscribing to both AppDomain.CurrentDomain.UnhandledException and AppDomain.CurrentDomain.FirstChanceException and then logging if either occur.

Does any of this give you an idea for how you can give more feedback when webpack runs out of memory?

The simplest improvement would be to log before and after webpack runs. Then users would know that the issue is with webpack.

[loreanvictor]

The simplest improvement would be to log before and after webpack runs. Then users would know that the issue is with webpack.

It does already:

# building ........ docs/assets/codedoc-bundle.js
# Build Failed.

And I'm not sure about @ntimo , but I do get more detailed error messages when following reproduction steps here as well.