[RFE] unbundled / preserveModules support

[akvadrako]

I quite like the design and speed of esbuild, but I found that when I use it with bundle: false it doesn't do what I would expect. The dependencies of entrypoints are not built unless they are also listed as entrypoints. Even if the deps are listed as entrypoints, the import paths are not adjusted so they won't work.

So it would be nice if esbuild bundle: false worked like rollup's preserveModules.

[graup]

Seconding this. Especially for library modules (e.g. component libraries) it seems unnecessary to bundle everything into one file. I wonder what the use case for the current implementation of bundle: false is. The docs don't talk much about it either.

[beeequeue]

With this libraries that need tree-shaking would be able to use esbuild directly, right now you have to run it through either rollup or webpack to not have one big bundle.

[charlie632]

Just commenting as this would be a great addition to esbuild

[graup]

Hi @evanw, is there anything we can do to help with this?

Another use case is building design systems with Vanilla Extract. Currently, all my CSS is bundled into one file. I'd much prefer if each component would be retained as its own file so that the CSS can be separated more easily. Then when you import a tree-shaken part of the library, you don't import unneeded CSS.

[hyrious]

@graup Tree-shaking has little relation with code splitting. I even doubt whether it's possible to apply vanilla-extract on thrid-party libraries.

• Tree-shaking means "remove unused tree nodes". If you think of the js code as an AST, this process is like shaking the tree to remove nodes that are not attached on it. It has only one relation in ESM that you can mark an import statement as no-side-effect, so that this statement can be removed if no code is using it.
  In other words, if all your code is "tree-shakeable", there's no need to mark side effect on statements.

• vanilla-extract works as a bundler plugin to replace ".css.(ts|mjs|js)" files with corresponding js and css. i.e. It actually needs to see these files in your sources. However, vite itself has a pre-bundling process that bundles all third-party libraries into single files, which means it actually forbids plugins to work on files in node_modules. Thus I think we'd better not exporting something needing plugins to work.

[graup]

@hyrious Thanks for your reply. I don't want to apply vanilla-extract to third-party libraries. I want my library to build all its ".css.ts" files to corresponding js and css, but avoid merging all css into one file so that in the app I can import only the css that's needed for the components that are actually imported. I was under the impression that esbuild bundling is what combines all css into one, which is why I thought preserveModules might help, but I may be wrong. There's probably a way to solve this with a custom plugin. Anyway, this is a discussion for vanilla-extract: vanilla-extract-css/vanilla-extract#620

[jacobrask]

I tried some different configurations in our component library by checking what a typical app would do with the output. Bundling everything to a single file apparently prevents a default Next.js 12 app from tree shaking unused code. The following setup works fine though. Using it through tsup so can't paste a raw esbuild config but it should be the same.

• One main entry that exports everything
• Separate entries for every component (passing glob.sync('src/components/*/index.tsx') to entries)
• Code splitting enabled

[charlag]

I will explain our use case here, perhaps it helps maintainers.
At the moment we are using Rollup for release builds and Nollup for debug builds. Nollup doesn't work that well for us and we wish to move to esbuild for debug builds. We have few things that complicate migrating to esbuild at the moment:

• we need files to be compiled eagerly because we build electron/mobile apps so vite.js doesn't work for us
• we try to check initialization order at runtime. e.g. if you run some code that should not be run during bootstrap we then assertion will fail
• we check chunk dependency at compile time so that heavy chunks are not accidentally imported from the wrong places

Exposing module graph post-build would allow us to solve all of that by invoking the compiler and our checks but also having an unbundled build will help us with the first two points

[nikolay-govorov]

In April, Snowpack was deprecated, and now there is no such possibility almost anywhere. This is the only feature we're missing in esbuild in order to migrate. It seems that even the simple possibility of giving the tree of resulting chunks before merging would solve most cases.

[Conaclos]

@akvadrako

Even if the deps are listed as entrypoints, the import paths are not adjusted so they won't work.

Could you clarify this?

[akvadrako]

@Conaclos It's been a while, but I think I meant that even if one were to list every source file as an entrypoint, causing esbuild to create one output file per input file, it wouldn't produce usable output, since the import paths wouldn't match the output directory layout.

[jacobrask]

@Conaclos It's been a while, but I think I meant that even if one were to list every source file as an entrypoint, causing esbuild to create one output file per input file, it wouldn't produce usable output, since the import paths wouldn't match the output directory layout.

This can now be done with package.json exports and self-referencing, e.g.

// src/button-group.js, esbuild entry point
import { Button } from 'design-system/button.js';

// src/button.js, esbuild entry point
export const Button

// package.json
{
  "name": "design-system",
  "exports": {
    "./button.js": "./dist/button.js",
    "./button-group.js": "./dist/button-group.js",
    ...
  }

It's a reasonable practice anyway - do you really want to expose all your internal util files to the world? This keeps the public API explicit

[Conaclos]

@akvadrako
If you use relative imports and you compile all files at once, esbuild preserves the layout. Otherwise you can use the option source-root to have more control on the layout of the output.

I am using esbuild to transpile my TypeScript projects. I basically run the following command to compile my source files:

esbuild --outdir=dist src/*/*.ts src/*.ts

If I use a tests directory, then all test files import the code to test via a self package import:

// test file
import * as myPackage from "my-package"

By the way, it could be nice to have some option to build all files of a project. Something like:

esbuild --build-imports src/index.ts

[nikolay-govorov]

Unfortunately, if you simply pass all files as entrypoints, then in the case of TS, for example, aliases will not work.

Another problem is that dependencies from node_modules will not be glued together (I would like them to be in separate chunks).

[Conaclos]

if you simply pass all files as entrypoints, then in the case of TS, for example, aliases will not work.

You mean using tsconfig's paths?

Another problem is that dependencies from node_modules will not be glued together (I would like them to be in separate chunks).

I am not sure to follow you there. You want to compile individually every source file, except node_modules dependencies that are bundled together?

[btopro]

Why I'm monitoring this thread:

• Keep tree structure
• resolve node_modules dependencies
• copy matching glob structure to dist folder + the matching node_modules resolutions
• compile the results in place so the tree structure is intentionally not shaken

Not sure this is possible in esbuild, but it is how our polymer cli based implementation currently works that we ship to CDNs

https://github.com/elmsln/HAXcms/tree/master/build/es6/node_modules

[nikolay-govorov]

Yes, this is exactly the result I want to get.

I designed a solution with this result, but it works slowly and unstable (for example, tracking changes):

• I run the assembly of input points (bundle: true)
• in onresolve, I intercept all dependencies, accumulate them in the queue
• while there are items in the queue, to repeat this cycle for items.

As a result, I get the original structure, both for my sources and for node_modules. Nevertheless, it seems that it is better to support such functionality in the collector than to pervert with your own queue.

[coryvirok]

Unfortunately, if you simply pass all files as entrypoints, then in the case of TS, for example, aliases will not work.

I solve this with a small plugin that uses tsc-alias to fix path aliases in my transpiled (from TS) code:

import esbuild from 'esbuild'

/** @type (any) => esbuild.Plugin */
const tscAliasPlugin = () => {
  return {
    name: 'tsc-alias',
    setup(build) {
      build.onEnd(async (_result) => {
        await replaceTscAliasPaths({ outDir: './dist', resolveFullPaths: true })
      })
    },
  }
}

[zwcloud]

This is a critical issue in my case for esnext. This makes js and css file versioning impossible, if you only use esbuild to force browser to invalidate js and css cache after changed js and css source files.

[arvindell]

This is keeping my team from migrating to the much-faster esbuild for building our UI library.

We currently use rollup because of the preserveModules option, which tells next.js knows how to treeshake properly. This feature would be a great improvement to the DX of thousands of developers.

[elramus]

@arvindell There is a rollup plugin to use esbuild for compilation under the hood, and then you can still use rollup's preserveModules. Seems like the best of both worlds.

[arvindell]

@arvindell There is a rollup plugin to use esbuild for compilation under the hood, and then you can still use rollup's preserveModules. Seems like the best of both worlds.

Thank you so much, it works like a charm @elramus

[brianjenkins94]

I designed a solution with this result, but it works slowly and unstable (for example, tracking changes):

@nikolay-govorov can you share your plugin/workaround?

[dcecile]

Here's my version of the onResolve-to-queue algorithm:

import * as esbuild from 'esbuild'
import * as nodePath from 'node:path'

export interface BuildOptions
  extends Omit<
    esbuild.BuildOptions,
    | 'metafile'
    | 'mangleCache'
    | 'entryPoints'
    | 'stdin'
    | 'bundle'
    | 'outbase'
    | 'outExtensions'
  > {
  entryPoints: string[]
  outbase: string
}

export interface BuildResult<
  ProvidedOptions extends BuildOptions = BuildOptions,
> extends Omit<
    esbuild.BuildResult<ProvidedOptions>,
    'metafile' | 'mangleCache' | 'outputFiles'
  > {
  outputFiles: esbuild.OutputFile[]
}

export async function build<T extends BuildOptions>(
  options: esbuild.SameShape<BuildOptions, T>
): Promise<BuildResult<T>> {
  const result: BuildResult<T> = {
    errors: [],
    warnings: [],
    outputFiles: [],
  }
  const allEntryPoints = new Set(options.entryPoints)
  let entryPoints = options.entryPoints.map(
    (entryPoint) => ({
      in: entryPoint,
      out: nodePath.relative(options.outbase, entryPoint),
    })
  )
  while (entryPoints.length) {
    const newEntryPoints: { in: string; out: string }[] = []
    const plugin: esbuild.Plugin = {
      name: 'buildModules',
      setup(build) {
        build.onResolve({ filter: /.*/ }, async (args) => {
          if (args.pluginData === true) {
            return undefined
          }
          const resolveResult = await build.resolve(
            args.path,
            {
              importer: args.importer,
              namespace: args.namespace,
              resolveDir: args.resolveDir,
              kind: args.kind,
              pluginData: true,
            }
          )
          if (
            !resolveResult.errors.length &&
            !resolveResult.external &&
            ['import-statement', 'dynamic-import'].includes(
              args.kind
            )
          ) {
            if (!allEntryPoints.has(resolveResult.path)) {
              newEntryPoints.push({
                in: resolveResult.path,
                out: nodePath.relative(
                  options.outbase,
                  resolveResult.path
                ),
              })
              allEntryPoints.add(resolveResult.path)
            }
            const relativePath = `${nodePath.relative(
              nodePath.dirname(args.importer),
              resolveResult.path
            )}.mjs`
            return {
              ...resolveResult,
              path: relativePath.startsWith('.')
                ? relativePath
                : `./${relativePath}`,
              namespace: 'buildModules',
              external: true,
            }
          } else {
            return resolveResult
          }
        })
      },
    }
    const moduleResult = await esbuild.build({
      ...options,
      bundle: true,
      entryPoints,
      outExtension: { ['.js']: '.mjs' },
      plugins: [...(options.plugins ?? []), plugin],
    })
    result.errors.push(...moduleResult.errors)
    result.warnings.push(...moduleResult.warnings)
    result.outputFiles.push(
      ...(moduleResult.outputFiles ?? [])
    )
    entryPoints = newEntryPoints
  }
  return result
}

[eatsjobs]

Generate an entry point for each file and then set bundle:true would do what expected?
I mean keeping each file separated with the imports but transformed?

example

import glob from 'glob';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

esbuild.build({
	entryPoints: Object.fromEntries(
		glob.sync('src/**/*.ts').map(file => [
			// This remove `src/` as well as the file extension from each
			// file, so e.g. src/nested/foo.js becomes nested/foo
			path.relative(
				'src',
				file.slice(0, file.length - path.extname(file).length)
			),
			// This expands the relative paths to absolute paths, so e.g.
			// src/nested/foo becomes /project/src/nested/foo.js
			fileURLToPath(new URL(file, import.meta.url))
		])
	),
		format: 'es',
		outDir: 'dist’,
                loader:{ 
                     ".svg":"dataurl"
                }
}); 

Update:

I managed to have something close of what I want(having a treeshakable output) with this configuration.

esbuild.build({
    entryPoints: ['src/main/**/*.ts'],
    entryNames: '[dir]/[name]',
    outbase: "src/main",
    splitting: true,
    bundle: true,
    minify: false,
    minifyIdentifiers: true,
    minifySyntax: true,
    minifyWhitespace: false, // this is important otherwise we're gonna loose PURE annotations
    outdir: OUT_DIR_ESM,
})

Problem is that in a lot of files I have some bar imports like this. Since I have sideEffects: false in my package.json, when I consume the library the ignore-bare-import warning appears but everything works as expected.
My question is: why there are these bare imports if they're not used?
Any clue(thanks in advance)? @evanw

`
import {
a
} from "./chunks/chunk-ISMNMLQH.js";

import "./chunks/chunk-FQDWJHHW.js";
import "./chunks/chunk-DVOYNPVA.js";
import "./chunks/chunk-T4T43I6T.js";
import "./chunks/chunk-TNPMOBA2.js";
import "./chunks/chunk-YS5VQUVZ.js";
import "./chunks/chunk-SKGJYV3P.js";

export {
a as Finder
};
`

[charlag]

Here's my version of the onResolve-to-queue algorithm:

thanks for the code @dcecile !

from my testing:

• it does not work with named entry points (but it is easy to fix)
• it needs outbase and it didn't work with it correctly when I tried otherwise but it could have been my fault too
• most importantly, it is about 4 times slower than just running with bundle: true

We are once again running into issues with this because dynamic imports get rolled up into the bundle that starts not as es module (a worker).

We would like to once again bring it to the attention of the maintainers that without both unbundled output and splitting it is very hard to use esbuild

[antl3x]

I accomplish to maintain files/folder structure with a glob pattern on entryPoints

 await build({
    entryPoints: ["src/**/*.ts"],
    bundle: false,
    ...
    });