Proposal for enhancing ways to extend tsconfig file

[sheetalkamat]

🔍 Search Terms

"extends", "merge", "compilerOptions", "paths", "outdir", "include", "exclude", "typeRoots", "references", "tsconfig", "ability to merge tsconfig", "ability to make tsconfig paths relative to final config"

✅ Viability Checklist

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This isn't a request to add a new utility type: https://github.com/microsoft/TypeScript/wiki/No-New-Utility-Types
• This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion

Today when you extend a config the options written in the final config override the options from the base config. Here is sample of what happens today:

// @fileName: /temp/test/base.tsconfig.json
{
    "compilerOptions": {
        "paths": {
            "@libs/*": ["libs/*"],
            "@project1/*": ["project1/*"]
        },
        "typeRoots": ["node_modules/@types"],
        "outDir": "dist"
    }
}

// @fileName: /temp/test/project1/tsconfig.json
{
    "extends": "../base.tsconfig.json",
    "compilerOptions": {
        "paths": {
            "@environment/*": ["env/*"]
        },
        "typeRoots": ["typings"]
    }
}

Here is what it would mean that project1 tsconfig was written as:

// @fileName: /temp/test/project1/computed.tsconfig.json
{
    "compilerOptions": {
        "paths": {
            "@environment/*": ["env/*"]
        },
        "typeRoots": ["./typings"],
        "outDir": "../dist"
    }
}

Over years users have asked us for ability to create a final config that adds to base config instead of overriding. This is extremely useful for paths from compiler options or say references that specify project references.

This would allow to users to write a base config that contains common options that shouldn't have to be copied in all projects and yet allowing projects to specify there additional specific options.

For example in above sample, one would expect a way to write config such that final paths property is:

// @fileName: /temp/test/project1/computed.tsconfig.json
{
    "compilerOptions": {
        "paths": {
            "@libs/*": ["../libs/*"],
            "@project1/*": ["../project1/*"],
            "@environment/*": ["env/*"]
        }
    }
}

Here is the proposal to add merge as root level config property that specifies options that need to merged instead of completely overwritten in the final config.
The values specified in merge can be root level properties: files, include, exclude and references or option names from either compilerOptions, watchOptions or typeAcquisition which are lists or object type eg. paths, typeRoots, libs etc.

From the above example writing the project1 config as:

// @fileName: /temp/test/project1/tsconfig.json
{
    "extends": "../base.tsconfig.json",
    "compilerOptions": {
        "paths": {
            "@environment/*": ["env/*"]
        },
        "typeRoots": ["typings"]
    },
    "merge": ["paths", "typeRoots"]
}

per @andrewbranch 's suggestion here: #56436 (comment)
other option is to write config as:

{
  "compilerOptions": {
    "extend": {
      "paths": {
        "@/foo/*": ["../packages/foo/dist/*"]
      }
    }
  },
  "extend": {
    "references": [
      { "path": "../packages/foo"  }
    ]
  }
}
// Note we probably need to call this merge since extends and extend are too close at root level ?

Would mean that project1 wrote the tsconfig as:

// @fileName: /temp/test/project1/computed.tsconfig.json
{
    "compilerOptions": {
        "paths": {
            "@libs/*": ["../libs/*"],
            "@project1/*": ["../project1/*"],
            "@environment/*": ["env/*"]
        },
        "typeRoots": ["../node_modules/@types", "./typings"],
        "outDir": "../dist"
    }
}

Note how the paths still remain relative to config they are defined in but allowing one to add more properties and if say baseConfig also had rule for @environment/* path mapping it is overwritten by the project1 specific. That is it does not recusively merge property values of path mappings.

Apart from having ability to specify specific options that get merged, users have often asked us for a way to specify config such that base config will specify option say outDir and instead of it being relative to the base config, it needs to be relative to project's tsconfig.

For example in the above example, instead of all projects output going into /temp/test/dist the output to be inside each dist folder of each project.
Apart from outDir this would also help in writing base configs such that extending base.tsconfig.json would include project's src files and extending base.test.tsconfig.json would include tests files

With below configs:

// @fileName: /temp/test/base.tsconfig.json
{
    "compilerOptions": {
        "typeRoots": ["node_modules/@types"],
        "outDir": "dist"
    }
}

// @fileName: /temp/test/project1/tsconfig.json
{
    "extends": "../base.tsconfig.json"
}

Today the resulting config of project1 is:

// @fileName: /temp/test/project1/computed.tsconfig.json
{
    "compilerOptions": {
        "typeRoots": ["../node_modules/@types"],
        "outDir": "../dist"
    }
}

Proposal here is to add property raw with names of properties in base config such that extending it would mean as if project extending it specified them, making them relative to final config instead of the base config.

So with change to base.tsconfig.json as:

// @fileName: /temp/test/base.tsconfig.json
{
    "compilerOptions": {
        "typeRoots": ["node_modules/@types"],
        "outDir": "dist"
    },
    "raw": ["typeRoots", "outDir"]
}

per @andrewbranch suggestion option 2 to consider:

// tsconfig.base.json
{
  "compilerOptions": {
    "outDir": "${projectRoot}/dist"
  }
}
// need to parse each file path, is projectRoot allowed anywhere in the path or just start

would result in final project1 config as if it wrote:

// @fileName: /temp/test/project1/computed.tsconfig.json
{
    "compilerOptions": {
        "typeRoots": ["./node_modules/@types"],
        "outDir": "./dist"
    }
}

and another project2 say with merge property as:

// @fileName: /temp/test/project2/tsconfig.json
{
    "extends": "../base.tsconfig.json",
    "compilerOptions": {
        "typeRoots": ["typings"]
    },
    "merge": ["typeRoots"]
}

would result in configuration as if written as:

// @fileName: /temp/test/project2/computed.tsconfig.json
{
    "compilerOptions": {
        "typeRoots": ["./node_modules/@types", "./typings"],
        "outDir": "./dist"
    }
}

The property raw can be used for:

1. Options that are paths like: outDir, declarationDir etc
2. Options that list of paths like: rootDirs, typeRoots from compilerOptions or excludeFiles, excludeDirectories from watchOptions
3. include and exclude root config properties. (files and references are not included here as not sure if they really make sense?)
4. paths is a special option where normally the base path (property pathsBasePath) is relative to config they are declared in. So adding paths to raw would mean that path mapping is done relative to config file extending the root config file.

There was consideration of specifying raw extend as part of config extending root config but that would mean either writing something like "rawExtend": ["outDir", "typeRoots"] or "merge": [{ "name": "include", "kind": "raw" }] in each config defeating the purpose of concise extend. Any feedback on that?

Should merge be in root config: probably not because if project wants to override or add should be part of project decision and it would need to add properties to list only if that config specifies to making this property of project config makes more sense.

TODO:
What happens to raw and merge when multi-level extending is in play. Probably they need to be merged instead of overwritten but need to think about that.

Here are issues this would help with:
#44589
#27098
#20110
#29172

📃 Motivating Example

A way to specify say project specific "paths" mapping without having to rewrite all the mappings from the base config.
And a way to specify root config that can specify include and/or outDir and not needing to write than in each project.

💻 Use Cases

1. What do you want to use this for?
2. What shortcomings exist with current approaches?
3. What workarounds are you using in the meantime?

[andrewbranch]

I would definitely like to see more powerful options like these here. A few thoughts:

For merge functionality, I like the syntax that Tailwind uses for distinguishing between overrides and extensions:

{
  "extends": "./tsconfig.base.json",
  "compilerOptions": {
    // Override:
    "paths": {
      "@libs/*": ["libs/*"],
      "@project1/*": ["project1/*"]
    },

    // Merge:
    "extend": {
      "paths": {
        "@libs/*": ["libs/*"],
        "@project1/*": ["project1/*"]
      }
    }
  }
}

I think the semantics can be the same as what you proposed, but (a) this scheme has prior art that will be familiar to many, and (b) the indicator that an extension is happening is more closely colocated with the property doing the extending.

Instead of raw, I think I prefer the use of interpolated variables, which I think have come up in some discussions in the past:

// tsconfig.base.json
{
  "compilerOptions": {
    "outDir": "${projectRoot}/dist"
  }
}

This is again just different syntax for the same semantics you proposed, but I like it because (a) it’s obvious what’s intended to happen just by looking at the option value, (b) it’s impossible to forget whether "raw" belongs in the extending or base config, and (c) the word “raw” doesn’t do as much for my intuition about what’s going to happen as seeing a variable placeholder (the name of which is debatable).

[sheetalkamat]

For merge i thought about having separate entry because of references which many people want to extend and is not part of compilerOptions and is array of objects and not object itself.

For raw using syntax while more clear${projectRoot}/dist, it means that we have to do file path parsing on each config instead of relying on one property to do "transform" to either absolute path or use raw one.

[andrewbranch]

I find the desire to extend references pretty suspicious, but I think extend covers that well since it could appear both inside compilerOptions and at the root:

{
  "compilerOptions": {
    "extend": {
      "paths": {
        "@/foo/*": ["../packages/foo/dist/*"]
      }
    }
  },
  "extend": {
    "references": [
      { "path": "../packages/foo"  }
    ]
  }
}

Overall, I think clarity should be the top priority—it’s hard to imagine config parsing being a significant part of the total compilation time of a large app.

[sheetalkamat]

we probably need to use "merge" instead of "extend" given we already have "extends" at root

[dannyfranca]

Writing extend under other options doesn't provide much clarity and conciseness. But I agree that extending references does not sound great.

The original merge property proposal looks fantastic. It seems natural for an object to be merged and an array to be concatenated.

The raw option, as described, also seems fantastic. It gives projects control over paths and allows them to keep standards easily.

Regarding merge and raw configs on the base config, I think options should be ignored entirely if the tsconfig does not extend another. But rawExtend looks like a great way of establishing those standards for path-like properties.

Making the merge too configurable seems too much. A simple object merge and array concat seem reasonable and beautifully address most users' requests.

Thanks a lot for bringing that to light, @sheetalkamat; I hope other users engage in this discussion.

[ArZargaryan]

+1 A very useful and necessary feature, especially in monorepositories (as in my case)

[antongolub]

Coincidentally, we are developing a system that requires processing of this kind of configs with a declarative extras (preset, extends), so on, etc. In short, the workaround for the mentioned case looks like this:

const tsconfig = await populate('tsconfig.json', {
  rules: {
    compilerOptions: 'merge',
    'compilerOptions.paths': 'merge',
    'compilerOptions.typeRoots': 'merge'
  },
  vmap({value, cwd, root, prefix, key}) {
    if (cwd !== root && (
      prefix === 'compilerOptions.outDir' ||
      prefix.startsWith('compilerOptions.typeRoots.') ||
      /^compilerOptions\.paths\.[^.]+\./.test(prefix))
    ) {
      return path.join(path.relative(root, cwd), value)
    }
    return value
  }
})

It's definitely too verbose. So we're planning to use this proposal syntax instead we've added support for the shorthand syntax instead:

const tsconfig = await populate('tsconfig.json', {
  compilerOptions:               'merge',
  'compilerOptions.paths':       'merge',
  'compilerOptions.typeRoots':   'merge',
  'compilerOptions.typeRoots.*': 'rebase',
  'compilerOptions.outDir':      'rebase',
  'compilerOptions.paths.*.*':   'rebase'
})

[sheetalkamat]

Closing this in favor of #57485 and #57486