[heft] Add tryLoadProjectConfigurationFileAsync API and use in plugins

[dmichon-msft]

Summary

Adds a new API to HeftConfiguration, tryLoadProjectConfigurationFileAsync, so that plugins can load riggable config files without taking a direct dependency (and risking version duplication) on @rushstack/heft-config-file.

Starts addressing #5092. The motivation is to get us to a world where Heft plugins don't need any runtime dependencies other than on any external tools they bring with them, because all the core functionality of a Heft plugin is available on arguments passed to apply, similar to prior art in webpack plugins, that have the compiler.webpack field that provides access to all functionality that would otherwise be obtained via import ... from 'webpack'.

This API change removes any need for Heft plugins to take their own dependency on the @rushstack/heft-config-file package for normal scenarios.

Details

Fixes an issue with PathResolutionMethod.resolveRelativeToProjectRoot when extending from a file that lives in a different package. It was previously resolving relative to the package that contained the file being extended from, rather than the file that contains the original loaded configuration file.

Adds a new customValidationFunction option to the configuration file APIs that can be used to perform additional validation after loading a file.

Uses this API in heft-api-extractor-plugin, heft-sass-plugin, and heft-typescript-plugin.
Currently doesn't expose the version that throws if the file cannot be found.

All needed types are re-exported under the ConfigurationFile namespace on @rushstack/heft.

How it was tested

Existing build tests for these plugins.

Impacted documentation

API Documentation for @rushstack/heft.

[octogonz]

tryLoadProjectConfigurationFileAsync, so that plugins can load riggable config files without taking a direct dependency (and risking version duplication) on @rushstack/heft-config-file.

The motivation is to get us to a world where Heft plugins don't need any runtime dependencies other than on any external tools they bring with them, because all the core functionality of a Heft plugin is available on arguments passed to apply, similar to prior art in webpack plugins, that have the compiler.webpack field that provides access to all functionality that would otherwise be obtained via import ... from 'webpack'.

The goal is a good one. But this solution is not very generalizable:

• tryLoadProjectConfigurationFileAsync only solves one particular API from @rushstack/heft-config-file.
• and although Webpack provides a way to access the webpack package, if an application's webpack.config.js wants to import a loader that is a dependency of the rig package, the exact same problem resurfaces -- the application has to add is own dependency that risks version duplication. Therefore Webpack hasn't really solved this problem very well either.

Here are a couple other ideas we could consider:

• resolve relative to: We could provide an API rigRequire("some-package") that calls require("some-package") from the folder of the rig. Or more generally, use IImportResolveOptions.baseFolderPath to provide an API like requireRelativeTo("some-package", "the-rig-package") that resolves relative to an arbitrary dependency name. This avoids the need to wrap/reexport API's, but it does create a SemVer problem -- how can we be sure the resolved package has a compatible API signature?

• service locator pattern: My first suggestion is most appropriate for custom scripts like webpack.config.js where the project owner can take responsibility to ensuring versions are compatible. For a plugin, whose author has little control over how their code gets loaded into someone else's context, a stricter contract is necessary. This could be addressed by using versioned interfaces, similar to DirectX QueryInterface() or Managed Extensibility Framework (MEF) dependency injection or SPFx ServiceScope. With this design, @rushstack/heft-config-file defines interfaces with versions (e.g. ILoadConfigFile2 extends ILoadConfigFile1), which plugins can query from the Heft context. In this way, Heft itself doesn't need to be involved with the definitions of ILoadConfigFile2 -- it merely passes them along from its service scope.

[dmichon-msft]

The webpack example solves the problem because no loader or plugin in the ecosystem ever has a runtime dependency on webpack, they always use the currently loaded version by being handed it at runtime.

Resolve relative to is something you can do with core NodeJS APIs (and we do this in a number of places, but it doesn't improve the situation, because it means that consumers are still ultimately trying to load modules, not just be handed the API. We want to make it possible to bundle @rushstack/heft and all its dependencies into a single file without any of the plugins noticing.