Suggestion: target specific runtimes instead of JS versions

[XaveScor]

First advantage is target to browser is native for user than target to js version. Don't need to know what each browser support.
Second is ts can transpilate more effective. Some browser can have partially support of next standard. Ts can transpilate unsupported features only.

[DanielRosenwasser]

So you're asking for something like babel-preset-env. I don't really know how many people actually have targets that end up being drastically different from ES5 and ES2015 to be honest, so I'd be open to hearing more feedback.

[eddiemoore]

I'd like to see this in as well, as browsers are adding more features all the time, if we are able to specify the browsers that we are targeting then it will only need to compile what is required.

Would suggest using https://github.com/ai/browserslist as it's pretty much the standard for this, and used by lots of other projects.

Many people are only targeting the last few versions of browsers, or if you are doing a project specifically for Electron then you only really need to target a specific "browser"

Would love to see this happen.

[ericdrobinson]

@DanielRosenwasser

I don't really know how many people actually have targets that end up being drastically different from ES5 and ES2015 to be honest, so I'd be open to hearing more feedback.

Here goes.

---

Background

As a TypeScrpt user, I do not write code that runs on a standards-compliant JavaScript engine. I do write code that runs on one-or-more custom JavaScript engines. I would like TypeScript to warn me when I make use of an API that does not exist in the target engine at compile time, rather than debug the issue when a problem occurs at runtime.

Today, TypeScript's core lib.d.ts file is not standards compliant. It is based on a spec file generated by the Microsoft Edge browser:

browser.webidl.xml: an XML spec file generated by Microsoft Edge.

While there is movement to replace these specs with specs generated from the standards themselves (see [1], [2], [3]), this still leaves us with a type system that will suggest/allow APIs that would cause runtime errors when run in certain targeted engines.

The following image (taken from this comment by @kitsonk) illustrates the issue well:

The numbers reported by that tool when "Specifications" is replaced with "Mozilla Firefox" (as of 2018/09/25):

• 7537 APIs in Microsoft Edge
• 11764 APIs in Google Chrome
• 10225 APIs in Mozilla Firefox
• 5697 APIs in Microsoft Edge ∩ Google Chrome ∩ Mozilla Firefox

As a TypeScript user, it is those 5697 APIs that I want to use. If I venture outside of the bounds of that set of APIs, at the very least I want TypeScript to warn me. That said, it would be even better if TypeScript could support me in that endeavor.

An extra challenge not addressed by the simplicity of that image is that the shape of the intersection in question changes between browser (JavaScript engine) releases. Some releases will add new APIs; some releases will remove APIs. Browsers are a moving target and that is considered as part of this proposal.

Proposal

Provide a new --target-engines compiler option that accepts a set of version-specific engines (e.g. dom.chrome.64) that TypeScript will use to source type information. Entries in this set may be open or closed ranges (e.g. dom.chrome.64+ or dom.chrome.48-64).

(Note: These examples are for illustrative purposes only and are not deeply considered. The Browserslist package may serve as a better source for format inspiration.)

When the --target-engines option is set, the --lib option is ignored.

Usage

When the --target-engines compiler option is set, TypeScript will retrieve the type libraries for each individual engine specified, as well as each engine within the ranges specified. It will then take the intersection (∩) of those libraries and populate the type system with the results.

Example

Specifying the following in a tsconfig.json file:

"target-engines": [
    "dom.ie.8+",
    "dom.firefox.42+",
    "dom.chrome.45+",
]

would restrict the types available in TypeScript's type system to those common across the ranges of the browsers specified.

Engine Guards and Differentiating Engines

Similar to TypeScript's Type Guards, Engine Guards would allow the developer to create scopes within which TypeScript's type system will adjust the available types based on the specified guard. Typically, implementing an engine guard would result in the type system expanding the set of available APIs within the guarded scope.

If a developer wishes to use FancyAPI but only two of the three engines they've specified support it, then they could use an engine guard to provide a scope within which they have access not just to FancyAPI, but the expanded set of all types that the two engines that support FancyAPI support. The developer can say "within this context, I would like access to every type that is available in engines where FancyAPI is supported."

The //@ts-engine-guard Comment

The // @ts-engine-guard comment is a directive to the TypeScript compiler that the following line should be evaluated as an engine guard and not produce a name-not-found/property-does-not-exist error.

An example:

// Types restricted to those supported in EngineA, EngineB, and EngineC.
let url: string = "https://github.com/Microsoft/TypeScript/issues/19183";

// Fetch is unsupported in EngineB. Using it here would cause TypeScript
//  to emit a helpful error.

// @ts-engine-guard
if (window.fetch !== undefined)
{
    // EngineB's type restrictions are removed. This scope now has
    //  access to all types provided in both EngineA AND EngineC.

    fetch(url); // Safe!
}

Note that if a fetch polyfill existed (or was shadowed locally in the given scope) then the engine guard functionality would not be triggered and a warning about the issue emitted. If a fetch polyfill is added after such a check was used as a guard and caused problems via some other API within the block, then the problematic API would replace the check against fetch to maintain the type expansion.

Further, if the guarded API happened to be one that existed only for a sub-range of a specified range of a given engine, then that sub-range would influence the expanded type intersection. This could happen if an API was added in one version and then removed in a later version within the range. Note that this could conceivably help developers running Automated/Continuous Integration systems identify issues resulting from API removal in new engine releases.

Generating Engine Library Files

There are several options for generating engine library files. A few to consider:

• Generate from vendor-specific WebIDL specs. It looks like some vendors (e.g. Google, Mozilla, Microsoft) support WebIDL in some manner.
• For each engine runtime, traverse the type system [as is done by Microsoft's API Catalog].
  • The FAQ claims that the API data is "obtained from enumerating each browser's DOM".
• Generate browser engine libraries from caniuse data or Mozilla's browser-compat-data.
  • Other engines would benefit from support tooling.
• Request engine vendor buy-in and support. Provide vendors with tools to assist them in generating the library files themselves such that it can simply become a part of their build/distribution process.

Distributing Engine Library Files

Distribution of engine library files could (should?) be done via Definitely Typed as is done with normal declaration files.

Beyond Browsers

Browsers aren't the only "JavaScript" (ECMAScript) engines that would benefit from such a system. Applications are frequently developed using embedded JavaScript engines, typically using some version of Chromium Embedded Framework or Webkit/JavaScript Core). Being able to specify a single engine version (e.g. dom.chrome.58) would allow developers to safely use all APIs provided in those platforms without restriction (other than quirks, of course).

A "Complex" Use Case

Another example includes Adobe's Creative Suite applications which support the Adobe Common Extensibility Platform (CEP). CEP extensions run multiple independent ECMAScript engines:

• Chromium Embedded Framework (via NW.js integration)
• Node (via NW.js integration)
• ExtendScript (runs in the host application)
  • Application-specific Extensions - Photoshop, InDesign, Illustrator, and Premiere Pro all provide different APIs in their respective environments.

Broadly, a project can be thought of as having two overarching engines:

1. A specific version of NW.js (Chrome+Node)
2. ExtendScript + Application extensions (e.g. the Photoshop API).

While the ExtendScript language has not seen updates in many years, the application APIs and NW.js integrations change with almost every application release. For developers working in this environment who frequently wish to support more than a single application (or version of an application), it is very desirable to have a programming environment where types are properly restricted to specific, targeted engines/contexts.

In other words, developers could specify dom.aftereffects.13+ and have their type system restricted to APIs consistently available across After Effects 13 to latest. (Such developers would similarly restrict their Node/Chromium versions to those supported by the platform for those contexts as well.)

Generating Libraries for CEP Applications

Adobe provides custom XML files that describe the ExtendScript API with their ExtendScript Toolkit (ESTK). Further, there are mechanisms by which to produce similar XML files for application-specific extensions. A simple conversion step between these files and TypeScript declaration files can be achieved. Any extra tooling would presumably assist in such conversions.

Universal Application Coding

This system could also assist developers writing "universal code" (e.g. with certain Server Side Rendering frameworks). From the Vue.js Server-Side Rendering Guide:

Universal code cannot assume access to platform-specific APIs, so if your code directly uses browser-only globals like window or document, they will throw errors when executed in Node.js, and vice-versa.

With this feature, the TypeScript language services could identify such errors at edit-time, rather than during execution. Further, TypeScript enabled IDEs/editors could be configured to understand such "universal" contexts and only show APIs (autocompletion/IntelliSense) that are supported by the intersection of the target engines in the first place (in the case quoted above, some set of NodeJS versions and some set of Browsers [and their versions]). This would help developers program with greater confidence and fewer bugs.

Wrap Up

At the heart of this proposal is the desire to restrict the type system to resolving only common types from amongst a set of versioned declarations for a single library. While this is most clearly understood in terms of browser support, one could also say "I would like to create a utility that works with every version of jQuery 2.x." TypeScript could take the intersection of the declaration files for 2.0, 2.1, and 2.2 and restrict the type system to only resolve APIs consistently available across those versions.

[MadaraUchiha]

I would definitely like to see this feature in TypeScript. It's incredibly useful in Babel.

[dgoldstein0]

Love this idea! The website I work on compiles to ES5, but practically, we are not shipping a website for ES5 compliant browsers; rather we ship a website that needs to work in a whole set of browsers. Which for me is currently last 2 versions of Edge, Safari, Firefox, Chrome, and IE 11, and a few mobile browsers too. An interesting anecdote here: we recently dropped IE 10 and noticed that our supported browsers now all have support for ES6 Set and Map... but not without some quirks in IE11, so we've forked the standard declarations to make a version that bans the stuff that won't work in IE11, and include our custom declaration file for our typechecking. So we are sort of already hacking our own version of this.

As far as implementation... I think @ericdrobinson gave an excellent in depth proposal. That said, I also think it looks like a lot to implement for a first version. My suggestions:

• make engine definitions - the data that defines which features are available in which browsers - pluggable. Then focus on getting a good canonical set for modern browsers (and servers, e.g. nodejs) and expanding from there; that way snowflakes like Adobe CEP could provide their own if they aren't supported out of the box.
• I also like the idea of supporting some sort of type intersection for joining different engine defintions together. I don't know how this would be best expressed in the language, but if we can figure out a good way for it, then engine declarations are just normal declarations that get fed into an intersection operation.
• ts-engine-guard seems unnecessary to me. At the least, the example given does not seem compelling, as I would expect regular feature detection to just do the trick. E.g. if some targets have window.fetch and others don't, isn't wrapping usage of fetch with if (window.fetch) {...} enough to convince the normal null checks that it exists? Perhaps there is a use case for the proposed ts-engine-guard but I don't find the given one compelling, and think until there is a more compelling one, we should avoid the complexity.

[XaveScor]

@ericdrobinson good proposal. thanks. But "usage" part have a bad idea. JS world already have tool for this problem. Browserlist. We don't need reinvent existing tool.

Next suggestion is compile multiple targets. It is great idea because IE11 has far less APIs. Binary size for IE11 and Chrome/FF/Edge/Safari can vary 2 or more times. Modern browser users can download binary much faster.

[ericdrobinson]

@//ts-engine-guard is About API Scope Expansion

@dgoldstein0 Some questions for clarification:

ts-engine-guard seems unnecessary to me. At the least, the example given does not seem compelling, as I would expect regular feature detection to just do the trick. E.g. if some targets have window.fetch and others don't, isn't wrapping usage of fetch with if (window.fetch) {...} enough to convince the normal null checks that it exists? Perhaps there is a use case for the proposed ts-engine-guard but I don't find the given one compelling, and think until there is a more compelling one, we should avoid the complexity.

Perhaps a closer read would be useful here? The point of //@ts-engine-guard is not to guarantee that window.fetch is usable - to "convince the normal null checks that it exists". Rather, it is about API resolution scope expansion. As explained in the comments in the example script, "Engine B" does not have window.fetch. "Engine B" would never process the code in that if-block. The purpose of //@ts-engine-guard in this case is to act as a hint to the TypeScript language service that you would like it to re-evaluate the API intersection for the following context based on the specified conditions. In essence, the guard would allow the TypeScript language service to resolve (through IntelliSense, say) APIs that both "Engine A" and "Engine C" have without having to consider "Engine B"'s restrictions.

The hint was added under the assumption that evaluating every context against all engines could either be an expensive process or confusing to the user if used globally. That said, if this could be entirely inferred by the TypeScript language service automatically and it simply became something that developers could rely upon, then great!

Does that make sense?

Browserlist is Insufficient for this Proposal

But "usage" part have a bad idea. JS world already have tool for this problem. Browserlist. We don't need reinvent existing tool.

@XaveScor We're not reinventing the existing tool. Some information:

• As it stands today, Browserlist does not work in a useful way with TypeScript. By "useful" I mean with respect to IntelliSense autocompletion features.
• Browserlist is limited to Web Browsers. There are other ECMAScript (JS) environments that Browserlist does not support.
• Browserlist checks for API existence. This proposal goes much, much further in that it would replace the default declaration files (e.g. lib.d.ts) with a set of environment-specific (JavaScript 'contexts', of which Browsers may be one) declaration files that might then undergo an intersection merge. Declaration files aren't just about "existence" as they also provide documentation that systems like IntelliSense use to provide useful information to programmers during autocomplete operations and the like.

Browserlist is a good source for inspiration but using it directly for this feature would not work. This is not a reimplementation but a completely different feature.

Binary Size?

Next suggestion is compile multiple targets. It is great idea because IE11 has far less APIs. Binary size for IE11 and Chrome/FF/Edge/Safari can vary 2 or more times. Modern browser users can download binary much faster.

@XaveScor Can you please expand upon what you mean by "binary"? Do you mean WebAssembly?

[dgoldstein0]

The purpose of //@ts-engine-guard in this case is to act as a hint to the TypeScript language service that you would like it to re-evaluate the API intersection for the following context based on the specified conditions.

Ah that makes some sense.

That said I would love to use browser / server targets even without ts-engine-guard; it feels like a nice add-on rather than a required part of this proposed feature. Perhaps in part because my use case is mostly pure browsers and it's very rare for us to want to write code that won't work universally; I could see it coming more in handy if we have stuff that's more like if (process /* detect nodejs */) {...} else {...} as that's a use case where I could see wanting code to diverge in behavior and acceptable apis to use.

[ericdrobinson]

it feels like a nice add-on rather than a required part of this proposed feature.

Yes, that is true as written. It would be a "more complete" implementation with //@ts-engine-guard or an automatic version. I don't have any specific examples, but the thought was that some features (e.g. Promises) may also imply additional support APIs beyond the root class. With an engine guard feature, all APIs supported by the newly expanded area that support a 'guarded feature' (e.g. APIs that make use of or return a Promise) might also become available.

But, yes, this would be a secondary feature implementation - to be implemented once the core mechanisms of specifying a set of target runtime environments were in place. :)

[wizzard0]

i'd support this, specifically for targeting non-browser environments like Duktape, Espruino, Adobe ExtendScript, etc. Or "weird" browsers like embedded devices stuck with old Android versions (e.g. car multimedia).

An obvious problem is a combinatorial explosion of cases to test if we try to support "downlevel to run on Duktape + nw.js".

Possible long-term benefit is that breaking down engines into feature sets may enable easier maintenance for new features, though.

[kitsonk]

@wizzard0 those using esoteric runtimes can always --noLib and provide something else that is more suited for their runtime. For embedded systems, rolling your own lib is likely a better long term solution anyways. It would be really hard to force TypeScript to keep up with n esoteric runtimes.

[ericdrobinson]

@kitsonk I'll point out that this proposal isn't about asking TypeScript to keep up with esoteric runtimes. As you point out, you can already pass the --nolib option for specific use cases. This doesn't help people writing code for multiple runtimes, however. In the most common case, this is the idea that each of today's major browsers do not support standards-compliant runtimes - each has subtle differences, especially when you deal with browser versions. As mentioned in the proposal, TypeScript's core libs are to a degree generated from Microsoft Edge and has lots of its own issues as a result.

The idea is that each runtime provider [the esoteric included], could provide their own declaration files. This wouldn't be anything special and would not warrant a new feature proposal by itself. The difference is that the proposal outlines a suggestion to take a union of several different type declaration libraries based on setup.

Imagine that you're a web developer working on an Open Source library. You want to make sure (or at least have some assurance) that the code you write will work in a specific set of browsers and their versions. What's more, you want to ensure that the code (or some portion thereof) can be run in NodeJS. Today's TypeScript may help quite a bit but you can still end up using APIs that simply aren't supported in one runtime or another. Wouldn't it be nice if you could tell TypeScript which runtimes you wanted to target and it would simply adjust the types it resolves (and, therefore, the errors it throws) as a result?

Support for esoteric engines would come for free, provided the engine developers provided whatever necessary [versioned] libraries would be required to support such a system.

As for downlevel compilation, I think that should be raised as a separate issue, especially as TypeScript language services can be leveraged directly in pure JavaScript files as well.

[dgoldstein0]

ideally:

• each environment (browsers, nodejs, other obscure things) could have a declaration file describing what's available in that environment in terms of globals & builtins
• these declarations could be managed the same way as @types modules

In which case the core problem becomes adding a type system feature to combine the different environments' type declarations.

re downlevel compilation, this task does seem to have talked mostly about the type system implications of multiple environment support; so I think I agree with @ericdrobinson that whether to support any changes to the typescript output should probably be a separate issue; so far, I've found using es5 output to be suitable for my use cases.

Perhaps it's time to start proposing the necessary type system changes?

[tomwayson]

For my own purposes, I'd be fine w/ exactly what babel-preset-env does. In the babel community it's effectively an anti-pattern at this point to target anything other than whatever environment you intend to support (i.e. to target specs).

In many of the responses to this and related issues (#20095, #23156 (comment)) I hear the notion that it's not reasonable to expect TS to keep up what browsers support, and I totally agree.

Which made me think, how does babel able to keep up with the evolving browser landscape? Turns out they dont.

So, I wonder, is it possible to TS to implement or to extend TS w/ something that works just like that?

[dgoldstein0]

I think there are two threads to this conversation:
a) should typescript auto-inject missing polyfills?
b) should typescript typecheck code against definitions for the browsers it plans to run in?

These actually seem to be disjoint approaches to the problem of "how do we handle nonstandard parts of our environment" - e.g. do you polyfill missing features (option a), or do you disallow their usage (option b)?

I'm arguing for (b), and I believe @ericdrobinson is on the same page. @tomwayson you seem to want (a). Which is fine, but it's a different use case - e.g. I don't want random polyfills showing up in my compiled code, as they tend to add unexpected bloat and some do sketchy things to builtins (e.g. the closure compiler's WeakMap overrides Object.freeze & Object.seal); I want to use types to restrict ourselves to only what's implemented natively, plus what we've explicitly decided to polyfill.

In particular, what I'm hoping for is infrastructure to be able to plug in type declarations from arbitrary browsers - so I can get typings that are IE11|Edge|Chrome|Firefox and not have to manually manage a typings file that is the merge of all those browsers. And then hopefully the community will manage types for browsers. I think that's a reasonable compromise between "typescript doesn't want to manage declarations of all browsers" and "everyone writes their own typings" that seems to be recommended in #23156 (comment)

[ericdrobinson]

I'm arguing for (b), and I believe @ericdrobinson is on the same page.

That is correct.

In particular, what I'm hoping for is infrastructure to be able to plug in type declarations from arbitrary browsers - so I can get typings that are IE11|Edge|Chrome|Firefox and not have to manually manage a typings file that is the merge of all those browsers. And then hopefully the community will manage types for browsers.

That is precisely the idea. The main proposal covers the why (the problem) and the what (a mechanism by which TypeScript may be engineered to help solve the problem).

When it comes to sourcing the data that is actually used by the "mechanism" (the "Engine Library Files" [ELF]), I did not specify that the TypeScript team should take on the impossible task of providing/sourcing this data themselves. To my mind, the strongest solution would be the last one presented in the proposal: "Request engine vendor buy-in and support." This would mean that engine developers are responsible for delivering type declarations for their products. See:

Engine	Vendor Responsible for ELF
V8	Google
Chrome	Google
Chakra	Microsoft
Edge	Microsoft
JavaScriptCore	Apple
Safari	Apple
ExtendScript	Adobe

[Note: There is nothing magical about the term "Engine Library Files". That is simply a term I used to keep the conversation focused. The term is 100% equivalent to "Type Declarations for specific JS engines".]

There is actually nothing stopping those vendors from doing so today: they could generate engine-and-release-specific Type Declaration files and TypeScript would happily consume them. What is not currently possible is the ability to specify multiple target engines (Type Declaration files/ELFs) and select only those APIs that are available across those specified engines. Which... is what the proposal is all about :)

[wrumsby]

@ericdrobinson re:

Browserlist is limited to Web Browsers.

That's no longer true. As per "browsers" Browserslist can also target Electron and Node versions which I suspect might cover most of the "other" targets a TypeScript project may have.

[ericdrobinson]

That's no longer true. As per "browsers" Browserslist can also target Electron and Node versions which I suspect might cover most of the "other" targets a TypeScript project may have.

@wrumsby That's neat! While not a comprehensive list of engines, it's certainly a strong one. There's still the other issues I brought up in that section, though. :/

[wrumsby]

One thing I have done is to run ESLint with eslint-plugin-compat over the JavaScript code generated by TypeScript. With this plugin you can specify polyfills so a workflow can be:

1. compile TS to JS
2. lint the JS using eslint-plugin-compat + browserslist
3. for any missing polyfills add these using https://polyfill.io (for instance) and specify these in the settings.polyfills section of your ESLint config

(Ideally you would describe the polyfills in a way that both eslint-plugin-compat can use them and they're automatically added to the page using https://polyfill.io)

[simon-robertson]

TSC support for browserslist would be very good to see.

At the very least, using browserslist can reduce the number of polyfills that have to be injected into distributable files. If we only need to target the last 2 Chrome, Edge, and Firefox versions, the number of required polyfills would be dramatically reduced, and would result in smaller (and potentially more performant) runtime code.

Adding a path to a standard browserslist file, to the TSC config, should be sufficient.

[slavafomin]

This is indeed a very nice feature to have.

By the way, guys, do you know if there is existing compiled data that we can use to at least manually map different browser versions to TypeScript targets?

[dgoldstein0]

Off the top of my head I don't know of any that lists browser -> js spec version. But https://github.com/mdn/browser-compat-data will give you browser -> which js features are supported. This data is also used to power caniuse.com There's also https://kangax.github.io/compat-table/es6/ <https://kangax.github.io/compat-table/es5/> which might be closer to what you want. Haven't used it before but it seems likely to be accurate

…

On Fri, Nov 29, 2019, 8:59 PM Slava Fomin II ***@***.***> wrote: This is indeed a very nice feature to have. By the way, guys, do you know if there is existing compiled data that we can use to at least manually map different browser versions to TypeScript targets? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19183?email_source=notifications&email_token=AAIGO52JFWZGGHQQZTIKLQDQWHCINA5CNFSM4D7FYYQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFPXLHI#issuecomment-559904157>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIGO5YMRQIAXRCO7JQTQULQWHCINANCNFSM4D7FYYQA> .

[MadaraUchiha]

@dgoldstein0 Babel has this feature through the use of the preset-env. (Which likely uses that behind the scenes).

[thw0rted]

I came here looking not for automatic pollyfilling or anything like that, I just wanted to have a warning when my target compiler option is set too high and would cause failure in one or more targets from my browserslist. I'm only trying to avoid having tsc emit JS that uses a feature unavailable in my target platform. A compiler that's even smarter than this sounds great, but I think just issuing a warning/error when the values mis-match would be a great start, and a lot easier to design and implement.

[ericdrobinson]

I just wanted to have a warning when my target compiler option is set too high and would cause failure in one or more targets from my browserslist.

@thw0rted While this seems related, I would highly suggest creating a separate Feature Request for this (and perhaps referencing this proposal). Another option would be to suggest this in a comment on #16607 as a "Browserlist-aware API Usage Checker" sounds like a great candidate for a compiler plugin.

[dgoldstein0]

Yeah that is a bit different than what we're suggesting here: we're saying instead of target: es5 or target: es2015 or similar, why not target: ie11,chrome 70+,Firefox 60+, or something similar. (The particular proposal is target-engines) I think they solve similar use cases, though it seems better for typescript to just do the right thing when possible, rather than warning about doing the wrong thing.

…

On Wed, Dec 4, 2019, 10:23 AM Eric Robinson ***@***.***> wrote: I just wanted to have a warning when my target compiler option is set too high and would cause failure in one or more targets from my browserslist. @thw0rted <https://github.com/thw0rted> While this seems *related*, I would highly suggest creating a separate *Feature Request* for this (and perhaps referencing this proposal). Another option would be to suggest this in a comment on #16607 <#16607> as a "Browserlist-aware API Usage Checker" sounds like a great candidate for a compiler plugin. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19183?email_source=notifications&email_token=AAIGO56OW5CYOGJMAOBW6F3QW7YTXA5CNFSM4D7FYYQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF6AMHY#issuecomment-561776159>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIGO57BYYNSO2B3MFLGW6TQW7YTXANCNFSM4D7FYYQA> .

[thw0rted]

I think I was a bit confused, reading through the history of this issue, because at times the thread conflates target issues with lib issues. I've gone back over it and I think things are much clearer to me now. There are definitely some valuable ideas here, and I want to take a crack at synthesizing them myself:

The way Typescript behaves today places the burden on the developer to either learn a great deal about how each browser does or doesn't support various features of various ES versions, or to use a post-processor like babel-preset-env to do it for them. Devs are also required to understand the implications of which libs they include and how features might need to be polyfilled in specific environments. Pick an "old" lib and you can't write code with new, more efficient features; pick one that's too new and tsc might emit something that won't run on your target platform without a polyfill. This is further complicated by target, since some platforms will support one feature of a given ES version but not another, or support for the feature will be subject to limitations.

As a solution, I would like to slightly modify @ericdrobinson's proposal. Instead of trying to maintain a bunch of platform-specific definition files (already proposed in #23156 and rejected), we could throw in all the APIs we might conceivably use, then have tsc remove the ones that aren't supported by all our targets. I believe this could be possible using a combination of browserslist and caniuse-lite.

For each API declared in lib, the compiler could consult caniuse to see if all the targeted platforms support it, and remove/ignore the declaration if not. This might require some kind of metadata attached to the API declaration -- if implemented as "opt-in", the metadata could be added incrementally, without a large initial investment of effort or significant ongoing maintenance overhead. The same process could be used to replace target: instead of deciding whether to emit a feature based on the output ES level, each feature could be individually checked against caniuse for support. (I could also see this as a place to hook polyfill injection, but can you need polyfills if the compiler didn't include the API in the first place?)

I came to this solution because David is right -- why warn when you can just fix it? I believe all the pieces are already in place to do this. As Eric says, this might be better implemented as a plugin, but I do think it would improve the onboarding experience for new Typescript devs if they could declare target platforms in their config instead of worrying about lib or target. lib could default to something like ["dom", "esnext"] and the compiler would quietly ignore any declarations there that aren't in the project's browserslist. target could safely default to esnext and individual emit features could ratchet down to accommodate the least-capable platforms. Why not make it this easy?

[ericdrobinson]

because at times the thread conflates target issues with lib issues.

@thw0rted This is unsurprising: most people conflate the ECMAScript-level APIs with Web-platform APIs when they refer to "JavaScript". This is understandable, too, as the interpreter (the "engine") does not distinguish between the two, either - they are simply "built in".

The way Typescript behaves today places the burden on the developer to either learn a great deal about how each browser does or doesn't support various features of various ES versions, or to use a post-processor like babel-preset-env to do it for them. Devs are also required to understand the implications of which libs they include and how features might need to be polyfilled in specific environments. Pick an "old" lib and you can't write code with new, more efficient features; pick one that's too new and tsc might emit something that won't run on your target platform without a polyfill. This is further complicated by target, since some platforms will support one feature of a given ES version but not another, or support for the feature will be subject to limitations.

Beautifully stated! 🍻

Instead of trying to maintain a bunch of platform-specific definition files (already proposed in #23156 and rejected)

Two things:

1. The main proposal does not suggest maintaining a bunch of platform-specific definition files (termed Engine Library Files for clarity). Rather, it suggests that engine vendors themselves should maintain their own and that they be uploaded somewhere common (e.g. Definitely Typed). The distinction is important because the onus is on JavaScript engine creators to clearly declare what APIs a given release of their software actually supports (which, come on, they should be doing already anyway...).
2. Browser specific lib options #23156 was not "rejected". It was closed by the author as a duplicate of this one and Generate lib.d.ts from w3c dom spec #3027 (which tracks the desire to shift from an Edge browser to the W3C DOM spec as the "source" for lib.d.ts generation).

we could throw in all the APIs we might conceivably use, then have tsc remove the ones that aren't supported by all our targets.

This is effectively what the main proposal is all about.

I believe this could be possible using a combination of browserslist and caniuse-lite.

Such an approach would not address the issue raised here. To be more specific, browserslist maintains a very specific subset of "engines": major web browsers and Node.js. The two tools you mention may work well when targeting the web platform, but they would not be of use to developers writing plugins for Adobe CEP, Adobe UXP, Omni Group's Omni Automation (which runs on Apple's JXA [1]), or a host of other JavaScript engines and environments. What's more, the [incomplete] data contained in the caniuse database is surface-level - it may be helpful for identifying whether an API might cause an issue in a [major] browser (or Node), but it wouldn't be enough to enable the full suite of features that TypeScript enables when backed by declaration files.

I believe all the pieces are already in place to do this.

How does your solution differ from simply using the Browserslist/CanIUse combo at some stage in your development pipeline? It seems as though eslint-plugin-compat is pretty close already and supports TypeScript (possibly via/with typescript-eslint) - it even supports specifying your own selection of polyfills.

Your suggestion is pretty powerful - it's just very specific to the web platform (a specific [admittedly major] use case). In the main proposal, the caniuse data is suggested as one possible source for generating "Engine Library Files", but it specifically calls out that this would be limited to browsers. TypeScript is larger than browsers.

As Eric says, this might be better implemented as a plugin, but I do think it would improve the onboarding experience for new Typescript devs if they could declare target platforms in their config instead of worrying about lib or target. lib could default to something like ["dom", "esnext"] and the compiler would quietly ignore any declarations there that aren't in the project's browserslist. target could safely default to esnext and individual emit features could ratchet down to accommodate the least-capable platforms.

Perhaps it would make sense, then, to suggest this as a "plugin candidate" on #16607? That might seriously help by providing a solid use case; one that clearly states the requirements and expectations.

[thw0rted]

I saw your earlier comments about the limited scope of browserslist, how it's web-centric and excludes less popular embedded platforms like those weird Adobe ones. You also say that caniuse only has "surface level" data. My first thought was that since browserslist and caniuse are open source projects, why not just add the missing platforms and details? That may or may not be practical -- I found at least one issue in browserslist where somebody asked to add a less popular platform and the package maintainer declined. But my core point stands: I think the effort is more likely to succeed if the community is asked to maintain the "Engine Library". Remember, the problem with #23156 was the TS team doesn't want (and shouldn't have!) the responsibility of understanding all possible platforms.

Your proposal mentions Web IDL as a standardized way of expressing API surface and #3027 already links to a script to turn IDL files into a lib.d.ts. If browserslist / caniuse can't or won't be expanded to solve this problem, maybe the answer lies in a community-maintained repository of platform IDLs akin to DefinitelyTyped. This would also be useful outside the Typescript community, I'm sure.

I'd still prefer to see one source of truth with one syntax used for declaring target platforms, though. browerslist has a big head start in that area, even though it currently excludes some targets. Does the existing implementation at least ignore unknown platforms? That would allow us to use the current conventions for a new purpose (building a lib collection automatically) without breaking other tools that consume the same information, or making a new syntax and keeping the same information in multiple places.

[ericdrobinson]

But my core point stands: I think the effort is more likely to succeed if the community is asked to maintain the "Engine Library". Remember, the problem with #23156 was the TS team doesn't want (and shouldn't have!) the responsibility of understanding all possible platforms.

Once again, the proposal never suggests that the TypeScript team should be responsible for maintaining the "Engine Library Files". As this follow-up comment makes more explicit, the people actually building the engines (core JavaScript interpreters, browser "layers", etc.) should produce and maintain their own ELFs.

The original "get engine vendor buy-in for generating ELFs" option reads as follows in its entirety:

Request engine vendor buy-in and support. Provide vendors with tools to assist them in generating the library files themselves such that it can simply become a part of their build/distribution process.

That second sentence is pretty crucial and shouldn't be that difficult: Microsoft already has some tooling for generating Type Declaration Files (TDF) from WebIDL specs. Browser/interpreter vendors could use these tools on WebIDL that they output during their respective build processes, meaning that generating a new ELF would be automatic.

Some environments already have their own specs available. Adobe's ExtendScript-capable host applications (mostly) provide "Scripting Dictionaries" (XML files with type information that powers their own "documentation viewer"). The community has produced a tool to produce TDFs from those source files. Apple provides "sdef" ("scripting definition") files to power its documentation viewer for JXA. Once again, the community has produced a tool to produce TDFs from those source files.

The question of "where does the data come from" is not one for the TypeScript team to answer - it is one for the community and, mainly, the Engine vendors themselves to handle - which they should be doing already anyway.

To quote the previously referenced follow-up comment:

There is actually nothing stopping those vendors from doing so today: they could generate engine-and-release-specific Type Declaration files and TypeScript would happily consume them. What is not currently possible is the ability to specify multiple target engines (Type Declaration files/ELFs) and select only those APIs that are available across those specified engines. Which... is what the proposal is all about :)

---

maybe the answer lies in a community-maintained repository of platform IDLs akin to DefinitelyTyped. This would also be useful outside the Typescript community, I'm sure.

I completely agree that this would be useful outside the TypeScript community. That said, I think the engine vendors themselves should get their collective acts together and make exporting some form of "supported APIs" documentation/listing (WebIDL format would be peachy) a basic requirement. You could argue that they haven't "needed" to do this in the past, but I do think it would go a long way to powering enhancements with pure JavaScript tools and especially the growing TypeScript ecosystem.

The community could maintain these but it makes a heck-of-a-lot more sense to have the vendors simply handle it natively. We have a bit of a chicken-and-egg problem at the moment: there's little direct benefit for engine vendors to put in the work to generate such data natively without something to consume them. And without something to consume, it makes little sense to invest a lot of time in a system (as proposed here) that can merge disparately sourced data.

I'd still prefer to see one source of truth with one syntax used for declaring target platforms, though.

Agreed!

Does the existing implementation at least ignore unknown platforms? That would allow us to use the current conventions for a new purpose (building a lib collection automatically) without breaking other tools that consume the same information, or making a new syntax and keeping the same information in multiple places.

I'm not sure I follow. Can you clarify? What "existing implementation" are you referring to? The proposal? The browserslist toolset?

[thw0rted]

Can you clarify? What "existing implementation" are you referring to?

Yes, browserslist. They have, for better or worse, a pretty powerful and easy to use syntax for specifying sets of platforms and a convention for where to put those declarations. Rather than making a new syntax (dom.ie.8+ et al) and a new convention (target-engines), I think it would make sense to use the existing ones, provided that the existing ones can be used without conflicting with their current function.

[ericdrobinson]

Got it! Thanks for clarifying. Some thoughts:

Rather than making a new syntax

I'm not sure that it makes sense to use the browserslist syntax here, at least not in its entirety. Perhaps in terms of how the engines/versions are identified? But usage percentage and "maintenance" status would be a bit more difficult to define/maintain in the greater scheme of things, I think.

I should point out that the main proposal also suggests looking towards browserslist for format inspiration:

(Note: [The engine version identification examples provided] are for illustrative purposes only and are not deeply considered. The Browserslist package may serve as a better source for format inspiration.)

---

and a new convention (target-engines)

I have to disagree with this one. TypeScript does not care about your package.json or .browserslistrc files and it shouldn't. target-engines is proposed as a new compiler option, which would allow it to be passed to the compiler via the command line or in a tsconfig.json / jsconfig.json file.

[n1ru4l]

My concerns are mainly Node.js specific and a bit more targeted against lib:

Node.js>=12.10 has the full support of ES2019 and already supports some bits of ES2020. However, there is no option to only enable Promise.allSettled (ES2020) without enabling the whole ES2020 feature-set (which is only partially supported by Node.js 12).

It would help to allow more granular configuration of what APIs are available and which are not. However, the best solution would be a configuration flag where you specify your Node.js (or Browser) version and the TypeScript compiler automatically picks the APIs available for that Browser/Node.js.

Example with Promise.allSettled:

Node.js<12.10: Should cause a compiler error
Node.js@12.10: Should compile fine without errors

[dgoldstein0]

@n1ru4l I think part of what you want can be done with the lib setting today. though there's no es2020.promise, in theory that should exist and define Promise.allSettled for you, and you can choose to include or not include it. Much like es2015.promise which toggles the base promise typings, and es2018.promise which includes Promise.finally

that said it's totally valid that targetting specific node versions would be useful.

[n1ru4l]

@dgoldstein0 Yes, I also was a bit confused that there was no es2020.promise option. Seems like those granular flags were not added for newer versions.

[dgoldstein0]

according to https://www.typescriptlang.org/docs/handbook/compiler-options.html there's no es2019 or es2020 options for lib yet either. maybe these docs are stale? or maybe not. I think those deserve a separate issue - as adding es2019, es2020, and es2020.promise and similar should be immediately actionable with no extra designing.

[thw0rted]

I believe all those identifiers (es2020.promise etc) are just the names of files in the node_modules/typescript/lib directory. I have an es2020.promise.d.ts file in there, and it does define an allSettled method on PromiseConstructor. Have you tried just including es2020.promise and seeing what happens?

ETA: I found this because you can select any method call, like one to Promise#then, and use your IDE's "Go To Definition" command to find the lib file where it's declared. For me, that one is in lib.es5.d.ts in the same directory.

[n1ru4l]

@thw0rted Yes you are right "lib": ["ES2019", "ES2020.Promise"], works fine.

[simon-robertson]

So 18 months after posting my previous comment here, I'm now happy pointing tsconfig.json at ES2020 and letting Babel/Webpack handle the compiling for browsers/workers/node. TSC is now only used with VSCode and doesn't actually produce the output

Workspaces in VSCode help to separate the various runtime environments for TS libs, e.g. DOM, WebWorker, so the correct type definitions are made available

It could still be improved a bit but overall it does the job quite well

[dgoldstein0]

@simon-robertson making sure I understand: you are using target=ES2020? what about lib? and presumably using babel (as a webpack plugin) with bazel-preset-env insert any polyfills you need? am I missing any major details?

[simon-robertson]

@dgoldstein0

@simon-robertson making sure I understand: you are using target=ES2020? what about lib? and presumably using babel (as a webpack plugin) with bazel-preset-env insert any polyfills you need? am I missing any major details?

That is correct

As an example, consider the following project directory structure ...

source/app/components/application.tsx
source/app/index.tsx
source/app/tsconfig.json
source/workers/service.ts
source/workers/tsconfig.json
.browesrslistrc
.gitattributes
.gitignore
.eslintrc
tsconfig.json
vs.code-workspace

The base tsconfig.json file would be extended by the other tsconfig files and would look something like this ...

{
    "include": [],
    "compilerOptions": {
        "target": "ES2020",
        "lib": [],
        "types": [],
        "typeRoots": [
            "node_modules/@types"
        ],
        "allowJs": false,
        "allowSyntheticDefaultImports": true,
        "allowUnreachableCode": false,
        "allowUnusedLabels": false,
        "forceConsistentCasingInFileNames": true,
        "noImplicitAny": true,
        "noImplicitReturns": true,
        "noImplicitThis": true,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "strictBindCallApply": true,
        "strictFunctionTypes": true,
        "strictNullChecks": true,
        "strictPropertyInitialization": true
    },
    "typeAcquisition": {
        "enable": false
    }
}

The source/app/tsconfig.json file would look something like this ...

{
    "extends": "../tsconfig.json",
    "include": [
        "**/*.ts",
        "**/*.tsx"
    ],
    "compilerOptions": {
        "jsx": "react",
        "lib": [
            "DOM",
            "ES2020"
        ],
        "types": [
            "react",
            "react-dom"
        ]
    }
}

The source/workers/tsconfig.json file would look something like this ...

{
    "extends": "../tsconfig.json",
    "include": [
        "**/*.ts"
    ],
    "compilerOptions": {
        "lib": [
            "ES2020",
            "WebWorker"
        ]
    }
}

Notice how the lib and types arrays can be changed in each tsconfig file. This works because a VSCode workspace treats each workspace folder as a separate project, and TSC in VSCode is happy working this way

You define the workspace folders in a code-workspace file ...

{
    "folders": [
        {
            "name": "app",
            "path": "source/app"
        },
        {
            "name": "workers",
            "path": "source/workers"
        }
    ]
}

In VSCode, you would open a workspace instead of a folder

Webpack along with the Babel plugin are then used to build each of the main modules, so in this example that would be source/app/index.tsx and source/workers/service.ts, the output files can be created in the same directory, e.g. build, but you also have full control over that. I typically use the Babel environments @babel/preset-env @babel/preset-react and @babel/preset-typescript

Using browserslist gives you a lot of control over the browsers/platforms you want to target. Babel will automatically use a .browserslistrc file if you have one in your project. Any polyfills that are needed by your target browsers/platforms will be added by Babel, it uses polyfills provided by the core-js library

In a nutshell, TSC does not do any compiling; in my opinion Webpack and Babel can do a much better job

Obviously doing things this way requires more work to setup a project but it is worth doing, and we now have the option of creating/using GitHub template repositories for base project setups if needed

[codepunkt]

So 18 months after posting my previous comment here, I'm now happy pointing tsconfig.json at ES2020 and letting Babel/Webpack handle the compiling for browsers/workers/node. TSC is now only used with VSCode and doesn't actually produce the output

Workspaces in VSCode help to separate the various runtime environments for TS libs, e.g. DOM, WebWorker, so the correct type definitions are made available

It could still be improved a bit but overall it does the job quite well

Care to share your setup? I'm very interested! 👍

[simon-robertson]

@codepunkt
I have a work-in-progress project here if you're curious. You'll have to excuse the mess though, it's still a baby

https://github.com/simon-robertson/io