Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: target specific runtimes instead of JS versions #19183

Open
XaveScor opened this issue Oct 14, 2017 · 20 comments

Comments

@XaveScor
Copy link

commented Oct 14, 2017

First advantage is target to browser is native for user than target to js version. Don't need to know what each browser support.
Second is ts can transpilate more effective. Some browser can have partially support of next standard. Ts can transpilate unsupported features only.

@DanielRosenwasser DanielRosenwasser changed the title Suggestion: target to browser instead of js version Suggestion: target specific runtimes instead of JS version Oct 14, 2017

@DanielRosenwasser DanielRosenwasser changed the title Suggestion: target specific runtimes instead of JS version Suggestion: target specific runtimes instead of JS versions Oct 14, 2017

@DanielRosenwasser

This comment has been minimized.

Copy link
Member

commented Oct 14, 2017

So you're asking for something like babel-preset-env. I don't really know how many people actually have targets that end up being drastically different from ES5 and ES2015 to be honest, so I'd be open to hearing more feedback.

@eddiemoore

This comment has been minimized.

Copy link

commented Oct 23, 2017

I'd like to see this in as well, as browsers are adding more features all the time, if we are able to specify the browsers that we are targeting then it will only need to compile what is required.

Would suggest using https://github.com/ai/browserslist as it's pretty much the standard for this, and used by lots of other projects.

Many people are only targeting the last few versions of browsers, or if you are doing a project specifically for Electron then you only really need to target a specific "browser"

Would love to see this happen.

@ericdrobinson

This comment has been minimized.

Copy link

commented Jan 27, 2018

@DanielRosenwasser

I don't really know how many people actually have targets that end up being drastically different from ES5 and ES2015 to be honest, so I'd be open to hearing more feedback.

Here goes.


Background

As a TypeScrpt user, I do not write code that runs on a standards-compliant JavaScript engine. I do write code that runs on one-or-more custom JavaScript engines. I would like TypeScript to warn me when I make use of an API that does not exist in the target engine at compile time, rather than debug the issue when a problem occurs at runtime.

Today, TypeScript's core lib.d.ts file is not standards compliant. It is based on a spec file generated by the Microsoft Edge browser:

browser.webidl.xml: an XML spec file generated by Microsoft Edge.

While there is movement to replace these specs with specs generated from the standards themselves (see [1], [2], [3]), this still leaves us with a type system that will suggest/allow APIs that would cause runtime errors when run in certain targeted engines.

The following image (taken from this comment by @kitsonk) illustrates the issue well:

API Overlap

The numbers reported by that tool when "Specifications" is replaced with "Mozilla Firefox" (as of 2018/09/25):

  • 7537 APIs in Microsoft Edge
  • 11764 APIs in Google Chrome
  • 10225 APIs in Mozilla Firefox
  • 5697 APIs in Microsoft Edge ∩ Google Chrome ∩ Mozilla Firefox

As a TypeScript user, it is those 5697 APIs that I want to use. If I venture outside of the bounds of that set of APIs, at the very least I want TypeScript to warn me. That said, it would be even better if TypeScript could support me in that endeavor.

An extra challenge not addressed by the simplicity of that image is that the shape of the intersection in question changes between browser (JavaScript engine) releases. Some releases will add new APIs; some releases will remove APIs. Browsers are a moving target and that is considered as part of this proposal.

Proposal

Provide a new --target-engines compiler option that accepts a set of version-specific engines (e.g. dom.chrome.64) that TypeScript will use to source type information. Entries in this set may be open or closed ranges (e.g. dom.chrome.64+ or dom.chrome.48-64).

(Note: These examples are for illustrative purposes only and are not deeply considered. The Browserslist package may serve as a better source for format inspiration.)

When the --target-engines option is set, the --lib option is ignored.

Usage

When the --target-engines compiler option is set, TypeScript will retrieve the type libraries for each individual engine specified, as well as each engine within the ranges specified. It will then take the intersection (∩) of those libraries and populate the type system with the results.

Example

Specifying the following in a tsconfig.json file:

"target-engines": [
    "dom.ie.8+",
    "dom.firefox.42+",
    "dom.chrome.45+",
]

would restrict the types available in TypeScript's type system to those common across the ranges of the browsers specified.

Engine Guards and Differentiating Engines

Similar to TypeScript's Type Guards, Engine Guards would allow the developer to create scopes within which TypeScript's type system will adjust the available types based on the specified guard. Typically, implementing an engine guard would result in the type system expanding the set of available APIs within the guarded scope.

If a developer wishes to use FancyAPI but only two of the three engines they've specified support it, then they could use an engine guard to provide a scope within which they have access not just to FancyAPI, but the expanded set of all types that the two engines that support FancyAPI support. The developer can say "within this context, I would like access to every type that is available in engines where FancyAPI is supported."

The //@ts-engine-guard Comment

The // @ts-engine-guard comment is a directive to the TypeScript compiler that the following line should be evaluated as an engine guard and not produce a name-not-found/property-does-not-exist error.

An example:

// Types restricted to those supported in EngineA, EngineB, and EngineC.
let url: string = "https://github.com/Microsoft/TypeScript/issues/19183";

// Fetch is unsupported in EngineB. Using it here would cause TypeScript
//  to emit a helpful error.

// @ts-engine-guard
if (window.fetch !== undefined)
{
    // EngineB's type restrictions are removed. This scope now has
    //  access to all types provided in both EngineA AND EngineC.

    fetch(url); // Safe!
}

Note that if a fetch polyfill existed (or was shadowed locally in the given scope) then the engine guard functionality would not be triggered and a warning about the issue emitted. If a fetch polyfill is added after such a check was used as a guard and caused problems via some other API within the block, then the problematic API would replace the check against fetch to maintain the type expansion.

Further, if the guarded API happened to be one that existed only for a sub-range of a specified range of a given engine, then that sub-range would influence the expanded type intersection. This could happen if an API was added in one version and then removed in a later version within the range. Note that this could conceivably help developers running Automated/Continuous Integration systems identify issues resulting from API removal in new engine releases.

Generating Engine Library Files

There are several options for generating engine library files. A few to consider:

  • Generate from vendor-specific WebIDL specs. It looks like some vendors (e.g. Google, Mozilla, Microsoft) support WebIDL in some manner.
  • For each engine runtime, traverse the type system [as is done by Microsoft's API Catalog].
    • The FAQ claims that the API data is "obtained from enumerating each browser's DOM".
  • Generate browser engine libraries from caniuse data or Mozilla's browser-compat-data.
    • Other engines would benefit from support tooling.
  • Request engine vendor buy-in and support. Provide vendors with tools to assist them in generating the library files themselves such that it can simply become a part of their build/distribution process.

Distributing Engine Library Files

Distribution of engine library files could (should?) be done via Definitely Typed as is done with normal declaration files.

Beyond Browsers

Browsers aren't the only "JavaScript" (ECMAScript) engines that would benefit from such a system. Applications are frequently developed using embedded JavaScript engines, typically using some version of Chromium Embedded Framework or Webkit/JavaScript Core). Being able to specify a single engine version (e.g. dom.chrome.58) would allow developers to safely use all APIs provided in those platforms without restriction (other than quirks, of course).

A "Complex" Use Case

Another example includes Adobe's Creative Suite applications which support the Adobe Common Extensibility Platform (CEP). CEP extensions run multiple independent ECMAScript engines:

  • Chromium Embedded Framework (via NW.js integration)
  • Node (via NW.js integration)
  • ExtendScript (runs in the host application)
    • Application-specific Extensions - Photoshop, InDesign, Illustrator, and Premiere Pro all provide different APIs in their respective environments.

Broadly, a project can be thought of as having two overarching engines:

  1. A specific version of NW.js (Chrome+Node)
  2. ExtendScript + Application extensions (e.g. the Photoshop API).

While the ExtendScript language has not seen updates in many years, the application APIs and NW.js integrations change with almost every application release. For developers working in this environment who frequently wish to support more than a single application (or version of an application), it is very desirable to have a programming environment where types are properly restricted to specific, targeted engines/contexts.

In other words, developers could specify dom.aftereffects.13+ and have their type system restricted to APIs consistently available across After Effects 13 to latest. (Such developers would similarly restrict their Node/Chromium versions to those supported by the platform for those contexts as well.)

Generating Libraries for CEP Applications

Adobe provides custom XML files that describe the ExtendScript API with their ExtendScript Toolkit (ESTK). Further, there are mechanisms by which to produce similar XML files for application-specific extensions. A simple conversion step between these files and TypeScript declaration files can be achieved. Any extra tooling would presumably assist in such conversions.

Universal Application Coding

This system could also assist developers writing "universal code" (e.g. with certain Server Side Rendering frameworks). From the Vue.js Server-Side Rendering Guide:

Universal code cannot assume access to platform-specific APIs, so if your code directly uses browser-only globals like window or document, they will throw errors when executed in Node.js, and vice-versa.

With this feature, the TypeScript language services could identify such errors at edit-time, rather than during execution. Further, TypeScript enabled IDEs/editors could be configured to understand such "universal" contexts and only show APIs (autocompletion/IntelliSense) that are supported by the intersection of the target engines in the first place (in the case quoted above, some set of NodeJS versions and some set of Browsers [and their versions]). This would help developers program with greater confidence and fewer bugs.

Wrap Up

At the heart of this proposal is the desire to restrict the type system to resolving only common types from amongst a set of versioned declarations for a single library. While this is most clearly understood in terms of browser support, one could also say "I would like to create a utility that works with every version of jQuery 2.x." TypeScript could take the intersection of the declaration files for 2.0, 2.1, and 2.2 and restrict the type system to only resolve APIs consistently available across those versions.

@MadaraUchiha

This comment has been minimized.

Copy link

commented Mar 4, 2018

I would definitely like to see this feature in TypeScript. It's incredibly useful in Babel.

@dgoldstein0

This comment has been minimized.

Copy link

commented May 10, 2018

Love this idea! The website I work on compiles to ES5, but practically, we are not shipping a website for ES5 compliant browsers; rather we ship a website that needs to work in a whole set of browsers. Which for me is currently last 2 versions of Edge, Safari, Firefox, Chrome, and IE 11, and a few mobile browsers too. An interesting anecdote here: we recently dropped IE 10 and noticed that our supported browsers now all have support for ES6 Set and Map... but not without some quirks in IE11, so we've forked the standard declarations to make a version that bans the stuff that won't work in IE11, and include our custom declaration file for our typechecking. So we are sort of already hacking our own version of this.

As far as implementation... I think @ericdrobinson gave an excellent in depth proposal. That said, I also think it looks like a lot to implement for a first version. My suggestions:

  • make engine definitions - the data that defines which features are available in which browsers - pluggable. Then focus on getting a good canonical set for modern browsers (and servers, e.g. nodejs) and expanding from there; that way snowflakes like Adobe CEP could provide their own if they aren't supported out of the box.
  • I also like the idea of supporting some sort of type intersection for joining different engine defintions together. I don't know how this would be best expressed in the language, but if we can figure out a good way for it, then engine declarations are just normal declarations that get fed into an intersection operation.
  • ts-engine-guard seems unnecessary to me. At the least, the example given does not seem compelling, as I would expect regular feature detection to just do the trick. E.g. if some targets have window.fetch and others don't, isn't wrapping usage of fetch with if (window.fetch) {...} enough to convince the normal null checks that it exists? Perhaps there is a use case for the proposed ts-engine-guard but I don't find the given one compelling, and think until there is a more compelling one, we should avoid the complexity.
@XaveScor

This comment has been minimized.

Copy link
Author

commented May 10, 2018

@ericdrobinson good proposal. thanks. But "usage" part have a bad idea. JS world already have tool for this problem. Browserlist. We don't need reinvent existing tool.

Next suggestion is compile multiple targets. It is great idea because IE11 has far less APIs. Binary size for IE11 and Chrome/FF/Edge/Safari can vary 2 or more times. Modern browser users can download binary much faster.

@ericdrobinson

This comment has been minimized.

Copy link

commented May 10, 2018

@//ts-engine-guard is About API Scope Expansion

@dgoldstein0 Some questions for clarification:

ts-engine-guard seems unnecessary to me. At the least, the example given does not seem compelling, as I would expect regular feature detection to just do the trick. E.g. if some targets have window.fetch and others don't, isn't wrapping usage of fetch with if (window.fetch) {...} enough to convince the normal null checks that it exists? Perhaps there is a use case for the proposed ts-engine-guard but I don't find the given one compelling, and think until there is a more compelling one, we should avoid the complexity.

Perhaps a closer read would be useful here? The point of //@ts-engine-guard is not to guarantee that window.fetch is usable - to "convince the normal null checks that it exists". Rather, it is about API resolution scope expansion. As explained in the comments in the example script, "Engine B" does not have window.fetch. "Engine B" would never process the code in that if-block. The purpose of //@ts-engine-guard in this case is to act as a hint to the TypeScript language service that you would like it to re-evaluate the API intersection for the following context based on the specified conditions. In essence, the guard would allow the TypeScript language service to resolve (through IntelliSense, say) APIs that both "Engine A" and "Engine C" have without having to consider "Engine B"'s restrictions.

The hint was added under the assumption that evaluating every context against all engines could either be an expensive process or confusing to the user if used globally. That said, if this could be entirely inferred by the TypeScript language service automatically and it simply became something that developers could rely upon, then great!

Does that make sense?

Browserlist is Insufficient for this Proposal

But "usage" part have a bad idea. JS world already have tool for this problem. Browserlist. We don't need reinvent existing tool.

@XaveScor We're not reinventing the existing tool. Some information:

  • As it stands today, Browserlist does not work in a useful way with TypeScript. By "useful" I mean with respect to IntelliSense autocompletion features.
  • Browserlist is limited to Web Browsers. There are other ECMAScript (JS) environments that Browserlist does not support.
  • Browserlist checks for API existence. This proposal goes much, much further in that it would replace the default declaration files (e.g. lib.d.ts) with a set of environment-specific (JavaScript 'contexts', of which Browsers may be one) declaration files that might then undergo an intersection merge. Declaration files aren't just about "existence" as they also provide documentation that systems like IntelliSense use to provide useful information to programmers during autocomplete operations and the like.

Browserlist is a good source for inspiration but using it directly for this feature would not work. This is not a reimplementation but a completely different feature.

Binary Size?

Next suggestion is compile multiple targets. It is great idea because IE11 has far less APIs. Binary size for IE11 and Chrome/FF/Edge/Safari can vary 2 or more times. Modern browser users can download binary much faster.

@XaveScor Can you please expand upon what you mean by "binary"? Do you mean WebAssembly?

@dgoldstein0

This comment has been minimized.

Copy link

commented May 11, 2018

The purpose of //@ts-engine-guard in this case is to act as a hint to the TypeScript language service that you would like it to re-evaluate the API intersection for the following context based on the specified conditions.

Ah that makes some sense.

That said I would love to use browser / server targets even without ts-engine-guard; it feels like a nice add-on rather than a required part of this proposed feature. Perhaps in part because my use case is mostly pure browsers and it's very rare for us to want to write code that won't work universally; I could see it coming more in handy if we have stuff that's more like if (process /* detect nodejs */) {...} else {...} as that's a use case where I could see wanting code to diverge in behavior and acceptable apis to use.

@ericdrobinson

This comment has been minimized.

Copy link

commented May 11, 2018

it feels like a nice add-on rather than a required part of this proposed feature.

Yes, that is true as written. It would be a "more complete" implementation with //@ts-engine-guard or an automatic version. I don't have any specific examples, but the thought was that some features (e.g. Promises) may also imply additional support APIs beyond the root class. With an engine guard feature, all APIs supported by the newly expanded area that support a 'guarded feature' (e.g. APIs that make use of or return a Promise) might also become available.

But, yes, this would be a secondary feature implementation - to be implemented once the core mechanisms of specifying a set of target runtime environments were in place. :)

@ericdrobinson ericdrobinson referenced this issue Jun 21, 2018
0 of 15 tasks complete
@wizzard0

This comment has been minimized.

Copy link

commented Jul 29, 2018

i'd support this, specifically for targeting non-browser environments like Duktape, Espruino, Adobe ExtendScript, etc. Or "weird" browsers like embedded devices stuck with old Android versions (e.g. car multimedia).

An obvious problem is a combinatorial explosion of cases to test if we try to support "downlevel to run on Duktape + nw.js".

Possible long-term benefit is that breaking down engines into feature sets may enable easier maintenance for new features, though.

@kitsonk

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2018

@wizzard0 those using esoteric runtimes can always --noLib and provide something else that is more suited for their runtime. For embedded systems, rolling your own lib is likely a better long term solution anyways. It would be really hard to force TypeScript to keep up with n esoteric runtimes.

@ericdrobinson

This comment has been minimized.

Copy link

commented Jul 29, 2018

@kitsonk I'll point out that this proposal isn't about asking TypeScript to keep up with esoteric runtimes. As you point out, you can already pass the --nolib option for specific use cases. This doesn't help people writing code for multiple runtimes, however. In the most common case, this is the idea that each of today's major browsers do not support standards-compliant runtimes - each has subtle differences, especially when you deal with browser versions. As mentioned in the proposal, TypeScript's core libs are to a degree generated from Microsoft Edge and has lots of its own issues as a result.

The idea is that each runtime provider [the esoteric included], could provide their own declaration files. This wouldn't be anything special and would not warrant a new feature proposal by itself. The difference is that the proposal outlines a suggestion to take a union of several different type declaration libraries based on setup.

Imagine that you're a web developer working on an Open Source library. You want to make sure (or at least have some assurance) that the code you write will work in a specific set of browsers and their versions. What's more, you want to ensure that the code (or some portion thereof) can be run in NodeJS. Today's TypeScript may help quite a bit but you can still end up using APIs that simply aren't supported in one runtime or another. Wouldn't it be nice if you could tell TypeScript which runtimes you wanted to target and it would simply adjust the types it resolves (and, therefore, the errors it throws) as a result?

Support for esoteric engines would come for free, provided the engine developers provided whatever necessary [versioned] libraries would be required to support such a system.

As for downlevel compilation, I think that should be raised as a separate issue, especially as TypeScript language services can be leveraged directly in pure JavaScript files as well.

@dgoldstein0

This comment has been minimized.

Copy link

commented Jul 30, 2018

ideally:

  • each environment (browsers, nodejs, other obscure things) could have a declaration file describing what's available in that environment in terms of globals & builtins
  • these declarations could be managed the same way as @types modules

In which case the core problem becomes adding a type system feature to combine the different environments' type declarations.

re downlevel compilation, this task does seem to have talked mostly about the type system implications of multiple environment support; so I think I agree with @ericdrobinson that whether to support any changes to the typescript output should probably be a separate issue; so far, I've found using es5 output to be suitable for my use cases.

Perhaps it's time to start proposing the necessary type system changes?

@tomwayson

This comment has been minimized.

Copy link

commented Sep 22, 2018

For my own purposes, I'd be fine w/ exactly what babel-preset-env does. In the babel community it's effectively an anti-pattern at this point to target anything other than whatever environment you intend to support (i.e. to target specs).

In many of the responses to this and related issues (#20095, #23156 (comment)) I hear the notion that it's not reasonable to expect TS to keep up what browsers support, and I totally agree.

Which made me think, how does babel able to keep up with the evolving browser landscape? Turns out they dont.

So, I wonder, is it possible to TS to implement or to extend TS w/ something that works just like that?

@dgoldstein0

This comment has been minimized.

Copy link

commented Sep 25, 2018

I think there are two threads to this conversation:
a) should typescript auto-inject missing polyfills?
b) should typescript typecheck code against definitions for the browsers it plans to run in?

These actually seem to be disjoint approaches to the problem of "how do we handle nonstandard parts of our environment" - e.g. do you polyfill missing features (option a), or do you disallow their usage (option b)?

I'm arguing for (b), and I believe @ericdrobinson is on the same page. @tomwayson you seem to want (a). Which is fine, but it's a different use case - e.g. I don't want random polyfills showing up in my compiled code, as they tend to add unexpected bloat and some do sketchy things to builtins (e.g. the closure compiler's WeakMap overrides Object.freeze & Object.seal); I want to use types to restrict ourselves to only what's implemented natively, plus what we've explicitly decided to polyfill.

In particular, what I'm hoping for is infrastructure to be able to plug in type declarations from arbitrary browsers - so I can get typings that are IE11|Edge|Chrome|Firefox and not have to manually manage a typings file that is the merge of all those browsers. And then hopefully the community will manage types for browsers. I think that's a reasonable compromise between "typescript doesn't want to manage declarations of all browsers" and "everyone writes their own typings" that seems to be recommended in #23156 (comment)

@ericdrobinson

This comment has been minimized.

Copy link

commented Sep 25, 2018

I'm arguing for (b), and I believe @ericdrobinson is on the same page.

That is correct.

In particular, what I'm hoping for is infrastructure to be able to plug in type declarations from arbitrary browsers - so I can get typings that are IE11|Edge|Chrome|Firefox and not have to manually manage a typings file that is the merge of all those browsers. And then hopefully the community will manage types for browsers.

That is precisely the idea. The main proposal covers the why (the problem) and the what (a mechanism by which TypeScript may be engineered to help solve the problem).

When it comes to sourcing the data that is actually used by the "mechanism" (the "Engine Library Files" [ELF]), I did not specify that the TypeScript team should take on the impossible task of providing/sourcing this data themselves. To my mind, the strongest solution would be the last one presented in the proposal: "Request engine vendor buy-in and support." This would mean that engine developers are responsible for delivering type declarations for their products. See:

Engine Vendor Responsible for ELF
V8 Google
Chrome Google
Chakra Microsoft
Edge Microsoft
JavaScriptCore Apple
Safari Apple
ExtendScript Adobe

[Note: There is nothing magical about the term "Engine Library Files". That is simply a term I used to keep the conversation focused. The term is 100% equivalent to "Type Declarations for specific JS engines".]

There is actually nothing stopping those vendors from doing so today: they could generate engine and release-specific Type Declaration files and TypeScript would happily consume them. What is not currently possible is the ability to specify multiple target engines (Type Declaration files/ELFs) and select only those APIs that are available across those specified engines. Which... is what the proposal is all about :)

@Timer Timer referenced this issue Oct 4, 2018
11 of 11 tasks complete
@wrumsby

This comment has been minimized.

Copy link

commented Dec 26, 2018

@ericdrobinson re:

Browserlist is limited to Web Browsers.

That's no longer true. As per "browsers" Browserslist can also target Electron and Node versions which I suspect might cover most of the "other" targets a TypeScript project may have.

@ericdrobinson

This comment has been minimized.

Copy link

commented Dec 27, 2018

That's no longer true. As per "browsers" Browserslist can also target Electron and Node versions which I suspect might cover most of the "other" targets a TypeScript project may have.

@wrumsby That's neat! While not a comprehensive list of engines, it's certainly a strong one. There's still the other issues I brought up in that section, though. :/

@wrumsby

This comment has been minimized.

Copy link

commented Dec 28, 2018

One thing I have done is to run ESLint with eslint-plugin-compat over the JavaScript code generated by TypeScript. With this plugin you can specify polyfills so a workflow can be:

  1. compile TS to JS
  2. lint the JS using eslint-plugin-compat + browserslist
  3. for any missing polyfills add these using https://polyfill.io (for instance) and specify these in the settings.polyfills section of your ESLint config

(Ideally you would describe the polyfills in a way that both eslint-plugin-compat can use them and they're automatically added to the page using https://polyfill.io)

@simon-robertson

This comment has been minimized.

Copy link

commented Jan 31, 2019

TSC support for browserslist would be very good to see.

At the very least, using browserslist can reduce the number of polyfills that have to be injected into distributable files. If we only need to target the last 2 Chrome, Edge, and Firefox versions, the number of required polyfills would be dramatically reduced, and would result in smaller (and potentially more performant) runtime code.

Adding a path to a standard browserslist file, to the TSC config, should be sufficient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.