Explore enabling type acquisition for web TS Server

[mjbvz]

Follow up on microsoft/TypeScript#39707

Overview

We now have TSServer running in web environments, but it is currently limited to single file intellisense. It would be nice if we could also offer some IntelliSense for third party packages that are imported in the current file.

If a user is trying out react for example, we should be able to offer IntelliSense for React and ReactDOM:

//@ts-check

import * as React from 'react';
import * as ReactDOM from 'react-dom';

class HelloMessage extends React.Component {
    render() {
        return (
            <div>
                Hello {this.props.name}
            </div>
        );
    }
}

ReactDOM.render(
    <HelloMessage name="Taylor" />,
    document.getElementById('hello-example')
);

[mjbvz]

Lots of open questions about how this should behave and how we can implement it:

• Do we try looking in the user's package.json or at their tsconfig when determining which types to use?

• Do we handle packages that bundle their own typings?

• How far should type acquisition go? If one type package references another, do we pull that in too?

• Would this work differently in JS files vs TS files? Type acquisition works differently for the two on desktop, but on web I think we'd want the behavior to be more similar

• Could we use a service like unpkg for this? Or could we stand up a more specialized service that just service up a single types file for a given package?

• Can the type acquisition be async like it is on desktop?

[orta]

Could we use a service like unpkg for this? Or could we stand up a more specialized service that just service up a single types file for a given package?

Having NPM provide a CDN like how unpkg works would be a nice solution for this (because I have this same issue in the TS playground)

[orta]

I recently re-wrote the ATA for the Playground ( microsoft/TypeScript-Website#1997 ) in what I think is a pretty good conceptual pattern:

• TypeScript can probably already give you a list of dependencies it knows about from the package.json, and the source files it has read (in the playground I don't have this, so I use ts.preprocessFile to extract that)
• You make an API call which gets a list of all the files in each module tarball (I used jsDelivr's) from which you can figure out the list of .d.ts files you need to download, or that you need to check DT.
• If you need to check for @types/module and grab the .d.ts files for that module
• Loop through all the downloaded .d.ts files and recurse grabbing their modules etc

It's not exactly like that in practice (you need the version number before you can make that API call to jsdelivr) - but that's good enough and you can read the implementation linked above for more details. The implementation of the TypeScript Playground ATA is built for third parties, so you could be a consumer of @typescript/ata when that PR lands.

An ideal state for VS Code which would keep this entirely first-party, is that the npm registry adds two APIs like:

• https://registry.npmjs.org/npm/:version_or_tag/contents which returns some sort of file list
• https://registry.npmjs.org/npm/:version_or_tag/file/package.json which returns a CDN'd file from the tarball (in this case the package.json)

Or vscode creates an API which proxies these APIs to jsdelivr (they're real nice folks, I can make an intro if needed) which allows for other implementations behind the scenes in time.

[MartinKolarik]

Or vscode creates an API which proxies these APIs to jsdelivr (they're real nice folks, I can make an intro if needed)

We are already watching - ping me any time if needed 😄

[orta]

The Playground ATA re-write is now available on npm at @typescript/ata and deploying to the web as we speak.

[mjbvz]

microsoft/TypeScript#46862 Adds a draft implementation of this but I definitely need some help from the TS team to get it into a shippable state

[mjbvz]

After discussion, we'd like to move the core auto import implementation to VS Code

@orta I looked into using preProcessFile to get all the imports but it looks like this isn't currently exposed over the TSServer protocol? Is that correct?

After thinking about this more, I am also wondering if using preProcessFile is too low level. We can debounce how often we end up calling preProcessFile to avoid calling it on every key stroke, but it feels like it would make more sense if TS could tell us what auto imports it needs and then VS Code could then go and resolve them

[mjbvz]

@orta @amcasey I have a basic prototype now running on the VS Code side and can use this to load global typings such as jquery

The problem I'm running into now though is telling TS that jquery in import 'jquery' maps to a given file the VS Code has opened against the TypeScript server

Here's how the files are currently opened after ATA has completed (I'm using the paths that TS server sees):

^/memfs/sample-folder/file.js — This is the file that the user has opened up

import * as $ from 'jquery'

Then we have a list of jQuery d.ts files that VS Code has opened up with the correct content:

• ^/ts-typings/node_modules/@types/jquery/JQueryStatic.d.ts
• ^/ts-typings/node_modules/@types/jquery/JQuery.d.ts
• ^/ts-typings/node_modules/@types/jquery/misc.d.ts
• ^/ts-typings/node_modules/@types/jquery/legacy.d.ts
• ^/ts-typings/node_modules/@types/jquery/index.d.ts

All of these are in-memory files so TS Server cannot read them directly. I believe we also have to keep the different scheme (memfs vs ts-typings) as part of the implementation on the VS Code side

What type of API do you think we need to get typescript to understand that import 'jquery' should resolve to ^/ts-typings/node_modules/@types/jquery/JQuery.d.ts?

[orta]

There's a similar tension in doing the full TypeScript compile in the ts-vscode playground - I wonder if a reasonable strategy lays in making the TypeScript web server smarter, and keeping some of the file content inside that. As it exists, there is no way to influence what is available to that TSServer:

    function createWebSystem(args: string[]) {
        Debug.assert(ts.sys === undefined);
        const webHost: WebHost = {
            readFile: webPath => {
                const request = new XMLHttpRequest();
                request.open("GET", webPath, /* asynchronous */ false);
                request.send();
                return request.status === 200 ? request.responseText : undefined;
            },
            fileExists: webPath => {
                const request = new XMLHttpRequest();
                request.open("HEAD", webPath, /* asynchronous */ false);
                request.send();
                return request.status === 200;
            },
            writeMessage,
        };

I think somewhere there needs to be a vfs map which the tsserver prioritises for sys lookups for dirs/files and it might need to be insider the web server. That could be controlled via additional tsserver actions/events.

What type of API do you think we need to get typescript to understand that import 'jquery' should resolve to ^/ts-typings/node_modules/@types/jquery/JQuery.d.ts?

I think that might get tricky with the schema change, I think you'd need to hardcode that somehow into the TS web server code somehow (for examples it is kinda safe that all paths with @types could be treated that way)

[mjbvz]

@orta Thanks! Not sure I understand the first part though.

With my current approach, on web all TS files are opened against as in-memory resources. This means that TS knows the file content but should not try to read these files from disk. Are you proposing a different approach where readFile would be needed?

For the second part, who is the best point of contact to discuss this with? If needed I think we could rewrite the paths that VS Code sends to TS Server to make the project system happy, but I'd like to explore a proper solution first

[mjbvz]

My current proposal would be a new TS Server api that let a client explicitly pass global typings files to the TypeScript Server. This could be used both to implement ATA, and also for applications like Office Scripts where you have a VS Code extension that wants to augment the global typings in JS files

Some of this discussion also gets into our discussions around virtual file system support for vscode.dev

One idea: could a client for example tell TS Server about a type root directory and then the server would simply treat any files opened under this directory as typings

From a protocol point of view, this would look something like:

1. Client passes a root directory for typings to TS Server: configure { typeRoots: ['^/ts-typings'] }

2. Client then opens files under this types root: updateOpen { path: '^/ts-typings/jquery/jquery.d.ts' }

3. When resolving imports, the server then looks in the types root.

   Since the server only has an in-memory representation of these read files/folders, the server may have to construct its own little virtual file system representation to implement this

[orta]

(sorry, been a busy few weeks)

I'm not sure that you need to special case the vfs as only being typings specifically, for the playground ATA we emulated having files in node_modules/@types/jquery/index.d.ts which meant it was picked up by the default behavior in TypeScript

If you have a generic system to put vfs files into the web tsserver's sys then you can put files into the right place and it would automatically be respected by the rest of the tsserver

[mjbvz]

Opened microsoft/TypeScript#47600 to continue discussion of generic virtual file system support for TS Server. Using this, I believe we could implement ATA by writing the acquired typings to the correct location in the virtual file system

[ameerthehacker]

is the intellisense between typescript files also in scope? like will I get intellisense for a hello function declared in a different file when I import it?