Allow testing another TypeScript version

[SamVerschueren]

I recently thought about this, and not sure if this is something we want. But I can assume that some people want to know if the type definitions work for TypeScript >=3.1.0 for instance.

I think it would be possible if we dynamically installed TypeScript depending on engines.typescript or something in package.json.

One thing to figure out though is that we need two private methods from within TypeScript which aren't exposed. I worked around this by shipping TypeScript myself with two extra manually added lines. So if we want to do this, we need to find a way to dynamically add them. Don't think it's impossible, just makes it a bit harder.

Don't think it would be a good idea to ship this library with 10 TypeScript versions 😂.

[ExE-Boss]

Which two private methods do you need?

I think it’d be a better idea to file a feature request on the TypeScript repository, rather than maintaining your own fork.

[SamVerschueren]

Read from here to the bottom. You can't say I didn't try. It's a PR of 2016.

[pzuraq]

Definitely would like to see this feature, this is one of the main things we're looking to test. Would it also take into account typeVersions? Ideally, we would write a single test file, and verify that it worked for all of our supported versions of TS, including older type definitions if we needed to.

[SamVerschueren]

The idea would be to be able to set typescript: '>=3.2' and then it will test it for versions 3.2, 3.3, 3.4, 3.5, 3.6 and 3.7.

[sindresorhus]

While I don't need this personally, I can see how it can be useful for some projects. However, I don't think you should bundle TypeScript versions. The default should also stay, that tsd always uses the latest minor version of TypeScript (non-bundled).

[SamVerschueren]

Yes, while bundling is the easiest solution, I don't want to do it either so I will explore the dynamic path.

I agree that the default should always be the latest minor version, 3.7 at the moment.

[phryneas]

Couldn't you release all minor TypeScript-versions as separate patched packages?
Something like @tsd/TestableTypeScript3.3 etc.
Then users could decide to install those they want to test against and provide them in a config file - tsd itself could always have the latest version as a dependency and use that per default.

[ExE-Boss]

What about @tsd/typescript‑3.3, or even @tsd/plugin‑typescript‑3.3?

[SamVerschueren]

Although releasing a custom typescript version might be a good idea, I don't think I would be able to keep up with releasing patches etc.

That path would be easier to do though then dynamically downloading a certain version, inserting code dynamically and running the tests... Although the latter would be much more maintainable from my side.

[phryneas]

@ SamVerschueren would it help if I scripted a CI pipeline that would auto-release those packages using github actions or travis?
Would only require maintaining some .patch files (and those probably wouldn't change very often), as any other solution probably would, too :)

[likern]

How is this done in tslint? I saw they can check against multiple typescript versions.

I'm also not going to use this feature in near future, but see this as very interesting and helpful, especially in a long run.

[phryneas]

@SamVerschueren I was a little bored and followed up on my idea of automating this - take a look over here: https://github.com/phryneas/testable-ts/commit/6baa0ef9493e4f8ee5578f12f9081cd65639af50/checks?check_suite_id=373940820

Essentially, the repo has a github action that adds the isTypeIdenticalTo: isTypeIdenticalTo line to checker and renames the package to @testable-ts/v3.7 etc. for all recent versions from 2.5 to 3.7.

I explicitly chose the path of multiple versions as package names, as this will make it easier to install multiple of those in parallel - the npm: syntax is not available in all package managers afaik.

The whole thing is pretty low-maintenance - running ./makeVersion.sh 3.7.4 generates a patch for version 3.7.4 and spews out the lines that should be added to default.nix.
I'd be happy to maintain that repo to contain at least one version of every minor, which should be enough for testing.

The only thing I haven't done yet is actually publishing those packages to npm, currently they are only available as Artifact download from the github action.

Would love to hear your feedback (and if you need any additional methods exposed) before I take that step :)

[phryneas]

Okay, now they're already published as -alpha.0 versions. I played around a little more and ended up with #62

[Ethan-Arrowood]

Could you make the TypeScript version tsd uses pluggable? So like add a new CLI option --ts that allows the user to pass a path to their typescript install or something more dynamic (and probably similar to what you have proposed) would be if they specified a version number and tsd would fetch that version of typescript when executed (could use caching for subsequent runs).

[delvedor]

This feature would extremely useful, especially given that TypeScritp drop breaking changes in minor releases, which makes it hard to be sure that a feature works well in different versions.

[delvedor]

@SamVerschueren I've just found this package, and by its description, it looks like it could help with this issue, what do you think?

https://www.npmjs.com/package/byots

[chriskrycho]

We're looking at the space for how we can validate our types across the Ember ecosystem, and while I'd prefer to use tsd, this issue is basically a hard blocker for us: one of our core constraints is that we need to be able to test published types against different versions of TS, so that we can maintain Ember's strong backwards-compatibility guarantees. I have bandwidth and can help dig in if there's interest in changing this!

Related: microsoft/dtslint#284 (comment)

[papb]

Hi @chriskrycho, may I suggest you take a look at the expect-type package? I am personally planning to move from tsd to it.

[chriskrycho]

That was super helpful, and we’re currently going exactly that way. It works extremely well. Thanks!

[SamVerschueren]

@delvedor It doesn't expose the isTypeIdenticalTo internals which we need.

@phryneas Your project looks very nice. I have a good idea on how we could set this up to allow us to spawn multiple tsd processes in parallel for different TS versions. Would you be interested in continueing the project and maybe automate it even more so that on every TS release, we also do a new patch release?

[phryneas]

@SamVerschueren I've added version for 3.8.3 and 3.9.5.
I wouldn't automate it much further as the autogenerated patches might not always be perfect going in the future if something is renamed.

But I've adjusted the script a little and now it comes down to me just running

./makeVersion.sh 3.8.3 
./makeVersion.sh 3.9.5 

and then sight-checking & committing the generated files. I think that's pretty effective ^^

(and of course I accidentally just now committed it without the extra suffix = "-alpha.0";... I hope the packages are fine 😄 )

[SamVerschueren]

Thanks to the work of @phryneas I am already able to do something like this.

In my type definition, I used the nullish coalescing operator which was introduced in TypeScript@3.7. The older versions report issues on that, which is pretty slick!

It still need some more work though. Let's say it still in proof-of-concept phase :).

[voxpelli]

I would love it if one could specify to use the TS version installed outside of tsd if one has such a one.

Then one could run tsd as part of something like: https://github.com/sindresorhus/type-fest/blob/951a21130bfbd3b605a0b28f9677fe1874653b5b/.github/workflows/main.yml#L23-L42

Would it maybe be possible to use import-local or something similar to achieve that?

[SamVerschueren]

@voxpelli the problem is that tsd requires a patched TS version...

[tommy-mitchell]

@SamVerschueren did you ever upload that proof of concept to a feature branch or anything? I'd like to flesh it out and update it with @tsd/typescript.

[skarab42]

@voxpelli the problem is that tsd requires a patched TS version...

FI: I created a hybrid solution unleashed-typescript for my vitest plugin, not ideal but it avoids maintaining a custom version of TS and it allows to have a version of TS synchronized with your environment. I think it's possible to modify it a bit to make several patched versions of TS cohabit, a kind of package manager dedicated to TS.