A way to always import the correct(*) std version and runtime compatibility

[rsp]

Consider this scenario: You are writing a 3rd part module. It uses std modules. What is the correct std version that you should use?

The obvious answer would be to use the same std version as the version of the runtime, but you don't know what version of the runtime will your library run under because this is outside of your control.

(*) Note that the "correct" version is not obvious right now even if you were to hardcode the version yourself - is the correct std version the one that your module was developed with, no matter what Deno runtime is currently running (potential Deno runtime-std incompatibility)? Or is the correct std version the one that is the same as the current version of the runtime, no matter what version the library was developed with it (potential module-std incompatibility)? (Either or none of those answers can be true depending on the convention/discipline of developing the std modules but it has to be considered explicitly.)

Something to consider as well: Is there a risk of importing multiple versions of std modules by different dependencies of the same program? Do we want to avoid that or should this be a common practice? (If the std version is hardcoded into each dependency then it will be more than likely.)

This is a question that cannot be avoided, because if you publish a module that uses std (as probably most of nontrivial modules will), you need to choose a version number. If you hardcode the current version of Deno that you currently have, then when a new version od Deno (and std) comes out, you can either:

1. Keep the version in the std imports
2. Update the version in the std imports

In both cases, either newer versions of Deno will use older versions of std, or older versions of Deno will use newer versions of std, both of which can lead to compatibility problems.

As a more general case, the question is whether we should:

1. Import the earliest std version that the module is compatible with
2. Import the latest std version that the module is compatible with (hopefully, the current one, but not necessarily)
3. Import the exact std version for this runtime (cannot be hardcoded)

(This question is important for both hardcoding the version in the import and using a mechanism of dispatching to the correct version.)

Semver doesn't actually let us do what we need here, which would really be something like this:

1. define what deno/std version range(s) the module is compatible with
2. use the exact version of the current runtime if it is in the range of compatible versions
3. fail if the current runtime is not in the range(s)

Normal semver semantics will only allow us to:

1. define what std version range(s) the module is compatible with
2. use the latest compatible version of std, potentially different than the version of Deno runtime (it can both be newer or older, depending on the ranges in imports and the current runtime)

Currently we can probably expect most of Deno early adopters to use the latest version of Deno and update as soon as a new version is available, but in the future we will want our libraries to be compatible with multiple versions of Deno and std modules, and to also cooperate with other libraries that also use std.

The current versioning scheme is time based with predictable version schedule. Hopefully, when the 1.x is out, it will follow semver rules so we can be sure that if our module is compatible with deno/std 1.5.x then it may not be compatible with 1.4.x but it will certainly be compatible with 1.35.x (unless there was a mistake with a version bump somewhere, but in principle this should at least be a reasonable expectation), but I would like it to use std 1.20 when running on Deno 1.20 and 1.35 when running on Deno 1.35 and the question is: How can this be done?

Making sure that the std and runtime versions match will make it easier to develop both std and the runtime. I may be wrong but as far as I see there is an implicit assumption that a certain version of std does not have to be compatible with older versions of Deno, but it would be useful to be explicit about:

1. should std modules' exposed API be backwards compatible?
2. should new std modules be compatible with old Deno runtime?
3. should old std modules be compatible with new Deno runtime?

For the problems above I see few solutions, all with their own problems:

1. bundle std with Deno and always use the bundled version (not desirable)
2. use a default import map for the correct std version as described here (not a URL that can be dispatched e.g. in the browser, though using a Deno std module would not work in the browser anyway so this may or may not be a problem)
3. use a different logic when dispatching std version than with other semver ranges (not standard)
4. make sure that all std modules are always forward compatible with future runtimes (not practical, all std versions would have to be updated indefinitely for incompatible changes in the runtime and use a version compatibility suffix)
5. make sure that all std modules are always backwards compatible with previous runtimes and their API is always backwards compatible with previous versions of std modules (this is what people are de facto assuming when they do import { test } from "https://deno.land/std/testing/mod.ts"; which imports the latest commit on the master branch of the std)
6. use dynamic imports (not as nice for statical analysis and you need --allow-net to use it)
7. freeze Deno's internal API when it reaches 1.x and only add backwards compatible features
8. something else?

I've read few discussions about std module aliasing and bundling but I didn't see any ideas on how to handle it for the future when we will have people using dependencies that may not always be updated as often as std, and running on unknown versions of the runtime.

Has there been any brainstorming about this already?

Related issues: #8, #288, #1397, #2288, #3268
Update: #3395

[axetroy]

Not sure if this will solve the problem

deno/cli/flags.rs

Lines 14 to 18 in 2cd22b5

	macro_rules! std_url {
	($x:expr) => {
	concat!("https://deno.land/std@v0.23.0/", $x)
	};
	}

 macro_rules! std_url { 
   ($x:expr) => { 
-     concat!("https://deno.land/std@v0.23.0/", $x)
+     concat!("https://deno.land/std@v", env!("CARGO_PKG_VERSION"), "/", $x)
   }; 
 } 

[ry]

#3446

[lucacasonato]

std and cli are synchronized once again. not the same versions, but they release at the same time.

[filmaj]

I was hoping there would be more discussion on this issue than what I see.

Given this statement on the std lib README: https://deno.land/std@0.180.0#releases

To check compatibility of different version of standard library with Deno CLI see this list.

... does this mean that any libraries / frameworks we build on deno, we should take into account the minimum version of the deno runtime we wish to support, and based on that, using the above versions.json, stick to the specific version of the stdlib specified under cli_to_std?

A bit more guidance would be appreciated as we build our application ecosystem tooling on top of deno 🙏