Repository organization

[sunfishcode]

Currently the main directory organization in the WASI repository looks like this:

phases/

• ephemeral/
• snapshot/
• old/
  • snapshot_0/

I've recently talked with several folks about how to better organize proposal repos, and we came up with several ideas which seem to fit well together which I'd like to discuss:

• Switch to having stable URLs for snapshots rather than maintaining a snapshot/ pointing at the current snapshot and old/ add old snapshots. We can have a link in a README file pointing to the latest revision if that would help, but being able to refer to a specific snapshot URL would be useful. This was suggested in Consider giving the current snapshot a number #261, and I now think it makes sense to do in combination with the other ideas here.
• Replace numbered snapshots ("0", "1", etc.) with date-based snapshots (eg. "2020-12" or so)
• Replace the contents of the ephemeral/ directory with a proposal template. The way this will work is:
  • Development will move into the proposal repos, such as wasi-filesystem.
  • When we create a new proposal repo, it'll get a copy of the template, and the people working on the proposal can use that if they wish, but they can also choose to organize development however they see fit.
  • We can also then rename this to proposal-template/ and move it out of phases/.
• Add a new standard directory, which in the main WASI repo here will be empty until some proposal reaches stage 5, but which in the proposal repo forks will contain the most recent version of the proposal.
• Add a script, similar to update-testsuite.sh in the wasm testsuite repo, which copies the standard/ contents from the proposal repos into a date-based snapshots/ subdirectory.

So the new structure would be:

• phases/ goes away
• add proposal-template/ at the top level contains a template for new proposals
• snapshot/
  • date-based snapshot directories (eg. 2020-12)
• standard/

This will, at last, be part of a clear path forward for releasing a new snapshot using the proposal repos. Thoughts?

[syrusakbary]

I'm not super fan of date-named versions since date doesn't convey meaning.

However if date-named versions are the way to go I'd prefer using a convention such as Dec2018 rather than 2020-12 (following a similar nomenclature than the GraphQL spec, for example).

[sbc100]

One good thing about 2020-12 it sorts. I prefer it over Dec2018 but maybe I'm biased because I'm very used to ubuntu's naming scheme of 16.4 which is very similar to 2016-12.

What extra meaning might we want to covey with out snapshot releases versions? At least date-based scheme is strictly more meaningful than monotonic integers.

[jedisct1]

All these changes would greatly improve the directory structure and scale better than the current organization.

Y-m-d dates have the advantage of being immediately lexically sorted, but they don't convey anything about the difference and compatibility between two versions. Was semver ruled out?

[syrusakbary]

What extra meaning might we want to covey with out snapshot releases versions? At least date-based scheme is strictly more meaningful than monotonic integers.

It's a bit easier to assess (breaking) changes on v1/v1.1/v2 vs 20.4/20.8/20.12

[pchickey]

Date-based versions are useful for communicating the WASI snapshot used by toolchains and libraries without confusing it for the actual version of that toolchain or library.

Another closely related problem it solves is the stagnation of snapshots. We have had snapshot 1 out for a year now without committing to a date to release the next one, because we don't have tooling that can handle ephemeral right now. We need to get unstuck from this situation by putting time into the tooling, but to prevent this from happening in the future, a release train schedule will keep everyone honest on only merging changes once the tools actually support them. Its possible for a release train to use monotonic integers, but if the goal of those integers is just to communicate a snapshot in time and not a semantic version it makes more sense to just use a date.

According to semver, every change we have made in WASI so far is a major version breaking change, and nearly every change I can think of for the future is one as well. So, using semver is the same as sticking to the current monotonic integer scheme.

We have put a lot of thought into describing the differences and compatibility between versions in mechanical ways, but one big problem is that both the WASI interfaces themselves, and the technology used to specify those interfaces (witx, proposals like reference imports, interface types, etc) are both in flux. Therefore, we concluded that the best we should do, until the technology to specify WASI is stable, is provide human-readable changelogs and use doc comments to describe similarities and differences to prior versions.

Since we don't yet know what the end state of this process will look like, lets defer figuring out how to describe the semantic differences to a machine (via semver, and other mechanisms) until we're on more stable ground. It will be at least a year, probably 2, before WASI is based on top of a stage 4 interface types proposal. At that point, semantic differences become a lot more meaningful and useful.

[sbc100]

Oh see, you are both talking about compatibility between versions. That issue I think should be considered separately. There has been fair amount of discussion on that issue in past: See #2.

[badeend]

If date-based names end up winning, please consider padding the months and days to 2 digits, eg. "2021-01" instead of "2021-1". This ensures that the alphabetic order == chronological order.

[sunfishcode]

Yes; this is also the ISO-8601 YYYY-MM format.

[sunfishcode]

wasi-filesystem, wasi-clocks, wasi-random, wasi-poll, wasi-classic-command, wasi-handle-index, and wasi-io now all have PRs updating them to be WASI forks with patches on top.

[sunfishcode]

The proposals have now been moved into their own repositories. There is a proposal template for new proposals. The central WASI repository has been reorganized and significantly simplified.