Repository reorganization for multiple Inform versions

[juhana]

Now that the latest Inform version has dropped, it would a good time to reorganize the extensions repository so that it works for both 6M62 and 10.x. Some extensions aren't compatible with 10.1.0 and syntax changes in 10.1.0 mean that new extensions made for 10.x won't be compatible with 6M62 anymore.

The proposal is this:

• Move all extensions to a new top-level folder called 6M62
• Create another top-level folder called latest and copy all extensions there that are confirmed to compile without errors in 10.1.0
• If an extension isn't compatible with either of them, create a folder for the last version it is compatible with and move it there
• When someone modifies an existing extension to be compatible with 10.1.0, they copy it to the latest folder and change the release number to use the new semantic versioning scheme that has the major version one larger than the old release number. E.g. release 4 becomes 5.0.0.

This way everyone using the repository knows which extensions are compatible with the version of Inform they're using and extensions can have both 6M62 and 10.x compatible versions available at the same time.

Does this sound reasonable? Any suggestions/objections/discussion?

[curiousdannii]

I had been thinking of just making a branch for 6M62, that would snapshot the repository at the present time before modifications are made for 10.1, as well as allowing further changes to be made in the future if anyone keeps working on 6M62 versions. The master branch would be for 10.1. If Inform 11 comes out in the future we can make a branch for it if that would help.

Advantage:

• Makes things simpler when telling people how to get extensions for the latest version of Inform

Disadvantage:

• Makes older version(s) more hidden

[juhana]

Right now there are more 6M62 users but that'll change at some point, it's just hard to predict when that might happen. Different branches could work, there can be a big link to the 6M62 branch in the readme so people will find it.

[curiousdannii]

More discussion here: https://intfiction.org/t/updating-extensions-in-the-github-repo-for-inform7-v10/55849

I think I'm leaning more towards folders than branches now. Nope, back to branches, but branches that are automatically populated by a Github Actions script.

[uecasm]

Which way is better may also depend on how easy it is to integrate with Inform; the previous layout was nice in that you could just clone or export the whole thing to your I7 user Extensions folder and it would Just Work™. Branches could do the same thing but only for a single version of Inform at a time.

I haven't looked at I10 yet -- does it have a separate Extensions folder from the 6M62 one or do they both try to read the same folder? If they are separate but not adjacent then you can separately clone each branch; if they're separate and adjacent then a single branch with subfolders might be nicer; if they're the same folder then either one is painful if you want to swap back and forth but branches is probably the least pain.

[david-a-wheeler]

I think this should NOT be done for most extensions. If the extension is written to work on arbitrary Inform versions - and would hope most are - then I think most shouldn't be split this way. If they're split by version then you'll have multiple copies of the same thing, which will inevitably going out of sync. If it's hard to write an extension that works across Inform versions, then let's fix that.

If an extension only works with a certain range of extensions, then I think that instead there should be a standard marking on or about the extension. That way, modifying the extension will happen only once (on its main version), and it can appear in lists of "what works on your version".

Of course, there are extensions that only work with a specific version. For those cases, a version-specific subdirectory would make sense, but only those. In those cases, "latest" is not a good name, because what is latest will change over time. If it's going to be split up by Inform7 versions, I suggest using a version number for each subdirectory.

[awlck]

If the extension is written to work on arbitrary Inform versions - and would hope most are - [...] If it's hard to write an extension that works across Inform versions, then let's fix that.

There have been a number of breaking changes to the language itself over the past few years (not to mention the extensions that rely on what are effectively implementation details), so we definitely have to account for those. And looking into the future, I don't think that I7 going open-source should be taken as a promise of "no more breaking changes" either.

If an extension only works with a certain range of extensions, then I think that instead there should be a standard marking on or about the extension.

There currently isn't a standard (i.e., recognized by Inform) way of denoting extension compatibility.

[david-a-wheeler]

There have been a number of breaking changes to the language itself over the past few years (not to mention the extensions that rely on what are effectively implementation details), so we definitely have to account for those. And looking into the future, I don't think that I7 going open-source should be taken as a promise of "no more breaking changes" either.

Fair enough :-). But many programming languages change over time. It should be possible to query versions & make variations based on that.

If the top level organization is by version#, then extensions that work on many versions will have many copies, forking each one. That's also unpleasant to deal with over time.

[dstelzer]

Is there really no way to query the I7 version number from within an extension? It's been a while since I worked with this directly, but I dimly remember I6 #defining a constant for the compiler version and library version that could be checked.

[awlck]

I haven't looked at I10 yet -- does it have a separate Extensions folder from the 6M62 one or do they both try to read the same folder?

The compiler has no "awareness" of these things, it just uses whatever paths are set through the -internal and -external options. Separating these folders out by version would be up to the IDEs. (If you use Inform from the command line, you can of course use whichever directories you want.)

Is there really no way to query the I7 version number from within an extension? It's been a while since I worked with this directly, but I dimly remember I6 #defining a constant for the compiler version and library version that could be checked.

I'm not aware of any. You can mark of sections as "for Glulx only" or "for use with/without" another extension, but not as "for some particular compiler version".

[juhana]

But many programming languages change over time. It should be possible to query versions & make variations based on that.

If a programming language's syntax changes, it's no longer possible to query versions inside the code itself because it won't compile anymore. The latest Inform's extensions (can) use semantic versioning so earlier compilers will throw a compiler error right on line 1 no matter what you do. It's likely that in the future there's going to be even bigger changes in how extensions are structured so having extensions be widely compatible with previous versions isn't a realistic goal.

Not that this would be a problem, either – I don't expect that people will be interested in maintaining extensions for multiple Inform versions. They haven't been so far, even though there have been much more radical language changes in the past. The storage for 6M62 versions would for all practical purposes be an archive.

I also think that it's important that any reorganization doesn't come with extra rules that extensions authors would have to abide by. The only thing that the extension author should be concerned about is that the extension compiles in the Inform version it's intended for and it's getting uploaded to the correct place, whatever it's going to be. Anything more strict than that would create either an unnecessary barrier of entry or maintenance load for the famous "someone" to keep the extensions in the correct form.

[david-a-wheeler]

If a programming language's syntax changes, it's no longer possible to query versions inside the code itself because it won't compile anymore.

Not true, I've been doing it for decades with C and C++ and Common Lisp. BUT: in all those cases, there were mechanisms so that the "stuff that can't be compiled" can be quickly ignored (such as the C/C+ preprocessor).

However, I don't think Inform has such a mechanism.

It's likely that in the future there's going to be even bigger changes in how extensions are structured so having extensions be widely compatible with previous versions isn't a realistic goal.

Fair enough. As I noted, it's a pain to update multiple branches (which is what's being proposed), so the result will be that only the most latest version (or maybe 1-2 back) is likely to be maintained. If people are okay with that, then making the "top" level a major release version number does make sense. Once a new major version is released, presumably copies will be made, updated for the new version (if necessary), and provided - correct?

[juhana]

Right, there's no preprocessor for I7 code. There's one on the I6 level but that comes too late for this purpose.

Once a new major version is released, presumably copies will be made, updated for the new version (if necessary), and provided - correct?

That was my original idea. If we go with the branches option then it's as simple as "tagging" the current main branch with a new branch that automatically becomes the working branch for the previous version.

...and typing that just now makes the branches option look a lot more appealing to me...

[juhana]

Ok, so far it looks to me that the branches option is slightly more popular. The revised plan looks like this:

• Create a branch called 6M62 from the head of current master branch. It becomes the main branch for extensions that are compatible with 6M62.
• Remove all extensions that don't compile on 10.1.0 or are otherwise obsolete (e.g. "6M62 Patches") from the master branch. (This won't affect the new 6M62 branch.) It becomes the main branch for extensions that are compatible with the latest version of Inform.
• Add a link to the 6M62 branch to README with instructions on how to upload extensions in the future (i.e. only 10.1.0 compatible extensions go to the master branch, 6M62 compatible go to the 6M62 branch)
• Change the i7.github.io/extensions page to list master and 6M62 branches separately
• (Optionally) do the same with extensions that work only on even older versions (create a branch, delete from 6M62 branch)

Additionally I suggest that another branch called WIP (or something similar) is created to store extensions that are known to not work in any Inform version.

[erkyrath]

I tentatively suggest it would be better to use the 9.1, 9.2, 9.3 terminology rather than continuing to say "6M62".

[curiousdannii]

Remove all extensions that don't compile on 10.1.0 or are otherwise obsolete (e.g. "6M62 Patches") from the master branch. (This won't affect the new 6M62 branch.) It becomes the main branch for extensions that are compatible with the latest version of Inform.

The big disadvantage of this is that most links will break. That's why my current idea is to have the master branch be the development branch, and make other branches be for confirmed-to-work extensions, even for the latest release of I7.

If anyone hasn't checked the forum, my idea is to insert comments into extensions to say which I7 releases they are compatible with:

[Supported releases: 6L38, 6M62]

I've also made a script which then takes those comments are copies the files into a releases folder. It wouldn't be hard to support both 6M62 and 9.3 in the comments. It will then be easy to make a Github Actions script to update other branches after a commit to master.

And if 10.1 or later ever adds some metadata to the extensions then the script can look at that instead of searching for the comment.

I tentatively suggest it would be better to use the 9.1, 9.2, 9.3 terminology rather than continuing to say "6M62".

Downside with that is that almost no one is familiar with those release numbers.

We could make branches for both 6M62 and 9.3. However a downside to that is that currently the only way to issue an update for an old release is to commit directly to a release branch, and people would probably forget to update both branches. So I'd say we should probably just stick with the release numbers everyone's familiar with, at least for the branches. We could easily support [Support releases: 9.3] comments, but then put the extension in the 6M62 branch.

[juhana]

Frankly I don't like the idea of organizing extensions by tags in comments. It's a non-standard, semi-complicated technical solution to something that's essentially a human problem. And since it requires extra effort from either the extension author or repository maintainers it's bound to eventually fall into disrepair.

I'm also concerned that if the main branch is also the WIP branch, there will be extension authors who are too timid to ever consider their extension good enough for release in the confirmed-to-work branches. Then the WIP branch will slowly turn into a de facto production branch because it has more useful content than the production branches.

I think that from the point of view of people who use this repository, which is the extension authors and Inform users who come to download the extensions and who have never visited Github before and have no idea how it works, the main page (which is to say, the main branch) should be what's "guaranteed" to work with the version of Inform they have (which is the latest version.) People don't read instructions so links in readme won't help there. Anyone working with an older version is expected to understand that they'll have to dig deeper to find the "historical" versions of the extensions.

In other words I think that the process of downloading, uploading and maintaining the extensions should be as simple as possible, as a priority over organizational accuracy. Otherwise the repository heads towards the fate of the Public Library and eventually someone creates a "Pals of Inform" service that makes the whole thing easier, and the circle begins anew.

That said, it's not a hill I'm particularly passionate about defending so as long as there's any kind of division of extensions by compatibility I'm fine with it.

[curiousdannii]

Then the WIP branch will slowly turn into a de facto production branch because it has more useful content than the production branches.

That's exactly the situation now. Explicitly marking extensions with their supported versions is a definite improvement, even if there are aspects that are non-ideal.

The main advantage I see to the comments idea is that it keeps things simple for people adding extensions - they only need to add them to one branch. The site can then take care of duplicating them into other branches.

I think that from the point of view of people who use this repository, which is the extension authors and Inform users who come to download the extensions and who have never visited Github before and have no idea how it works, the main page (which is to say, the main branch) should be what's "guaranteed" to work with the version of Inform they have (which is the latest version.) People don't read instructions so links in readme won't help there. Anyone working with an older version is expected to understand that they'll have to dig deeper to find the "historical" versions of the extensions.

We could keep the "master" branch be the development branch, but make the "10.1" branch be the Github default branch? (Ie, the branch you see if you haven't selected any others.) That should keep links intact, but also make the extensions they see be ones that are confirmed for the latest release.

We should probably also restrict commits to the non-master branch, so that only PRs can edit them? Then we won't have people accidentally adding WIPs to the production-ready 10.1 branch.

[curiousdannii]

@zedlopez Has done a lot of testing for extension compatibility, so whichever option we choose, we'll be able to start it with a lot of extensions in the correct release branches, even if we have to manually adjust it afterwards.
https://intfiction.org/t/inform-7-v10-1-0-is-now-open-source/55674/149

Eventually perhaps we could even run a similar sort of thing as a Github Action to check that extension updates to actually support the releases they claim to.

[zedlopez]

getting the version within running code:

Include (-
[ i7Version;
#Ifdef I7_VERSION_NUMBER;
print (PrintI6Text) I7_VERSION_NUMBER;
#ifnot;
print (PrintI6Text) NI_BUILD_COUNT;
#Endif;
];
-)

to say i7 version: (- i7Version(); -).

This will get you "6M62" or "10.1.0". (I7_FULL_VERSION_NUMBER would yield, for my current build, "10.1.0-beta+6V07".) There isn't anything within a 6M62 program that would let you get "9.3" (not that there's any reason that should influence the choice of which we should use for tagging).

Telling the versions apart for purposes of selectively including code in extensions:

Section Old School (for use without Basic Inform by Graham Nelson)

[...]

Section Shiny New (for use with Basic Inform by Graham Nelson)

Yeah, I'd rather have something explicit, but that'll work pretty well in practice.

We could keep the "master" branch be the development branch, but make the "10.1" branch be the Github default branch? (Ie, the branch you see if you haven't selected any others.) That should keep links intact, but also make the extensions they see be ones that are confirmed for the latest release.

We should probably also restrict commits to the non-master branch, so that only PRs can edit them? Then we won't have people accidentally adding WIPs to the production-ready 10.1 branch.

these sound really good.

[zedlopez]

Telling the versions apart for purposes of selectively including code in extensions:

I just checked -- v10 will let you have pre-v10 style Include (- -) instead of "Label" in "Template.i6t" inclusions within a "for use without basic inform" section. After suggesting it, I worried that the parser might not cooperate.

[curiousdannii]

I started an informal poll to see which of the two main ideas people prefer: https://intfiction.org/t/updating-extensions-in-the-github-repo-for-inform7-v10/55849/29

[curiousdannii]

The 9.3 and 10.1 branches have been populated, thank you @zedlopez for all your work on this!

Those branches have also had commit restrictions applied to them: you can now no longer directly commit to them, instead you must create a pull request against them.

The poll is still open - currently 50-50 on whether or not to set up an automated replication system.