IE-0001: Directory format for extensions with resources #1
Conversation
curiousdannii
changed the title
curiousdannii
added
the
formal-proposal
|
Multi-file extensions are a good idea and will surely help with quite a few things. Here are some of my thoughts about them:
I'm not sure what is included in the metadata currently, but useful things to include could be:
Metadata could also communicate things like whether an extension needs to be output to Glulx or whether it can be converted to C code etc. The user will discover problems when they try to compile it, but if it was in the metadata, then maybe the incompatibility could be raised earlier, and maybe the error message shown could be less obscure? |
|
Ideally, even "directories" of multiple files would still be a single file to install from the user's perspective -- e.g. to install an extension they just download a single .zip file and put it in their extensions folder. This also helps to keep the component files locked together and reduce version incompatibility weirdness. Within an Extension Project the files should be loose for easy editing and the install/publish step could zip them up. Having said that, within git repositories (such as the current I7 Extensions repo) it's nicer to have loose files as well, so that you can more readily see text diffs of the changes -- I haven't tested it, but I doubt GitHub would look into the contents of the zip to diff that nicely, and it would certainly inflate repository sizes faster than it should. So there's still a bit of logistic challenge there, unless the process of downloading a single extension from GH (and storing it as a zip) is automated in the IDEs, or both formats are supported so that the current practice of using the whole GH repo as your Extensions folder still works. |
|
i'm looping back here to better understand IE-0011 and IE-0003. since i don't fully understand the .inform/.materials distinction, i have a few clarifying questions:
|
|
A somewhat obvious addition here would be the option to have a Tests directory where one would expect a "Red Fire Hydrants.intest" file (I suppose I'm assuming here a mod to intest to be willing and able to ignore the -v.i7xd part of the directory name). Though I would prefer to not have the version number be part of the directory name. There's an explicit version number in the metadata and I don't see an advantage to redundantly putting it in the directory name. To encourage compliance with semantic version-ing, I think it's significant to ensure that incrementing a version number is painless. Accumulating a collection of old directories provides enough nuisance value that I fear it would discourage version number changes and it fits badly with how a developer would want to use source control. (Perhaps, though, a part of the IDEs' handling of these could include displaying the current version number prominently in a way that would provide the equivalent benefit to the author that the version number in the directory name would have had.) |
|
There are a couple of existing extension examples that don't compile because they need files for a Figure or a Sound. I briefly considered suggesting that any files needed by the examples would be under the extension's own Materials directory, but I don't like the idea of cluttering that with things needed only by examples. So maybe it would be appropriate for the Documentation directory to also (potentially) have Figures and Sounds subdirectories. (I figure it's acceptable that all of these would be provided to all the examples -- it should be easy to avoid creating conflicts.) |
|
Yes, I think I agree with that. We were going to need Documentation/Figures anyway so that documentation can potentially include diagrams, etc. |
The obvious problem that we need different names to support having multiple versions in a given project finally occurred to me. So, a mini-proposal: Under the project.materials directory, there may be a Development directory, under which there may be an Extensions directory, under which there may be, say, "Zed Lopez/Info Utility", which is an extension-in-directory-form. Building the project starts with looking at what's in Development and writing the corresponding thing to project.materials/Extensions with the appropriate version number. So, best of all worlds: a developer can keep something under source control nice and neatly and we can have as many distinct versions in a project as we want. (This scheme could eventually be extended to Templates, if desired.) |
|
So, I'm afraid I do very much want it to be the case that the canonical filename for a single-file extension should include the version number, and similarly for the canonical directory name. It's not just for the materials folder, where I agree, it may be a slight nuisance for version-control. It's also for all of the other potential places where inbuild may be handling things, such as archives of extensions. The only reliable way to store extensions is for the namespace of potential destinations to be able to represent faithfully the full range of potential extension versions. But with that said, Inform does read extensions which don't have the version number in the filename. If you drag a directory extension into your materials folder by hand, it can have the wrong version number or none, and Inform will detect and correct this. You can put more or less anything into the installer, and it will sort out what the naming should be and will apply it. So the zip files being served by the Public Library, for example, can be named however the PL likes, and that won't affect the ability of the installer to work it all out. And besides that: (i) It's not like people change version numbers for extensions every day, and (ii) Version control deals with file renames all the time. If you do change a version number, or change the version of an extension you're using, that's absolutely a change which version control should be registering. I may be over-making my case here, but it's just such a potential nest of vipers to start making exceptions. For example, the obvious exception would be to not bother with this version-numbers-in-the-title business for extensions in the materials folder (or, as Zed suggests, some special enclave of it). That implicitly imposes that a project can't have two versions installed - well, we could live with that. Except of course it doesn't, unless we actually forbid people putting number-in-name extensions there. But then people can't drag otherwise valid extensions into their own projects any more. So... we don't want to forbid that. But then, we end up with a situation in which extensions are being developed under filenames which are vague as to their content. When those extensions are distributed, maybe those filenames are still vague, because users, having no special incentive, haven't amended those names. So when the extensions are used in some other person's materials folder - supposing they are moved directly, not via the installer - now names can collide, and unpredictable things will happen. Inform's current practice of auto-maintaining the namespace of the extensions in the materials folder so that it is always correct seems to me the only safe way through what really is something of a swamp, |
|
I may not have communicated clearly: my proposal has none of the drawbacks you describe. Development/Extensions would not be an auxiliary extensions directory and its contents would never directly be built into a project. By my proposal, a new first step in compilation would be to check the Development/Extensions directory and write the corresponding extension into the real Extensions directory with the appropriate versioned directory name. This could be implemented by plainly copying Development/Extensions' contents to Extensions. The existing implementation's correction of any version-less directory names would then rewrite them to include the version. I'm not suggesting a change to the auto-maintenance of the extension namespace in the Extensions dir proper. My proposal creates no risk beyond whatever slight risk is inherent in the existing proposal. Even if someone drags and drops a misnamed extension into the Development/Extensions directory, the end-result is the same as if they'd dropped it in their Extensions directory: they end up with something whose name has been corrected. All of the tools can continue to consider misnamed extensions as directories-non-grata without change, with the sole exception being Development/Extensions. The (small) population of extension authors can be told that what they share should have the versioned name. It's straightforward to enforce by policy that things without correctly versioned names won't be accepted into the Friends repo or the Public Library. |
|
Will inbuild be able to build .i7x files from .i7xd directories (necessarily omitting Figures, Images, Sounds), or to deconstruct .i7x files to .i7xd ? (or to turn either into Public Library-friendly zip files?) |
|
Hmm. Okay, I suppose git doesn't care about the name of a repo's directory and it wouldn't interfere with source control or matter one way or another for that name to keep changing due to corrections by inbuild. I still like my suggestion for a Development dir, but I withdraw my making this out to be a big deal. I'll use any spare big-deal-making vis-a-vis source control-friendliness this leaves me to agitate for allowing the .materials directory to be the only thing in need of tracking. 😀 |
Technically, any edit that you make should include a bump to the version number (though in practice it probably doesn't matter unless you release the change to others -- but that line gets blurry if your release is just a git push). Also, git in particular only has fuzzy tracking of file renames -- if you simultaneously rename the file and make significant enough changes to its content (and this could even just be changing the indentation or rearranging some of the section ordering) then it will treat it as a unique file and you'll lose the history tracking without some heroics.
Isn't the solution to that to define a predictable rule for that case? Hopefully, people would use Extension Projects, and publishing from the EP to the main Extensions folder could automatically append/update the version -- which is good, as it allows multiple versions to exist side-by-side in the Extensions folder and in repositories like the PL or Friends github -- but less good in that adding new files isn't even a rename so they'll have disconnected history. (You can still diff them explicitly but you won't be able to do a blame. But if it's a binary format like a zip then that may not matter anyway.) So, what other cases do we have? If there's a private version in the Materials folder then that should pretty much always win over any version in the main Extensions folder (or produce a Problem if it doesn't match the explicitly included version in the story). There can't be two different versions in the Materials folder unless they do have the version included explicitly in the filename; then you just need to have a rule that if the story specifies a version explicitly then that's what it gets, otherwise it gets the highest one, or something like that. (Which is the same rule you'd use for two versions in the main Extensions with nothing private.) No unpredictability. I'm assuming that when you build your internal list of extensions, you use the versioned name as your lookup key regardless of whether the file was named that way or not, allowing you to keep side-by-side refs. This might require opening the file and parsing the first few lines, but that should still be reasonably fast, and can be made faster by caching it along with metadata (e.g. modification date) so that you know you don't have to re-parse it. |
|
Graham wrote, incorrectly:
This is not correct. I just want to confirm what uecasm said. First, you should be changing the version number EVERY TIME. Extension version numbers should, in fact, be changing constantly. While major version numbers might not change so often, since they reflect an interface, patchlevel should be changing on every upload. And, Git only does fuzzy pattern matching and will not always ID the file as the same file if you're changing its name simultaneously with changing its contents substantially. |
|
I am writing a Java-based IDE for Inform7, and I thought to come to the Inform-Evolution repository to make a comment here. It would be nice for the command-line Inform tooling to provide an API or tools for creating and validating Inform7 project directory layouts, such as "asdf.inform" and "asdf.materials". It is not necessary, but it would be a convenience that may ease development of other tooling Inform users have created, such as other CLI. I cannot say I have thoroughly prepared myself for making an IDE for Inform, but I have taken a glancing look at the GNOME IDE and with dissatisfaction from its installation process, especially on older machines, I thought a Java IDE would be a better solution (given that Java "should be" more portable). |
|
I7 can validly run with no materials directory at all; if it's missing when the compiler begins, then it is created. In any case, an empty directory called Whatever.materials is valid, because all contents and subdirectories are optional. |
I would be happy to try to fix the installation process (depending on how old of machines we are talking about...) Please let me know what the problem was at https://inform7.atlassian.net/jira/software/c/projects/I7/issues. |
Summary
Substantially expanding the range of resources an extension can provide.