Pull in the various SSOT versions as submodules

[andylolz]

Currently, the various SSOT versions are copy/pasted in the resources folder. I’m not sure this is really consistent with the idea of a single source of truth. There’s potential for discrepancies, and furthermore it’s not immediately obvious that what’s in those folders is the same as the SSOT.

If you instead use submodules, git guarantees consistency, and it’s immediately clear that the versions match up. In fact, the original SSOT proposal suggests this as the preferred method for referencing the SSOT.

You might consider doing it this way, which would hopefully decrease the overhead involved in staying up-to-date.

[hayfield]

As has been noted previously, the current SSOT does not act as a SSOT.

With the folder structure in pyIATI, part of the design decision was to look to design a structure that can actually be a useful and maintainable reference structure for what the IATI Standard is (and test it in earnest before any new design is adopted in a wider setting). As such, you'll note that things like Non-Embedded Codelists are handled differently to the SSOT (through the use of symlinks).

The current method of requiring manual updates is clearly not ideal, and over time will be replaced by an automated system (part of a module to fetch network resources from eg. the SSOT, the Registry). This has not yet reached the top of the list of priorities.

[andylolz]

the current SSOT does not act as a SSOT.

I’m just not sure what you mean by this. You didn’t provide a citation when you stated it previously. Could you explain what you mean? I ask because it sounds like quite a serious thing!

The current method of requiring manual updates is clearly not ideal, and over time will be replaced by an automated system (part of a module to fetch network resources from eg. the SSOT, the Registry). This has not yet reached the top of the list of priorities.

So it sounds like this is a willfix but not here and not yet (rather than a wontfix). That’s fine – I just wanted to raise it as a thing that needs fixing at some point.

[hayfield]

I’m just not sure what you mean by this.

A useful SSOT contains just the Standard.

The current SSOT, contains the Standard and a lot of other things, while also missing out some parts of the Standard such as Non-Embedded Codelists. It is the one true source of things like the Schemas (so parts of it are a true SSOT for some of the Standard), though there are many factors that make it challenging to work with in a practical manner and unsustainable long-term without significant changes.

So it sounds like this is a willfix but not here and not yet (rather than a wontfix).

Kinda - "create a method to auto-update the resources folder from the SSOT" is a willfix (I'll note to create a Github Issue that details this and notes down things from previous discussions in this area), while "use git submodules to keep pyIATI in sync with the SSOT" is a wontfix.

[andylolz]

Kinda - "create a method to auto-update the resources folder from the SSOT" is a willfix (I'll note to create a Github Issue that details this and notes down things from previous discussions in this area), while "use git submodules to keep pyIATI in sync with the SSOT" is a wontfix.

Ohhh, gotcha – Good point.

[andylolz]

The current SSOT, contains the Standard and a lot of other things

Again, it depends what you mean by this. I’m in favour of moving more guidelines out of IATI-Extra-Documentation (versioned docs) and into IATI-Guidance (unversioned docs) †. So if that’s what you mean, then I agree.

But if you mean the rulesets and versioned docs shouldn’t be part of the SSOT, then I don’t agree. They’re complementary to the schemas – the schemas can’t be fully understood (for publication or use) in isolation.

while also missing out some parts of the Standard such as Non-Embedded Codelists.

Based on the explanation provided here, I think it makes sense to exclude non-embedded codelists from the SSOT repository. However, the meaning of non-embedded has gradually shifted (and shifted a long way in v2.03) to be more about process than content (since changes to non-embedded involves less process).

many factors that make it challenging to work with in a practical manner and unsustainable long-term without significant changes.

It would be really good if you were able to write up your thoughts on this in more detail, and raise it with the community. I’d certainly be interested in reading.

---

†: If you find yourself frequently changing the exact same thing on five different version branches, then it’s worth considering whether it definitely needs to be versioned.