Create a process for ad hoc updates in between schema versions

[siwhitehouse]

We do not currently have an agreed mechanism for making ad hoc updates to the website in between schema versions. We need this so that we maintain consistency between the English and translated versions of the website.

[stevenday]

@siwhitehouse - my assumption is that as soon as we change any text, this will have to follow the same process as schema releases (in terms of freezing, translating, releasing). Obviously with less need for review, example testing, etc since we're not changing the schema/standard itself.

Elaborating on the options gives four kinds of release:

• No text changes: no translation needed, no new version, release immediately
• Text changes but no schema changes: translation needed, freeze and translate release process
• Minor schema changes (bug fixes, typos, etc): translation, new patch version, freeze and translate release process
• Schema changes: translation, new major or minor version depending on changes, full review and release process

Does that sound accurate?

Aside: I think some better tagging of released versions, and perhaps using a release branch system would help if we need to separate these kinds of changes in the future, and avoid the need to 'freeze' development on master.

[stevenday]

Talking in our sprint planning it seems like there's some appetite for a better git branching system to help manage translations and release fixes etc. @Bjwebb - do you have thoughts on this and whether it's something we could prioritise in our sprint call tomorroqw?

I also note there's a smaller related ticket in openownership/bods-dev-handbook#13 about branch names, which ties into another question I was going to ask about tags.

I note that we're very explicit about using branches, not tags: https://github.com/openownership/bods-dev-handbook/blob/master/standard_releases.md#versioned-releases but I'm not sure why that is? I was surprised when I tried to find the canonical v0.2 in github that it was a branch, not a tag.

[odscjames]

I note that we're very explicit about using branches, not tags: https://github.com/openownership/bods-dev-handbook/blob/master/standard_releases.md#versioned-releases but I'm not sure why that is? I was surprised when I tried to find the canonical v0.2 in github that it was a branch, not a tag.

This was to allow the possibilities of technical updates that did NOT change content being made to each version.

For instance, you can see we have used this option to add a banner redirecting people to the latest version from older versions (I looked into using the RTD features for this, and it turns out we couldn't because we had named our tags "wrong", and in the end I determined it was less work and less disruptive to just add a banner by hand to each branch.)

[siwhitehouse]

@stevenday Your assumption is correct, in that once we have an update to the docs or the schema we will need to go through the translation process.

It is better to have certainty that what is uploaded to Transifex is the finalised version for any of the scenarios. I have seen Transifex remove the translation for a whole paragraph based on a single character changing. To my mind the freeze is important as it removes the possibilities of confusion in Transifex as much as anything else.

As for the scenarios you've outlined, these look good to me. I've commented below

• No text changes: no translation needed, no new version, release immediately

Apologies for my ignorance here. What might have changed in this scenario?

• Text changes but no schema changes: translation needed, freeze and translate release process

Yes.

• Minor schema changes (bug fixes, typos, etc): translation, new patch version, freeze and translate release process

Yes. @kd-ods has flagged the difference between an update and a correction and that we might want to retrospectively change previous versions of the website documentation in the case of corrections.

• Schema changes: translation, new major or minor version depending on changes, full review and release process](url)

Yes. With the acknowledgement that we might decide a different git branching system that supports this.

[stevenday]

Great, thanks @siwhitehouse. With that agreement then, I think we could close this ticket? We've basically accepted that the 'ad-hoc' release process is much like any other (which I presume is already documented), unless there are no text changes at all (the example I was thinking of was purely styling/theming changes).

[siwhitehouse]

Thanks @stevenday. At the moment, the documentation describes undertaking a first translation. but not subsequent updates. My apologies, as I should probably have made that clear earlier.

Also, myself and @kd-ods have been discussing this and we think there are a few other factors to consider.

1. Decide the ad hoc update process in the context of our Github branching system

At the moment we use v.0.2.0 as the latest branch on Read The Docs. So, an ad hoc update should update this branch in order for it to appear on the website. We might decide a different branching system and so it would be good to have that discussion before (or at the same time) as we decide the way that this will be done.

The recent development work has all been merged into master. We think this means that the commit: Update copyright notice on the about page won't be updated immediately, which is probably not what you want.

The above is not documented on the translation page of the handbook at present.

2. Decide whether an update is for the current version of the standard or whether it applies retrospectively.

For instance, the privacy notice. If someone is browsing an older version of the standard, but we are now tracking them using Google Analytics, then the older version of the schema needs to have been updated retrospectively. There are other instances where this will be the case.

3. Update the documentation to explain the different processes behind an ad hoc update and a schema version update.

We left the documentation at the point where it describes undertaking a first translation.. We think that we need to come to decisions about 1 and 2 above so that we can properly document both ad hoc updates and version updates.

@kd-ods could you check and fill in anything that I have missed here, please?

[stevenday]

Aha, thanks @siwhitehouse - I realise now I've not really seen the full picture with the branches and what we'll need to do in order to 'release' our updates.

I think 'in the context of our branching system' we will just need to 'backport' the right changes (i.e. git commits) to all the live branches that need them. As you say, for the recent theming and GA work, that's all of them - we want all the versions to have the same styling and tracking.

Technically, I think the best way to do that is via pull requests to those specific branches, where I cherry-pick across the right commits onto v.0.1.0, v.0.2.0, etc and then we review them individually in those contexts. On reflection, we're effectively using something like gitflow already, but our release branches are long lived, rather than being created to get to a tagged version and then deleted. I don't think we need to change anything right now.

We can perhaps consider adding the explicit step of creating a branch when we're beginning to prepare a new schema release, rather than as the final act. However it'd only be necessary if the 'freeze' usually associated with that process would unduly block other work happening in parallel, which doesn't seem hugely likely at the moment.

I've started openownership/bods-dev-handbook#17 to put these thoughts to 'paper'. I'll also work through creating those pull requests at the same time so that I can test my ideas.

[siwhitehouse]

Cheers @stevenday (and the thanks are really due to @kd-ods who did most of the thinking about this).

I follow most of what you've written above. If I use the current PR for the spanish-translation branch as an example. I think that I should change that so that it merges into master and that you will then cherrypick the commits that need to go on to v0.2.0 from there. Is that correct?

[stevenday]

@siwhitehouse - yes, I think that's right (and I now realise it's what we say in the current docs about branching too).

[kd-ods]

I think that I should change that so that it merges into master and that you will then cherrypick the commits that need to go on to v0.2.0 from there. Is that correct?

@siwhitehouse - yes, I think that's right (and I now realise it's what we say in the current docs about branching too).

OK - I think that will work this time around for adding a new translation to an existing version release branch, but that's contingent on a number of factors (e.g. Simon's not having pulled in strings anew from master to the v0.2 project in Transifex). Part of the thinking and documenting we need to do is around robust process for this type of thing in the future. And we need to be very clear about the use cases we need to cater for. I've started thinking about this and shared the thinking with Simon.. Good to see WIP in the dev handbook. Will comment there.

[stevenday]

I think this can now be closed, as we've merged some changes to the dev handbook which describe the various ways to make changes to the schema (from releases to tiny ad-hoc changes) and we've followed them at least once.

[jpmckinney]

@stevenday FYI, for OCDS's non-normative content, we allow publication of updated English content before translation. We just put a box saying the page has changed in English (and try to translate asap). You can see an example on this staging page: https://standard.open-contracting.org/1.1-dev/es/guidance/

Assuming your translation policy is similar, you can do the same by modifying and running this script during deploy: https://github.com/open-contracting/standard/blob/1.1-dev/util/add_translation_notes.py

We run it after running make (which builds the docs with Sphinx and does some other work): https://github.com/open-contracting/standard/blob/1.1-dev/.github/workflows/ci.yml#L37