Fix branches problem, review branching strategy

[lornajane]

We changed our branching strategy after doing the 3.0.4 and 3.1.1 releases, it's documented in the CONTRIBUTING file: https://github.com/OAI/OpenAPI-Specification/blob/main/CONTRIBUTING.md#appendix-branch-strategy

Development has been going well but following the documented steps for releases had some side effects. We delete the tests and schemas before merging the renamed specification file to the main branch. This makes sense since the tests and schemas are not present on the main branch.

However that means that since the 3.1.2 and 3.2.0 releases (the first ones with this branching strategy) we cannot now merge main to dev without deleting those files on dev (and on the vX.Y-dev branches when the sync runs)

The sync jobs are currently turned off, we need to figure out what to do next to move forward.

Related information but only slightly: When merging multiple release branches in succession, we saw this error. The workaround is to close and re-open the pull request. I believe it happens because of the shared intermediate dev branch (but I'm speculating about that).

[lornajane]

One option is to make the merge from main to dev and then from dev to vX.Y-dev and then do a revert on the delete parts of that commit, does anyone have an opinion on that approach? Or an alternative suggestion?

[ralfhandl]

make the merge from main to dev and then from dev to vX.Y-dev and then do a revert on the delete parts of that commit

I found a (tedious) way of doing that:

• create branch dev-temp from dev
• merge main into dev-temp - this deletes files from src and tests that we'd rather want to keep
• git checkout dev src/* - restore "lost" files in src
• git checkout dev tests/* - restore "lost" files in tests
• commit & push to dev-temp
• merge dev-temp into dev - this will only merge the "new" files (with their history) from main and leaves the restored files (and their history) unchanged

After these steps dev is back in sync with main.

Then dev has to be merged into each of the vX.Y-dev branches by repeating these steps with a temp branch created from vX.Y-dev.

After that the vX.Y-dev branches are back in sync with dev.

Until the next release, which will bring us back into the same situation we are in now ☹️

PR

• main: sync workflows main -> dev -> vX.Y-dev use sync branches #5010

automates these steps, the updated sync workflows will automatically sync everything outside of the folders src and tests.

[ralfhandl]

an alternative suggestion

I think we need to define the branch roles more precisely, and define how we expect the branches to "look like" after a release. This includes

1. Which (new or changed) files do we expect in each branch?
2. Which content do we expect in each file?
3. Which history do we expect each file to show?

The current branch strategy implicitly answers these questions for main, and some of the questions for other branches.

After a release X.Y.Z we expect:

• in main:
  • new files versions/X.Y.Z.md and versions/X.Y.Z-editors.md
  • for versions/X.Y.Z.md:
    • content and history of src/oas.md from branch vX.Y-dev at time of release
  • for versions/X.Y.Z-editors.md:
    • content of ./EDITORS.md at time of release (we expect this file to be in sync across all branches)
    • no history, just a new copy of ./EDITORS.md
• in vX.Y-dev:
  • new files versions/X.Y.Z.md and versions/X.Y.Z-editors.md with content and history in sync with main
  • file src/oas.md:
    • same content as versions/X.Y.Z.md (next step is to bump the version and adjust the history table)
    • Which history do we expect it to have?
      1. No history because it is to become versions/X.Y.(Z+1).md in the future and starts as a fresh copy of versions/X.Y.Z.md?
      2. Same history as versions/X.Y.Z.md to reflect all changes since versions/X.Y.0.md?

Note: we currently have state (b) in v3.1-dev and v3.2-dev, and in addition a somewhat weird state in v3.3-dev with src/oas.md's history being all changes in 3.2.0 since its start as 3.1.1.

Note: I don't know how to truncate a file's history in git. The obvious way of "remove, commit, create, commit" does not truncate the history, but it does have the desired effect of blaming everything as new, which is sufficient for me.

The history of a file can be truncated by creating an orphan branch, removing everything from it, adding the file back whose history should be truncated, then merging this branch:

git checkout --orphan v3.1-orphan
git rm -rf .
git checkout v3.1-dev src/oas.md
git commit -m "fresh start"
git checkout v3.1-dev
git merge v3.1-orphan --allow-unrelated-histories
git push

The result is a file with a one-commit history.

[karenetheridge]

Why does dev need to exist at all? Can't everything in dev just stay in main?

[ralfhandl]

Why does dev need to exist at all?

Intention was

• dev is the primary branch for working with the src tree. Development infrastructure that is not needed on main is maintained here, and can be merged out to other non-main branches as needed.

In theory tools/scripts for working with stuff in src can be developed and tested in dev and then auto-synced to the vX.Y-dev branches.

In practice I develop and test it in (a temp branch of) v3.1-dev, and once it works there I test it in v3.2-dev before creating a PR to merge it into dev, from where it gets synced into all vX.Y-dev branches.

[lornajane]

I am not sure that this complex setup is working for us. I suggest that we:

• do some surgery so that we can safely merge main to the vX.Y-dev branches again (probably involves messy history since we didn't stop to sort it out when it happened)
• stop using dev
• change the release process to be a single commit to a release branch that creates the new specification and editors files, and then cherry pick it across rather than merging.

I'd like to apologise for not testing the documented workflow in a safe space ahead of the release, I'm not sure why I thought new instructions would work first time!!

[ralfhandl]

I am not sure that this complex setup is working for us.

The goal of this admittedly complex setup is to have spec files in the versions folder of the main branch that sbow the full history of all changes since the preceding spec version they started from.

All currently published spec versions have this feature (I checked all of them):

• https://github.com/OAI/OpenAPI-Specification/blame/main/versions/3.2.0.md
• https://github.com/OAI/OpenAPI-Specification/blame/main/versions/3.1.2.md
• ...
• https://github.com/OAI/OpenAPI-Specification/blame/main/versions/3.0.1.md
• https://github.com/OAI/OpenAPI-Specification/blame/main/versions/3.0.0.md

I think this is a very useful feature, and I would like to keep it for future spec versions.

do some surgery so that we can safely merge main to the vX.Y-dev branches again

This is exactly what the sync workflows in

• main: sync workflows main -> dev -> vX.Y-dev use sync branches #5010

automatically do, and will automatically do for future releases (until someone decides to change the process).

I'd like to apologise for not testing the documented workflow in a safe space ahead of the release

Me too. Beginning of this year I only tested the release process up to preparing the release branches and creating the PRs for merging them into main.

Now I've tested the whole cycle: merging the release PRs into main, syncing them into vX.Y-dev (via dev), starting new spec files without history, changing them, preparing release branches, merging the release branches into main, and syncing them to dev. The whole test cycle is documented in PR #5010 with links to PRs in the fork where I tested the stuff.

change the release process to be a single commit to a release branch that creates the new specification file

Then the specification file will have no history, and all changes since the preceding spec version will be one huge change in that single commit. No way to find out who changed what when in which PR to solve which issue. I think this would be an unacceptable loss.

stop using dev

That is definitely something we could do, syncing main directly into the vX.Y-dev bramches.

I would prefer to first fix the current two-step sync process via #5010 and try it out for a few syncs (we have a nice backlog of changes to main that will need to be synced).