Create separate dev branch for Wagtail development purposes

[laymonage]

This is a follow-up to @thibaudcolas' #175. That issue is more concerned about missing features to aid Wagtail development, whereas this one is concerned about the development process.

The bakerydemo is used to demonstrate the latest stable version of Wagtail. As a result, any new changes in Wagtail are only incorporated into bakerydemo after the stable version is released.

On the other hand, bakerydemo is often used as the primary testbed for developing Wagtail. As the main branch of bakerydemo needs to be compatible with the latest stable Wagtail release, it means each of us needs to make changes to our own fork (or a separate branch in this repository). This can sometimes be a pain point in developing Wagtail, especially when we need to test/review new features that's built on top of the latest changes.

To top it off, bakerydemo sometimes lags behind the latest Wagtail release for a while until someone makes a PR to update it and bring new features. One of the reasons for this is that our local changes are usually a quick and dirty solution to do some tests, and may not fit within the overarching theme of bakerydemo, so we tend to not make a PR for it.

We need to figure out a way to make testing against Wagtail's main or a PR branch easier, so we can speed up Wagtail's development process.

An idea that I had in mind is to have a separate bakerydemo branch, e.g. dev, that acts as the primary branch for core contributors to work on. How it would work:

• We create a dev branch from current bakerydemo main.
• For Wagtail PRs that need changes to the bakerydemo, a contributor should create a separate branch based on latest dev on their own fork of bakerydemo, and make a companion PR to bakerydemo's dev using that branch.
• A core team member reviews the Wagtail PR using the bakerydemo companion PR.
• When the Wagtail PR is ready to merge, a core team member merges the Wagtail PR into Wagtail's main and the bakerydemo companion PR to bakerydemo's dev.
• Repeat for new Wagtail PRs.
• When a new Wagtail release is out, we make a PR from dev to main. Any cleanups needed in dev would be ironed out before merging it to main.
• Repeat for the next release.

There are a few challenges here, though:

• Ideally, the companion PR should be something that would be "good enough" to eventually be merged into bakerydemo's main. However, this isn't always possible as it takes a good amount of time to figure out a use case for testing the PR that also fits well into bakerydemo. Thus, we may need to compromise when it makes sense. It's not a very big deal though, considering that dev is still separate at this point.
• If we do make compromises as above, merging bakerydemo's dev into main won't always be straightforward, as we'll need to make sure everything is good for the demo.
• An alternative would be to make dev and main (almost) completely separate branches. When a new Wagtail release is out, changes in dev that make sense would be cherry-picked into main, and the rest are left-out (or redone in a different way).
  • Afterwards, we could either keep dev and main diverge, or hard reset dev to be based on main. Contributors would need to update their forks accordingly.

What do you think? Please discuss! 😄

[thibaudcolas]

I like your proposal except for:

If we do make compromises as above, merging bakerydemo's dev into main won't always be straightforward, as we'll need to make sure everything is good for the demo.
An alternative would be to make dev and main (almost) completely separate branches.

It can get really tiresome to maintain long-lived branches that have to stay "somewhat in sync but not quite", and I don’t see ourselves wanting to dedicate that kid of time for bakerydemo. I think a dev branch would only work well if we plan for it to be regularly merged into main

So back to the use cases for this, I see three, and I think my preference would be for separate solutions:

• Making improvements to the demo that are clearly good enough to be included in main but are dependent on the dev version of Wagtail. For that scenario, a dev or release branch merged to main when Wagtail’s latest release becomes available sounds ideal.
• Making improvements for development / testing purposes. For that scenario, I’d expect a separate Django app (or apps) that otherwise lives in the same main branch to be ideal. There are other options (yet another branch?), but I think using the same branch as the above scenario conflates different needs.
• Making bakerydemo changes solely for the purpose of facilitating testing a specific Wagtail PR. Similar to the above but I guess in that case those changes really shouldn’t be merged anywhere if they’re not good enough for further reuse.

The alternative is to do some or all of the above in an ad-hoc capacity as it makes sense. So for example for GSoC, we could create our own gsoc branch which we’ll merge after Wagtail’s next release. I guess having a shared dev branch just formalises that process a bit more.

[laymonage]

@thibaudcolas ooh, using a separate app for testing sounds like a great idea, I hadn't thought of that! That would definitely be useful, even on the main branch. And yeah, for Wagtail PR testing purposes, a companion PR to bakerydemo's dev with changes in that test app would work.

I think we can also add an environment variable to enable/disable the app based on the environment, e.g. to disable it on https://demo.wagtail.org but leaving it enabled on our local copies.

The alternative is to do some or all of the above in an ad-hoc capacity as it makes sense. So for example for GSoC, we could create our own gsoc branch which we’ll merge after Wagtail’s next release. I guess having a shared dev branch just formalises that process a bit more.

Yeah that would work for the short term, but I'd like to see a formalised process so we don't have to think a lot about this in the future.