Merge postgrest-docs with main repo

[steve-chavez]

Problem

Keeping postgrest-docs in sync with the main repo is quite painful. Duplicate work is done in tagging a release on docs, duplicating the CHANGELOG on docs, etc. This makes us slower overall.

Solution

Merge postgrest-docs with main repo while preserving history. It's possible as previously discussed on: #1897 (comment)

We still need to figure out the steps for CI and readthedocs.

[steve-chavez]

If we only make releases through non-main branches, like we have been doing for some patch/minor versions:

• https://github.com/PostgREST/postgrest/tree/rel-10.1.2
• https://github.com/PostgREST/postgrest/tree/rel-10.2.0
• https://github.com/PostgREST/postgrest/tree/rel-11.2.1

Then we would be compatible with the way we release docs too, which is done exclusively on branches: https://github.com/PostgREST/postgrest-docs/branches

Then our main here would point to the latest docs, which is also done for postgrest-docs.

[steve-chavez]

Since CI for docs is fast, we could run it on every PR.

We would have to drop some old version of the docs though, which don't have equivalent branch releases here.

(Actually no, we can just create the branches)

I think this would be the smoother transition.

[wolfgangwalther]

Is there any idea about how to proceed with all the issues in the postgrest-docs repo, already? It would make the most sense to move them here, but we can't do that 1-by1...

[steve-chavez]

On #1897 (comment)

I think we could still maintain the separate repo for tracking issues, even if we move the docs into the code repo
Benefits
Docs-related issues/PRs separated from other ones

We agreed to keep the docs issues on its own repo. I think we should keep that for now.

[wolfgangwalther]

We agreed to keep the docs issues on its own repo. I think we should keep that for now.

I'm not sure this will work with an archived repo as mentioned in #3113 (comment). I think we should definitely archive the docs repo.

I also don't see the any good reason to keep it that way - except to avoid effort in moving things over. But I don't find that a good reason. I'd rather invest some time to get this right now than to keep having two repos.

It will be really inconvenient to deal with PRs in the main repo solving issues in the docs repo etc...

[laurenceisla]

Yes, I think moving them to core is the best if archiving the docs repo gives problems.

It would make the most sense to move them here, but we can't do that 1-by1...

Someone mentioned that bulk issue migration can be done using GitHub CLI (it doesn't keep the labels though):

https://github.com/orgs/community/discussions/9695#discussioncomment-5423988

[wolfgangwalther]

As mentioned in #3113 (comment) I archived the docs repo for now.

As expected, this also means that it's currently not possible to open new issues or comment on there - the full repo is read-only.

We'll keep it archived for now, to be able to work on the migration of branches without running risk of needing a rebase or something. We can then still discuss how to proceed regarding the issues. We can either temporarily unarchive the repo again to move the issues over - or permanently unarchive it, if we decide to use the issue tracker over there. Technically, all options are still possible.

[wolfgangwalther]

I merged the active docs branches (main, v12.0, v11.2 and v10.2) into the respective release branches in the core repo. I used the same approach outlined in #1897 (comment) and used in #3120 with the following changes:

• I did not move the whole docs repo into the docs subfolder, especially not the docs folder itself. So we don't have a nested docs/docs folder now, which also means all the actual docs file are not showing as "last edited by merging the repos", but rather as last edit by the respective commit.
• The commit that moved things into the docs subfolder also adjusted all the paths etc., so that at this stage CI keeps working already. This change was easy to backport, to make sure all the branches have minimal working CI from the beginning.
• I ran postgrest-style before the merge on the docs branch, to avoid having the main repo's style check fail on the python and nix files.

The other changes in #3120 are still open, i.e. merging the nix tools and static files. Those would happen on the main branch only, so we can work on them in that PR.

Since the basic approach is the same as discussed before, I assume @steve-chavez and @laurenceisla you are ok with that - and merged the branches immediately without PRs. This saves another long run of CI for each PR - I already ran CI on the branches directly in my fork.

[wolfgangwalther]

I did the readthedocs configuration now, but our new branch names now give us this:

Obviously, readthedocs is taking the branch name to show here - but also stable does not work anymore. Stable relies on the branch names being semver, so our rel- prefix doesn't work.

So we need to rename our release branches. The obvious choice is rel-X.Y -> vX.Y. This does not conflict with our tags, which always have 3 or 4 pieces. However, since we decided to only show the latest minor for each major in the docs... maybe we should just use the major only and nothing more? So our release branches would be v10, v11, v12.. and when we release v12.2.0, we force push the v12 branch from whatever it had for v12.0.x to the new v12.2.0. This also has the benefit, that we only need to enable new versions on readthedocs on a new major - and don't need to update after each minor version. Of course, all released versions are still available via tag, even if we force push the release branches.

WDYT?

[steve-chavez]

So we need to rename our release branches. The obvious choice is rel-X.Y -> vX.Y.
maybe we should just use the major only and nothing more?

Agree with both 👍

[steve-chavez]

We'll need a redirect on RTD too. Links like https://postgrest.org/en/v12.0/references/api/aggregate_functions.html are not working. We used those on the release page https://github.com/PostgREST/postgrest/releases/tag/v12.0.0.

[wolfgangwalther]

We'll need a redirect on RTD too.

Ah, thanks for the reminder. Will add those, too.

[wolfgangwalther]

Branches are renamed and redirects better than ever. One thing I was not able to do, yet, is to enable the stable build. This defaults to the latest tag, which is currently v12.0.2. But this tag doesn't have the docs branch, yet. So we can only enable the stable build for docs, once we have made the next release. I will add a TODO.

[wolfgangwalther]

So we can only enable the stable build for docs, once we have made the next release.

For the meantime, I added a temporary redirect, leading from /en/stable/.. to /en/v12/, which is almost as good.

[wolfgangwalther]

We can then still discuss how to proceed regarding the issues.

One argument against migrating all the issues to the main repo: If we migrate more than 100 issues over here, that means the first few pages of the issue tracker will be filled only with docs issues for a long time. They all receive new issue IDs and will show at the top. While I do want to have the docs issues visible in the main repo, so that they are not hidden and forgotten like right now... this would be way too much visibility.

As a first step, I decided to cut down on the number of issues we are actually talking about. We had 107 issues, before I started and after going through all of them, I was able to close 40 of them. If anyone feels strongly about an issue that I closed, we can of course re-open - but I think most of them should be pretty clearly already done or out of scope etc.

I then picked 14 issues that I was able to work on quickly in #3234. This leaves us with 53 issues, but with 25 issues per page, we'd still talk about more than 2 full pages in the issue tracker after migration.

An alternative could be to say:

• We'll keep the postgrest-docs repo locked down, so new issues will only be created in the main repo.
• We'll create some meta issues collecting links to the docs issues as TODO lists. This makes them a little bit more visible.
• When solving those issues, we will just check them off the lists.
• Every once in a while, we will go back to the archived docs repo, unarchive it, close all the checked off issues, and archive it again.

I think this could be a good compromise.

@steve-chavez WDYT?

[steve-chavez]

I then picked 14 issues that I was able to work on quickly in #3234. This leaves us with 53 issues, but with 25 issues per page, we'd still talk about more than 2 full pages in the issue tracker after migration.

Awesome work! 🔥

I think this could be a good compromise.

@wolfgangwalther Agree. LGTM!

[wolfgangwalther]

We'll create some meta issues collecting links to the docs issues as TODO lists. This makes them a little bit more visible.

All issues in the postgrest-docs repo have either been linked to in one of the issues I created or (are about to be) closed.