Readme/docs should not be published before release

[owais]

Today all Splunk Otel distros have at least one README.md file in their repos. This is the most common and obvious source of documentation for users especially when getting started.

The problem is that this document reflects the state of upcoming release instead of the current stable one. Let's say the current stable release for splunk-otel-js-web is 1.1 and the team is working on features that'll make it to 1.2. Every time we land a new feature or make changes, we'll update the README.md file (in addition to other markdown docs in the repo) to document the new feature. At this point any users reading this documentation and trying it with the latest available release could end up having a very bad experience as some listed features might not work and the distro might fail to function completely.

Possible solutions:

1. Use a new develop branch as the development branch and only merge it to main when a new release is issued. This requires us to use main as the default branch in github but develop against the develop branch which can be inconvenient or confusing for devs and advanced users both.

2. Add a new README_dev.md file which should be the main source of truth. Every release would copy this file to README.md. This works but can be a bit awkward especially if a project has a lot of markdown docs that are meant to be consumed directly from github.

3. Have another central github repository that would host markdown docs for all projects. Each project's README would have a permanent link to that repo. A script would copy docs from each repo to the docs repo during very release.

4. Add a notice on top of the README.md file saying that the documentation is for the in-development version with a link to the same doc for the stable version. For example, splunk-otel-python has the following header in readme right now:

The release process would have to include two additional steps:

4.a. Remove the header from README.md file above.
5.b. Add a new commit after the release commit that formally sets the repository in dev mode for the next release. This commit will add the header back with links to the latest stable release.

---

4. makes most sense to me. It does not involve duplicating files in the same repo or between repos, or setting up a weird development flow with branches. Docs for each version are already published to Github. We just need to link to them. The couple of additional steps can be easily automated as part of the release process.

[flands]

OTel has the same problem so even if we fix this here it remains upstream. I see a couple additional options:

1. Product documentation is what we refer people to -- README exists, but product documentation is source of truth
2. Use release tag -- if you go to release tag the README is correct -- might not be the best experience, but technically if you build main (some customers might) you need the latest release note.

[pellared]

I think ad.2 is the same as @owais option 4. Correct?

[owais]

Product documentation is what we refer people to -- README exists, but product documentation is source of truth

We can add a header that says the README is for dev version and link to product docs but then we need a way to sync product docs on every release. Which should happen anyway and might be the desired long term solution but I feel until we (GDI eng) own product docs relevant to us, it'll be hard to do.

I'd go with 2 for now. which seems to be the same as 4. option I suggested above.

[pellared]

Option 4. Also because I am using it for my personal side-project and it is simple.

We can consider also doing GH pages. They could be published to gh-pages branch when we do a release tag. But I think it is rather not worth the effort and complexity.

[owais]

I think when/if the docs get too big for a single readme file, this is a very viable thing to do and once we have that, this problem goes away completely. But the question still remains about what projects should do when the main project README is the only doc (most otel distributions).

[pellared]

Add a notice on top of the README.md file saying that the documentation is for the in-development version with a link to the same doc for the stable version.

I think it is the easiest way. This is how I was doing it: https://github.com/goyek/goyek/blob/e2a6db0e4d5ee1449657b78d1e29256711cbfb5b/README.md

I prefer having a single source of truth rather than e.g. making copies of the README in other locations. Because otherwise, it may be confusing for someone who wants to make a pull request. Moreover, someone may want to try using a not-yet-released version.

[iNikem]

In java repos we don't merge doc changes until after release

[mateuszrzeszutek]

Actually I've been planning to change that a little and merge them just before the release so that the release tag would always point to accurate docs, but yeah -- we're intentionally delaying merging doc changes so that main stays relevant to the latest release.

[owais]

This works if docs are published elsewhere like a website, GH pages or even a different repository. In that case, this is not even a problem to begin with. It's only an issue when the project README is used as (often the only) docs.

[flands]

Even if we solve the main issue the same problem exists for previous releases -- sounds like release notes should have an explicit link to docs so clear what version to refer to. Bigger picture -- this speaks to an issue with product docs that needs to be solved as well...

On the issue at hand, if docs are withheld then need that to be part of release process also means people are dependent on releases to get new things (some build collector manually today for example). I am good either way just again ask that we are consistent :)

[owais]

Even if we solve the main issue the same problem exists for previous releases

This isn't an issue for previous releases. It's strictly a problem when soomeone is reading the README file from the main repo page. We can easily link to README versions for specific tags in all existing releases to make finding old docs even more easier.

[mateuszrzeszutek]

One more idea, a combination of 1 & 3: we could make all documentation changes to some docs branch and merge it (automatically) into main as a part of the release process.
Maybe it's even possible to add a PR check that fails if you try to change README/docs directly on main.

[pellared]

Then you would need to create two PRs during development. One targeting main branch and another for docs branch.

[mateuszrzeszutek]

Well, that's what we already do in the Java repo 🤷‍♂️

[owais]

😬 I'd rather publish docs somewhere else like GH pages, another repo or just another directory in the same repo than having docs and code submitted as two separate PRs. It can be annoying for contributors and hard for reviewers to look at the whole feature end to end and spot UX issues.

[flands]

For Java and Python we landed with original number 4.

[MrAlias]

I do not see this as a problem as it is common practice to have HEAD of a repository contain not only documentation changes that have not been released, but also features and fixes. Unless an alternate git-flow is used, like mentioned in the first option, this is something all projects do.

I am hesitant to prescribe a solution in this specification if we want to resolve this. It would impose development practices that will likely not be native to many communities. Many development projects I have worked on go off the assumption that by using HEAD "your millage may vary". Particularly I have found many Go projects to take this approach.

Maybe as a corollary to this, I would add that this is a prime example as to why we need to link to tagged versions of a project and not the HEAD.

[MrAlias]

I'm going to lock this conversation due to the mentioned reason that this is likely not something we want to specify. Please open an issue if you think otherwise and we can discuss a resolution there.