Change release procedure to use only pull requests #5250

albertvillanova · 2022-11-16T14:35:32Z

This PR changes the release procedure so that:

it only make changes to main branch via pull requests
it is no longer necessary to directly commit/push to main branch

Close #5251.

HuggingFaceDocBuilderDev · 2022-11-16T14:40:33Z

The documentation is not available anymore as the PR was closed or merged.

lhoestq

cool thanks ! a few comments:

lhoestq · 2022-11-16T14:53:57Z

setup.py

+     ```
+     git checkout main
+     git pull upstream main
+     git checkout -b release


(nit) in case a branch with this name already exists

Suggested change

git checkout -b release

git checkout -b release-VERSION

I made it this way to make it simpler: as remote branches are deleted after the PR is merged.

If a local branch with the same name exists, then previous command generated this error:

fatal: A branch named 'release' already exists.

I have changed the release branch name so that it triggers docs generation: 88c9c83

Context:

Docs are not generated after latest release #5251

it's fine to not have it unique then

If we have it unique we won't need to delete the branch on a local clone of the repo though - the branch exists temporarily on upstream, but locally it's not deleted when the PR is merged

The unique naming for the dev version would be more complicated though: something like vX.X.X+1.dev0-dev-version?

Maybe it is easier to add a step before to delete the local branch dev-version? See cbd5874

Yea let's add a step to delete the local branch then

OK. That step was already added: cbd5874

lhoestq · 2022-11-16T14:56:56Z

setup.py

+   - Choose a tag: Introduce the new VERSION as tag, that will be created when you publish the release
+     - Create new tag VERSION on publish


I'm wondering if it will trigger the release-conda CI job - it's configured like

on: push: tags: - "[0-9]+.[0-9]+.[0-9]+*"

I guess related to:

Docs are not generated after latest release #5251

@lhoestq maybe we could align the triggering of release-conda to be the same as the one for generating the docs? That is: push to branch v*-release

Or maybe using release triggering instead?

EDIT:
~~Indeed, conda-release was not triggered for 2.7.0 release...~~
Indeed it was triggered: https://github.com/huggingface/datasets/actions/runs/3478362090/jobs/5815637910

and it was uploaded to huggingface channel: https://anaconda.org/huggingface/datasets

but not to conda-forge channel: https://anaconda.org/conda-forge/datasets

Any idea why, @lhoestq?

We maintain the huggingface channel but not the conda forge one, which is maintained by the community

EDIT: it's now available on conda forge btw ;)

I have reverted: 8694901

Push tag for both sounds good :)

lhoestq · 2022-11-16T14:58:22Z

setup.py

+     ```
+     git add -u
+     git commit -m "Release: VERSION"
+     git push upstream release


Suggested change

git push upstream release

git push upstream release-VERSION

HuggingFaceDocBuilderDev · 2022-11-16T16:18:31Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

HuggingFaceDocBuilderDev · 2022-11-16T16:31:22Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

HuggingFaceDocBuilderDev · 2022-11-16T16:39:38Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

albertvillanova · 2022-11-17T10:29:11Z

Little recap:

The release-conda GH action was properly triggered by push-tag event: therefore I guess this event is also created when we publish a release and create a tag within it (as it is the case in the new procedure)
- However, the package was only uploaded to huggingface channel and not to conda-forge channel
  - Why? Need to address this.
    - Reply by @lhoestq: Change release procedure to use only pull requests #5250 (comment)
      - We only maintain the huggingface channel
      - The conda-forge channel is maintained by the community; the 2.7.0 has been finally added as well to this channel
The generate-documentation GH action will be triggered by the push-to-branch event if we align the name of the release branch with the expected regex v*-release
- The naming has been aligned in the new procedure
- Question: why do we have different triggering events for generate-doc and release-conda? Maybe we could set the same for both: either push-tag (when publishing the release), or push-to-branch
  - I think it will be better to use the push-tag event because in the new release procedure this happens later (when we publish the release), once we have already tested that everything works using the test-PyPI; on the contrary, the push-to-branch event happens before, even before opening the release PR: we could see afterwards that there is an issue, and cancel the Pull Request, but the docs and conda-package will already be published.
For the naming of the dev-version branch/PR, instead of having a complicated version naming, I'm proposing:
- Using always the same branch name dev-version
- Just include a step to delete this branch locally if it exists: git branch -D dev-version
- The remote version will not exist because it is deleted once the PR is merged
- This approach is approved by @lhoestq: Change release procedure to use only pull requests #5250 (comment)

albertvillanova · 2022-11-17T13:05:39Z

Just one question to be addressed: why do we have different triggering events for generate-doc and release-conda? Maybe we could set the same for both: either push-tag (when publishing the release), or push-to-branch

I think it will be better to use the push-tag event because in the new release procedure this happens later (when we publish the release), once we have already tested that everything works using the test-PyPI; on the contrary, the push-to-branch event happens before, even before opening the release PR: we could see afterwards that there is an issue, and cancel the Pull Request, but the docs and conda-package will already be published.

We could even use the release-published event instead: 8694901

This reverts commit 8694901.

lhoestq

Looks all good to me ! Thanks for improving the release procedure :)

lhoestq · 2022-11-17T14:21:12Z

setup.py

+   - Choose a tag: Introduce the new VERSION as tag, that will be created when you publish the release
+     - Create new tag VERSION on publish


Push tag for both sounds good :)

albertvillanova · 2022-11-22T14:13:19Z

@lhoestq now that we have push-tag event for both build_documentation and release-conda, we have no constraint on the naming of the release branch:

we could name it simpler: maybe as you suggested above: Change release procedure to use only pull requests #5250 (comment)
release-VERSION instead of vVERSION-release (we do not use the prefix "v" anywhere in our repo)

Change release procedure

1f82270

albertvillanova mentioned this pull request Nov 16, 2022

Protect the main branch from inadvertent direct pushes #5249

Closed

4 tasks

lhoestq reviewed Nov 16, 2022

View reviewed changes

albertvillanova mentioned this pull request Nov 16, 2022

Docs are not generated after latest release #5251

Closed

Change release branch name to trigger docs generation

88c9c83

Trigger conda-release on published release

8694901

Delete local dev-version branch

cbd5874

albertvillanova requested a review from lhoestq November 17, 2022 10:32

Revert "Trigger conda-release on published release"

85a7e01

This reverts commit 8694901.

lhoestq approved these changes Nov 17, 2022

View reviewed changes

Trigger build_documentation on push tag as release-conda

c87ce98

Rename release branch

32836bd

albertvillanova merged commit 703b843 into huggingface:main Nov 22, 2022

albertvillanova deleted the change-release-procedure branch November 22, 2022 16:27

This was referenced Dec 1, 2022

No documentation for main branch is built #5326

Closed

Fix docs building for main #5328

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change release procedure to use only pull requests #5250

Change release procedure to use only pull requests #5250

albertvillanova commented Nov 16, 2022 •

edited

HuggingFaceDocBuilderDev commented Nov 16, 2022 •

edited

lhoestq left a comment

lhoestq Nov 16, 2022

albertvillanova Nov 16, 2022

albertvillanova Nov 16, 2022

lhoestq Nov 16, 2022

albertvillanova Nov 16, 2022

lhoestq Nov 16, 2022

lhoestq Nov 16, 2022

albertvillanova Nov 16, 2022 •

edited

lhoestq Nov 17, 2022

albertvillanova Nov 17, 2022

lhoestq Nov 16, 2022

albertvillanova Nov 16, 2022 •

edited

albertvillanova Nov 16, 2022 •

edited

albertvillanova Nov 16, 2022 •

edited

lhoestq Nov 17, 2022 •

edited

albertvillanova Nov 17, 2022

albertvillanova Nov 17, 2022

lhoestq Nov 17, 2022

lhoestq Nov 16, 2022

HuggingFaceDocBuilderDev commented Nov 16, 2022

HuggingFaceDocBuilderDev commented Nov 16, 2022

HuggingFaceDocBuilderDev commented Nov 16, 2022

albertvillanova commented Nov 17, 2022 •

edited

albertvillanova commented Nov 17, 2022 •

edited

lhoestq left a comment

lhoestq Nov 17, 2022

albertvillanova commented Nov 22, 2022 •

edited

		- Choose a tag: Introduce the new VERSION as tag, that will be created when you publish the release
		- Create new tag VERSION on publish

Change release procedure to use only pull requests #5250

Change release procedure to use only pull requests #5250

Conversation

albertvillanova commented Nov 16, 2022 • edited

HuggingFaceDocBuilderDev commented Nov 16, 2022 • edited

lhoestq left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

albertvillanova Nov 16, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

albertvillanova Nov 16, 2022 • edited

Choose a reason for hiding this comment

albertvillanova Nov 16, 2022 • edited

Choose a reason for hiding this comment

albertvillanova Nov 16, 2022 • edited

Choose a reason for hiding this comment

lhoestq Nov 17, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Nov 16, 2022

HuggingFaceDocBuilderDev commented Nov 16, 2022

HuggingFaceDocBuilderDev commented Nov 16, 2022

albertvillanova commented Nov 17, 2022 • edited

albertvillanova commented Nov 17, 2022 • edited

lhoestq left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

albertvillanova commented Nov 22, 2022 • edited

albertvillanova commented Nov 16, 2022 •

edited

HuggingFaceDocBuilderDev commented Nov 16, 2022 •

edited

albertvillanova Nov 16, 2022 •

edited

albertvillanova Nov 16, 2022 •

edited

albertvillanova Nov 16, 2022 •

edited

albertvillanova Nov 16, 2022 •

edited

lhoestq Nov 17, 2022 •

edited

albertvillanova commented Nov 17, 2022 •

edited

albertvillanova commented Nov 17, 2022 •

edited

albertvillanova commented Nov 22, 2022 •

edited