Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: fix wheel upload to anaconda [wheel build] #18598

Merged
merged 1 commit into from Jun 5, 2023
Merged

Conversation

andyfaff
Copy link
Contributor

Should hopefully fix #18597
@tylerjereddy

@andyfaff andyfaff requested a review from larsoner as a code owner May 31, 2023 17:30
@tylerjereddy tylerjereddy added backport-candidate This fix should be ported by a maintainer to previous SciPy versions. CI Items related to the CI tools such as CircleCI, GitHub Actions or Azure Official binaries Items related to the official SciPy binaries, including wheels and vendored libs labels May 31, 2023
@andyfaff andyfaff added this to the 1.11.0 milestone May 31, 2023
@andyfaff andyfaff removed Official binaries Items related to the official SciPy binaries, including wheels and vendored libs CI Items related to the CI tools such as CircleCI, GitHub Actions or Azure labels May 31, 2023
@andyfaff
Copy link
Contributor Author

Don't know why the page says I removed the labels, perhaps because I was adding labels at the same time?

@andyfaff andyfaff added Official binaries Items related to the official SciPy binaries, including wheels and vendored libs CI Items related to the CI tools such as CircleCI, GitHub Actions or Azure labels May 31, 2023
@tupui
Copy link
Member

tupui commented May 31, 2023

@andyfaff what is the upload logic with the maintenance branch? I could not find the information in our doc https://scipy.github.io/devdocs/dev/contributor/continuous_integration.html#wheel-builds

Are the builds going to scientific-python-nightly-wheels as well or somewhere else?

@andyfaff
Copy link
Contributor Author

For wheels built on cirrus the upload logic for the maintenance branch ("staging") is:

    # If it's a push event to a maintenance branch, and the commit message contains
    # '[wheel build]' then upload to staging
    COMMIT_MSG=$(git log --no-merges -1)
    if [[ "$COMMIT_MSG" == *"[wheel build]"* ]] && [[ $CIRRUS_BRANCH == maintenance* ]]; then
          export IS_PUSH="true"
    fi

The upload script for all wheel building is controlled by upload_wheels.sh. The uploads for staging are directed to is to multibuild-wheels-staging. Cron jobs get uploaded to scientific-python-nightly-wheels.

The logic for GHA generated wheels is:

      # upload to staging if it's a push to a maintenance branch and the last
      # commit message contains '[wheel build]'
      IS_PUSH: ${{ github.event_name == 'push' && startsWith(github.ref, 'refs/heads/maintenance') && contains(needs.get_commit_message.outputs.message, '1') }}
      IS_SCHEDULE_DISPATCH: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}

@andyfaff
Copy link
Contributor Author

The logic for GHA and cirrusci is pretty much congruent.

@tupui
Copy link
Member

tupui commented May 31, 2023

I see that but that's not very clear at first glance. Could you add some documentation itself? Right where we currently describe CirrusCI?

One point which was not clear is where we need to put the secrets. I was on a call with Sebastian looking at this for NumPy and it was not straightforward.

Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, let's give this a go and see if it resolves the problem.

@rgommers
Copy link
Member

rgommers commented Jun 5, 2023

One point which was not clear is where we need to put the secrets. I was on a call with Sebastian looking at this for NumPy and it was not straightforward.

Cirrus CI works very differently than GHA. Last time things worked as expected for me though after following https://cirrus-ci.org/guide/writing-tasks/#encrypted-variables.

@rgommers rgommers merged commit 6e018d9 into scipy:main Jun 5, 2023
37 checks passed
@tupui
Copy link
Member

tupui commented Jun 5, 2023

Yes but then did you update the secret on CirrusCI? The hash in our config should be updated too no?

@rgommers
Copy link
Member

rgommers commented Jun 5, 2023

This PR was about wheels uploading from the maintenance branch to the staging bucket, not the nightly one. That secret did not change. The jobs passed (https://cirrus-ci.com/task/4708269558595584), so this seems to be fine and this can be backported.

Separately, I'll look at the nightly uploads secret.

@tupui
Copy link
Member

tupui commented Jun 5, 2023

Yes I know, sorry I was maybe not clear as I was discussing this in multiple places. I am talking about the new one.

tylerjereddy pushed a commit to tylerjereddy/scipy that referenced this pull request Jun 13, 2023
@tylerjereddy tylerjereddy removed the backport-candidate This fix should be ported by a maintainer to previous SciPy versions. label Jun 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Items related to the CI tools such as CircleCI, GitHub Actions or Azure Official binaries Items related to the official SciPy binaries, including wheels and vendored libs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI, BUG: Cirrus wheel upload fails on maintenance branch
4 participants