Added automatic release to pypi on tag. #66

cmccully · 2021-10-27T19:55:53Z

No description provided.

.github/workflows/release.yml

saimn · 2021-10-29T14:40:25Z

@cmccully : could you rebase now that #59 is merged ? (This PR fixes a few useful things for the packaging)

pyproject.toml

cmccully · 2021-10-29T19:16:32Z

@saimn any ideas why it is not running the python tests?

saimn · 2021-10-29T21:14:46Z

@cmccully - It must configured on cibuildwheel: https://cibuildwheel.readthedocs.io/en/stable/options/#test-command
An example here: https://github.com/musevlt/mpdaf-wheels/blob/master/.github/workflows/build.yml#L17-L18
Since you already used pyproject.toml to configure cibuildwheel the config should go there.

cmccully · 2021-10-29T21:48:37Z

It's can't just be its own separate GitHub workflow file?

…

On Fri, Oct 29, 2021 at 5:14 PM Simon Conseil ***@***.***> wrote: @cmccully <https://github.com/cmccully> - It must configured on cibuildwheel: https://cibuildwheel.readthedocs.io/en/stable/options/#test-command An example here: https://github.com/musevlt/mpdaf-wheels/blob/master/.github/workflows/build.yml#L17-L18 Since you already used pyproject.toml to configure cibuildwheel the config should go there. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#66 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIVYHGVFCN35LERYVMCQIDUJMFFBANCNFSM5G3FO7LA> .

codecov · 2021-11-01T19:35:26Z

Codecov Report

Merging #66 (a403cc9) into master (09a1c21) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #66   +/-   ##
=======================================
  Coverage   93.09%   93.09%           
=======================================
  Files           7        7           
  Lines         782      782           
  Branches       12       12           
=======================================
  Hits          728      728           
  Misses         54       54

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09a1c21...a403cc9. Read the comment docs.

cmccully · 2021-11-01T20:10:25Z

@saimn Looks the unit tests workflow was disabled. I re-enabled it. I removed the cron scheduled tests because that appears to be why we got disabled. I'm not sure right now it the cron tests add enough value to keep them going. Otherwise the PR should be ready to go.

.github/workflows/release.yml

saimn · 2021-11-02T10:39:33Z

Looks the unit tests workflow was disabled. I re-enabled it. I removed the cron scheduled tests because that appears to be why we got disabled.

Indeed, Github does that when there is no activity on the repo since some time, but I think once you reenable it it will not be disabled again.

However the normal unit test workflow is not run with the wheels, which I think would be good to have. That's the point of cibuildwheel's CIBW_TEST_REQUIRES and CIBW_TEST_COMMAND, once the wheels are build it will install it in a virtualenv and run the tests.

pyproject.toml

cmccully · 2021-11-03T18:45:03Z

Indeed, Github does that when there is no activity on the repo since some time, but I think once you reenable it it will not be disabled again.

I feel like for this particular repo the cron tests are a little bit of overkill. We rely on very little outside packages. The packages we do test against (numpy) for example are tested at merge for specific versions. Given that the set of versions we run against is static (i.e. I don't think we would automatically start testing against numpy 1.25 if it was released tomorrow) I'm not sure what periodic runs of the tests buys us.

saimn · 2021-11-12T21:41:01Z

pyproject.toml

+
+[tool.cibuildwheel]
+# Skip pypy on mac due to numpy/accelerate issues
+skip = "pp37-macosx_x86_64"


I think we could skip pypy for all platforms. I'm not sure how binary wheels are supported on pypy, and astropy doesn't build pypy wheels.

pyproject.toml

saimn · 2021-11-16T08:24:13Z

@cmccully - Tests are now running after each wheel creation though some of them are failing, sometimes because some wheels are missing for dependencies (scipy/astropy for py3.10), and also it seems is a real error on linux 32bits with test_median_clean:
https://github.com/astropy/astroscrappy/runs/4217562413?check_suite_focus=true#step:4:1500
I guess we could forget about 32bits, at least for now, since the usage is probably very low and and other project are considering removing 32bits support.

cmccully · 2021-11-18T21:18:26Z

I think I agree with that @saimn. Should we flag that as a known issue? (Though I'm a little surprised there is an issue here).

saimn · 2021-11-18T21:27:54Z

I also skipped musllinux (raspberry I think) which is built by default by cibuildwhel since recently, so the jobs are now green and we have wheels for the most common platforms (except sadly cp10 on linux, where for some reason scipy won't install, but it's not specific to astroscrappy and we can fix that later hopefully).

astroscrappy-0.1.dev1+gf5963fd-cp310-cp310-macosx_10_9_x86_64.whl
astroscrappy-0.1.dev1+gf5963fd-cp37-cp37m-macosx_10_9_x86_64.whl
astroscrappy-0.1.dev1+gf5963fd-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
astroscrappy-0.1.dev1+gf5963fd-cp37-cp37m-win32.whl
astroscrappy-0.1.dev1+gf5963fd-cp37-cp37m-win_amd64.whl
astroscrappy-0.1.dev1+gf5963fd-cp38-cp38-macosx_10_9_x86_64.whl
astroscrappy-0.1.dev1+gf5963fd-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
astroscrappy-0.1.dev1+gf5963fd-cp38-cp38-win32.whl
astroscrappy-0.1.dev1+gf5963fd-cp38-cp38-win_amd64.whl
astroscrappy-0.1.dev1+gf5963fd-cp39-cp39-macosx_10_9_x86_64.whl
astroscrappy-0.1.dev1+gf5963fd-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
astroscrappy-0.1.dev1+gf5963fd-cp39-cp39-win32.whl
astroscrappy-0.1.dev1+gf5963fd-cp39-cp39-win_amd64.whl
astroscrappy-0.1.dev1+gf5963fd.tar.gz

saimn · 2021-11-18T21:30:25Z

Should we flag that as a known issue? (Though I'm a little surprised there is an issue here).

It was happening on several i686 builds, so it seems real but 🤷‍♂️

      def test_median_clean():
          # Because our image only contains single cosmics, turn off
          # neighbor detection. Also, our cosmic rays are high enough
          # contrast that we can turn our detection threshold up.
          _mask, clean = detect_cosmics(imdata, readnoise=10., gain=1.0,
                                        sigclip=6, sigfrac=1.0, cleantype='median')
      
          assert (clean[crmask] != imdata[crmask]).sum() == crmask.sum()
      
          # Run it again on the clean data. We shouldn't find any new cosmic rays
          _mask2, _clean2 = detect_cosmics(clean, readnoise=10., gain=1.0,
                                           sigclip=6, sigfrac=1.0, cleantype='median')
  >       assert _mask2.sum() == 0
  E       assert 2072 == 0

cmccully · 2021-11-18T21:36:52Z

It does seem real, but I have no idea where there would be a difference between 32 and 64 bits in the code.

So are we ready to merge this? @saimn

saimn · 2021-11-18T21:44:38Z

Yes, I think we are ! 🎉

Added automatic release to pypi on tag.

403d244

cmccully requested a review from saimn October 27, 2021 19:55

cmccully added 8 commits October 27, 2021 16:12

Minor fixes to build on different versions of python.

a57ed26

Syntax fix in github actions.

31b3959

Attempt to override numpy install on pypy on mac

a612d9c

Attempt to override numpy install on pypy on mac

87243b9

Attempt to override numpy install on pypy on mac

d1bebbc

Attempt to override numpy install on pypy on mac

8340221

Attempt to override numpy install on pypy on mac

11e9af8

Removed pypy build on macos due to accelerate backend issues with numpy.

7682c60

saimn reviewed Oct 29, 2021

View reviewed changes

.github/workflows/release.yml Outdated Show resolved Hide resolved

saimn reviewed Oct 29, 2021

View reviewed changes

.github/workflows/release.yml Outdated Show resolved Hide resolved

saimn reviewed Oct 29, 2021

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

Merged with main and followed Simon's suggestions.

c2e1b8f

cmccully added 2 commits November 1, 2021 15:14

Removed cron workflow for the moment.

9a51302

Updated release date.

1a0822e

saimn reviewed Nov 2, 2021

View reviewed changes

.github/workflows/release.yml Show resolved Hide resolved

.github/workflows/release.yml Show resolved Hide resolved

saimn reviewed Nov 2, 2021

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

saimn mentioned this pull request Nov 11, 2021

New release needed #67

Closed

Run tests after building wheels

48eaf80

saimn reviewed Nov 12, 2021

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

saimn added 5 commits November 12, 2021 22:42

Fix quotes for the test command

80fed90

Skip pypy and py36, more generic test command

124322d

Skip some wheels

b7fbfaf

Skip more wheels

f37d68f

Skip linux 32 bits wheels

98257f7

Skip musllinux

a403cc9

cmccully merged commit 3e909d9 into astropy:master Nov 18, 2021

cmccully deleted the feature/auto_release branch November 29, 2021 16:50

This was referenced Apr 17, 2024

Build with Numpy 2.0, remove setup.cfg and more #85

Merged

Investigate test_median_clean failures #86

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added automatic release to pypi on tag. #66

Added automatic release to pypi on tag. #66

cmccully commented Oct 27, 2021

saimn commented Oct 29, 2021

cmccully commented Oct 29, 2021

saimn commented Oct 29, 2021

cmccully commented Oct 29, 2021 via email

codecov bot commented Nov 1, 2021 •

edited

Loading

cmccully commented Nov 1, 2021

saimn commented Nov 2, 2021

cmccully commented Nov 3, 2021

saimn Nov 12, 2021

saimn commented Nov 16, 2021

cmccully commented Nov 18, 2021

saimn commented Nov 18, 2021

saimn commented Nov 18, 2021

cmccully commented Nov 18, 2021

saimn commented Nov 18, 2021

Added automatic release to pypi on tag. #66

Added automatic release to pypi on tag. #66

Conversation

cmccully commented Oct 27, 2021

saimn commented Oct 29, 2021

cmccully commented Oct 29, 2021

saimn commented Oct 29, 2021

cmccully commented Oct 29, 2021 via email

codecov bot commented Nov 1, 2021 • edited Loading

Codecov Report

cmccully commented Nov 1, 2021

saimn commented Nov 2, 2021

cmccully commented Nov 3, 2021

saimn Nov 12, 2021

Choose a reason for hiding this comment

saimn commented Nov 16, 2021

cmccully commented Nov 18, 2021

saimn commented Nov 18, 2021

saimn commented Nov 18, 2021

cmccully commented Nov 18, 2021

saimn commented Nov 18, 2021

codecov bot commented Nov 1, 2021 •

edited

Loading