TST: Use `meson` for testing `f2py` #25111

HaoZeke · 2023-11-11T16:58:52Z

Do not merge before #25073 and #25181.

Essentially:

Switch tests to the meson backend
Detect compilers correctly for tests via meson
Run full testsuite ~~(blocked by the .pyf + -m interaction currently)~~
- Linux
- MacOS
- ~~Windows~~ See CI, TST: have a f2py-specific CI run and properly skip f2py tests elsewhere #25134

Also reworked test_docs to run with spin.

Additionally (features added):

The compile() call will now use the meson backend as well
- Ideally I'd like this command to go away completely..

Note: It is now the users' responsibility to ensure a compatible toolchain is present. However, this does mean that on the CI there are more failure modes e.g. without distutils it is hard to figure out if the CI has msvc or not, so conditional skipping is harder than it used to be. Conversely, GFortran and MSVC simply do not mix in the meson world-view. So, there is a catch-all skip for those. However, the tests are run fully on the CI as well, just not all flavors, e.g. FreeBSD fails the fortran compiler check, some of the Windows CI variants only find cl by default.

Note II: It is worth remembering that most (compiled) Windows tests were skipped even under distutils since 2017 due to #9673, perhaps with the notable exception of the (non-Fortran) test_array_from_pyobj. This is more of a status quo PR, to ensure F2PY is tested beyond its parser on Linux and MacOS (and to whatever extent Windows was tested before as well).

numpy/f2py/f2py2e.py

mattip · 2023-11-15T08:31:05Z

Could we do the merging the other way around: get this PR to work first and then revisit #25114?

HaoZeke · 2023-11-19T03:10:07Z

Could we do the merging the other way around: get this PR to work first and then revisit #25114?

Unfortunately not, see #25181 (comment)

mattip

LGTM. A few nits.

numpy/distutils/tests/test_build_ext.py

mattip · 2023-11-19T06:04:56Z

numpy/f2py/__init__.py

@@ -92,17 +92,16 @@ def compile(source,
        with open(fname, 'w') as f:
            f.write(source)

-        args = ['-c', '-m', modulename, f.name]
+        args = ['-c', '--backend', 'meson', '-m',
+                modulename, f.name]


shouldn't meson be made the default?

Not yet, for python<3.12 the default (with a warning) is still distutils. >=3.12 it automatically (and noisily) switches to meson

However, in terms of the testing infrastructure, there's no point keeping the distutils build and test routines, so we just default to using meson explicitly here.

charris · 2023-11-21T17:17:40Z

Please squash this when it gets merged.

The version is too old to be of use

rgommers · 2023-11-23T20:18:15Z

@HaoZeke and I just discussed that we plan to fix the speed issue in stages: add some slow markers in this PR, merge this PR, switch to pytest fixtures to speed up the tests after, and then deal with some CI sanitation to run compile tests less often.

Should finish in around a minute now

HaoZeke · 2023-11-23T23:28:27Z

The tests (after annotating with slow) will finish in less than a minute now, further speed up needs #25239, so perhaps this is ready for now @rgommers :)

rgommers

Thanks Rohit, LGTM now - let's give this a go!

The user of this import was removed in the cleanup in numpy#25111, post-2.26.

* MAINT,TST: Use meson for compiler checks [f2py] * MAINT: Cache the compiler checks [f2py] * MAINT,TST: Always use meson [f2py] * TST: Rework to have a build_meson [f2py] * MAINT,TST: Simplify the meson backend [f2py] * TST: Use the build_meson function [f2py] * MAINT,TST: Minor cleanup [f2py] * TST: Ensure TestDocAdvanced runs with spin [f2py] * TST: Use cleanup meson backend [f2py] * MAINT,TST: Generalize build_meson [f2py] * TST: Use a helper for spin tests [f2py] * MAINT: Simplify meson backend [f2py] * TST: Handle unsupported compilers [f2py] * TST: Fix gibberish in [f2py] documentation test * CI: Add meson for cygwin runs * TST: Skips for 32-bit errors [f2py] * TST: Skip for cygwin since meson is old [f2py] * CI: Revert grabbing meson on cygwin The version is too old to be of use * TST: Cleanup old distutils builder [f2py] * TST: Skip cygwin better [f2py] * TST: Don't touch distutils * MAINT: Vendor in distutils testing requirement * TST: Try removing cygwin restrictions * MAINT: Cleanup some tests [f2py] * TST: Try to use concurrency for i/o bounds [f2py] * TST: Mark slow tests [f2py] Should finish in around a minute now

HaoZeke added 05 - Testing component: numpy.f2py component: CI Meson Items related to the introduction of Meson as the new build system for NumPy labels Nov 11, 2023

HaoZeke mentioned this pull request Nov 11, 2023

BUG: Disallow shadowed modulenames #25114

Closed

HaoZeke force-pushed the testf2py312 branch from e56227f to 2afaf25 Compare November 11, 2023 23:42

HaoZeke commented Nov 11, 2023

View reviewed changes

numpy/f2py/f2py2e.py Outdated Show resolved Hide resolved

HaoZeke mentioned this pull request Nov 11, 2023

TST: Windows f2py tests hang indefinitely with msys gcc-fortran on PATH #9342

Open

HaoZeke force-pushed the testf2py312 branch 2 times, most recently from 508490a to ac6cc34 Compare November 12, 2023 01:28

charris changed the title ~~TST: Use meson for testing f2py~~ TST: Use meson for testing f2py Nov 12, 2023

HaoZeke mentioned this pull request Nov 12, 2023

BLD,BUG: quadmath required where available [f2py] #25073

Merged

HaoZeke force-pushed the testf2py312 branch from 8b79255 to 2ad6a26 Compare November 12, 2023 04:39

This was referenced Nov 12, 2023

RFC: Getting rid of numpy.f2py.compile() #25122

Closed

DOC: f2py CLI documentation enhancements #25124

Merged

HaoZeke force-pushed the testf2py312 branch from 2ad6a26 to bab8809 Compare November 13, 2023 14:57

charris added the 09 - Backport-Candidate PRs tagged should be backported label Nov 13, 2023

This was referenced Nov 18, 2023

MAINT,BUG: Never import distutils above 3.12 [f2py] #25123

Merged

BUG: Disallow shadowed modulenames #25181

Merged

HaoZeke force-pushed the testf2py312 branch from bab8809 to ad469a4 Compare November 19, 2023 03:03

HaoZeke force-pushed the testf2py312 branch from ad469a4 to 7f01e49 Compare November 19, 2023 05:58

HaoZeke marked this pull request as ready for review November 19, 2023 05:59

mattip reviewed Nov 19, 2023

View reviewed changes

HaoZeke force-pushed the testf2py312 branch 2 times, most recently from c411a21 to 687d8ce Compare November 19, 2023 23:20

HaoZeke requested a review from mattip November 20, 2023 00:07

charris mentioned this pull request Nov 20, 2023

BUG: Disallow shadowed modulenames #25205

Merged

HaoZeke added 10 commits November 23, 2023 18:18

TST: Fix gibberish in [f2py] documentation test

74934a5

CI: Add meson for cygwin runs

4154877

TST: Skips for 32-bit errors [f2py]

fe5bf64

TST: Skip for cygwin since meson is old [f2py]

6cdecd0

CI: Revert grabbing meson on cygwin

dd6f221

The version is too old to be of use

TST: Cleanup old distutils builder [f2py]

ef17ab5

TST: Skip cygwin better [f2py]

8d7ec08

TST: Don't touch distutils

b0f418e

MAINT: Vendor in distutils testing requirement

5bce3b4

TST: Try removing cygwin restrictions

9fc2006

HaoZeke force-pushed the testf2py312 branch from cb92252 to 9fc2006 Compare November 23, 2023 18:18

HaoZeke mentioned this pull request Nov 23, 2023

TST: Use fixtures for f2py tests #25239

Open

HaoZeke added 3 commits November 23, 2023 21:47

MAINT: Cleanup some tests [f2py]

2ac2902

TST: Try to use concurrency for i/o bounds [f2py]

63cbffb

TST: Mark slow tests [f2py]

64f4fd2

Should finish in around a minute now

HaoZeke force-pushed the testf2py312 branch from f4d92cd to 64f4fd2 Compare November 23, 2023 22:15

HaoZeke requested a review from rgommers November 23, 2023 23:28

rgommers approved these changes Nov 24, 2023

View reviewed changes

rgommers merged commit 8ac314f into numpy:main Nov 24, 2023
60 checks passed

HaoZeke deleted the testf2py312 branch November 24, 2023 13:39

stefanor added a commit to stefanor/numpy that referenced this pull request Dec 6, 2023

Restore missing asstr import

cd29a90

The user of this import was removed in the cleanup in numpy#25111, post-2.26.

stefanor mentioned this pull request Dec 6, 2023

BUG: Restore missing asstr import #25323

Merged

charris changed the title ~~TST: Use meson for testing f2py~~ TST: Use meson for testing f2py Dec 19, 2023

charris mentioned this pull request Dec 19, 2023

TST: Use meson for testing f2py #25427

Closed

7 tasks

charris removed the 09 - Backport-Candidate PRs tagged should be backported label Dec 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TST: Use `meson` for testing `f2py` #25111

TST: Use `meson` for testing `f2py` #25111

HaoZeke commented Nov 11, 2023 •

edited

mattip commented Nov 15, 2023

HaoZeke commented Nov 19, 2023

mattip left a comment

mattip Nov 19, 2023

HaoZeke Nov 19, 2023

HaoZeke Nov 19, 2023

charris commented Nov 21, 2023

rgommers commented Nov 23, 2023

HaoZeke commented Nov 23, 2023 •

edited

rgommers left a comment

TST: Use meson for testing f2py #25111

TST: Use meson for testing f2py #25111

Conversation

HaoZeke commented Nov 11, 2023 • edited

mattip commented Nov 15, 2023

HaoZeke commented Nov 19, 2023

mattip left a comment

Choose a reason for hiding this comment

mattip Nov 19, 2023

Choose a reason for hiding this comment

HaoZeke Nov 19, 2023

Choose a reason for hiding this comment

HaoZeke Nov 19, 2023

Choose a reason for hiding this comment

charris commented Nov 21, 2023

rgommers commented Nov 23, 2023

HaoZeke commented Nov 23, 2023 • edited

rgommers left a comment

Choose a reason for hiding this comment

TST: Use `meson` for testing `f2py` #25111

TST: Use `meson` for testing `f2py` #25111

HaoZeke commented Nov 11, 2023 •

edited

HaoZeke commented Nov 23, 2023 •

edited