DOC: f2py command to generate c wrapper respecting pyf file directives

[JulianBothorn]

Issue with current documentation:

Python 3.11.4, numpy 1.26.3, macOS Sonoma 14.2.1 (Intel)

The documentation does not explain how to generate a c wrapper that uses custom directives in a pyf file or it is not possible and the documentation contradicts itself.

Im trying to move a fortran project from numpy.disutils to meson. The previous developers created a long pyf file with custom function annotations. The example for how to move to meson requires the c wrapper generation step. Extension module construction mentions using pyf files as a source for c wrapper generation.

Running poetry run Python3 -m numpy.f2py -m slalib --lower slalib.pyf addet.f cc2s.f dc62s.f dr2af.f dvn.f etrms.f h2fk5.f nutc80.f prebn.f rverot.f tpv2c.f afin.f cc62s.f dcc2s.f dr2tf.f dvxv.f euler.f hfk5z.f oap.f prec.f rvgalc.f ue2el.f airmas.f cd2tf.f dcmpf.f drange.f e2h.f evp.f idchf.f oapqk.f preces.f rvlg.f ue2pv.f altaz.f cldj.f dcs2c.f dranrm.f earth.f fitxy.f idchi.f obs.f precl.f rvlsrd.f unpcd.f amp.f clyd.f dd2tf.f ds2c6.f ecleq.f fk425.f imxv.f pa.f prenut.f rvlsrk.f v2tp.f ampqk.f combn.f de2h.f ds2tp.f ecmat.f fk45z.f intin.f pav.f pv2el.f s2tp.f vdv.f aop.f cr2af.f deuler.f dsep.f ecor.f fk524.f invf.f pcd.f pv2ue.f sep.f veri.f aoppa.f cr2tf.f dfltin.f dsepv.f eg50.f fk52h.f kbj.f pda2h.f pvobs.f sepv.f vers.f aoppat.f cs2c.f dh2e.f dt.f el2ue.f fk54z.f m2av.f pdq2h.f pxy.f slalib-f2pywrappers.f vn.f aopqk.f cs2c6.f dimxv.f dtf2d.f epb.f fk5hz.f map.f permut.f range.f smat.f vxv.f atmdsp.f ctf2d.f djcal.f dtf2r.f epb2d.f flotin.f mappa.f pertel.f ranorm.f subet.f wait.f atms.f ctf2r.f djcl.f dtp2s.f epco.f galeq.f mapqk.f pertue.f rcc.f supgal.f xy2xy.f atmt.f daf2r.f dm2av.f dtp2v.f epj.f galsup.f mapqkz.f planel.f rdplan.f svd.f zd.f av2m.f dafin.f dmat.f dtps2c.f epj2d.f ge50.f moon.f planet.f refco.f svdcov.f bear.f dat.f dmoon.f dtpv2c.f epv.f geoc.f mxm.f plante.f refcoq.f svdsol.f caf2r.f dav2m.f dmxm.f dtt.f eqecl.f gmst.f mxv.f plantu.f refro.f tp2s.f caldj.f dbear.f dmxv.f dv2tp.f eqeqx.f gmsta.f nut.f pm.f refv.f tp2v.f calyd.f dbjin.f dpav.f dvdv.f eqgal.f h2e.f nutc.f polmo.f refz.f tps2c.f gresid.F random.F

Results in a c wrapper that does not respect the directives in the pyf file.
For example the sla_refro function has parameter 10 (ref) marked as intent(out) but the c wrapper keeps the parameter

Other options mentions that numpy.f2py -m can't be used with pyf files explaining the behaviour but contradicting the extension module construction section.

a simpler way to reproduce this is to alter the fib example file like so:

C FILE: FIB1.F
      SUBROUTINE FIB(A,N)
C
C     CALCULATE FIRST N FIBONACCI NUMBERS
C
      INTEGER N
      INTEGER A
      A = N + 1

      END
C END FILE FIB1.F

editing the pyf file to include

python module fibx ! in 
    interface  ! in :fibx
        subroutine fib(a,n) ! in :fibx:fib1.f
            integer intent(out) :: a
            integer :: n
        end subroutine fib
    end interface 
end python module fibx

and finally running poetry run Python3 -m numpy.f2py -m fibx --lower fib1.pyf fib1.f

The resulting .c file still has "a" marked as an input

p.s. I believe that the pyf file is correct as The smart way results in a correct wrap.

Idea or request for content:

Either an example of how to generate a c wrapper using a pyf file or the mention of signature files should be removed from Extension module construction to avoid confusion

[rgommers]

The notes section of https://numpy.org/doc/stable/f2py/usage.html#numpy.f2py.get_include seems relevant. That's what SciPy does in scipy.linalg, which has a lot of .pyf files. This may be a gap in the docs indeed (@HaoZeke may be able to judge better than I can).

[JulianBothorn]

The notes section of https://numpy.org/doc/stable/f2py/usage.html#numpy.f2py.get_include seems relevant. That's what SciPy does in scipy.linalg, which has a lot of .pyf files. This may be a gap in the docs indeed (@HaoZeke may be able to judge better than I can).

I may be misunderstanding but I don't think that my issue comes from missing c sources as those are supplied by numpy when I compile the __module.c. And the resulting .so file is working except that it is supposed to transform a call by reference to a call by value/a return value which it doesn't. The __module.c looks like I never supplied my signature (.pyf) file and f2py is guessing the necessary c wraps. I know however that my signature file is correct as skipping the intermediate step of generating the __module.c with numpy.f2py -c results in a .so file that corresponds to my signature file. There must be some kind for difference between the way f2py -c and f2py -m generate the c file as I interrupted the execution of f2py -c to grab the .c file from /var/.../ and it is correct. There may be an option missing in Extension module construction.

[HaoZeke]

Thanks for bringing this up @JulianBothorn. The documentation needs an update. The solution here is to just not pass any Fortran files if the signature is provided.

So:

# Incorrect, .pyf is ignored!
python -m numpy.f2py -m fibx --lower fib1.pyf fib1.f

As noted, this generates the .pyf "on the fly" as it were, so the resulting .c file is unexpected:

grep "Parameters" fibxmodule.c -A 3 -B 2
static char doc_f2py_rout_fibx_fib[] = "\
fib(a,n)\n\nWrapper for ``fib``.\
\n\nParameters\n----------\n"
"a : input int\n"
"n : input int";
/* extern void F_FUNC(fib,FIB)(int*,int*); */

The correct approach is to not provide any Fortran sources here.

# Correct
❯ python -m numpy.f2py fib1.pyf
Reading fortran codes...
	Reading file 'fib1.pyf' (format:free)
Post-processing...
	Block: fibx
			Block: fib
Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
    Building module "fibx"...
    Generating possibly empty wrappers"
    Maybe empty "fibx-f2pywrappers.f"
        Constructing wrapper function "fib"...
          a = fib(n)
    Wrote C/API module "fibx" to file "./fibxmodule.c"

Then the generated .c file is correct in terms of the signature:

grep "Parameters" fibxmodule.c -A 3 -B 2
#
static char doc_f2py_rout_fibx_fib[] = "\
a = fib(n)\n\nWrapper for ``fib``.\
\n\nParameters\n----------\n"
"n : input int\n"
"\nReturns\n-------\n"
"a : int";

A PR to the existing documentation clarifying this behavior would be most welcome.

It might be more verbose for many cases but I wonder if the F2PY CLI shouldn't warn more, e.g. if there are .pyf and Fortran files a warning would be nice.. (PRs also welcome for this).