kwsmith soc09
- Author: Kurt W. Smith
- Mentor: Dag Sverre Seljebotn and Cython developers
Currently, calling a Fortran routine from Cython is difficult and error-prone if one desires, simultaneously: 1) efficient, direct calling of the Fortran code, 2) ease of wrapping the Fortran routines, 3) simple, seamless passing of numpy array objects or python buffer objects [1,2], 4) portability to different Fortran compilers, and 5) Fortran 90 code that uses assumed-shape arrays.
Our project proposes to address the above issues by improving the f2py-Cython integration. There are 2 aspects to the project: enhancing f2py (G3 F2PY specifically) to generate updated Fortran wrappers callable from C, and enhancing Cython to enable passing of python buffers using the buffer syntax. The Fortran wrappers will use the ISO C bindings that are a part of the Fortran 2003 standard. Provided the compiler supports the bindings (many do already, or will in the next version), they can be used to wrap any version of the language. These wrappers will make use of utility routines that can convert a numpy array or python buffer to an array passable to Fortran.
[1] http://docs.python.org/c-api/buffer.html
[2] wiki.cython.org/enhancements/buffer
The f2py Fortran-to-python interface generator serves an essential function, simplifying the interfacing of Fortran and python libraries. It has become, as of late, a bit long in the tooth, and could use some updating from its current distributed version (version 2). It is currently impossible for a programmer to pass an array from python to a Fortran routine without modifying the Fortran routine's signature to explicitly pass the array's dimensions. f2py wraps the compiled Fortran code inside a Fortran object, making all calls to the underlying routine pass through Python API calls; it would be nice to call the routine 'directly' from Cython, much as one would do when calling a Fortran routine from C code. Each Fortran compiler has a different 'name-mangling' scheme, but the ISO C bindings standardize the Fortran-C interoperability and allow portable wrapping of Fortran code, which would obviously be beneficial to support. Cython excels at producing optimized python extension modules that are Python 2 and Python 3 compliant, so leveraging Cython to make the extension module and f2py to make the Fortran wrappers is the best of both worlds.
Suppose one has the following Fortran 90 code using assumed-shape arrays and a derived type:
! file original.f90
module ex
implicit none
! contrived user derived type
type person
integer age
real height
end type person
contains
subroutine mod_sub(per1, age, height, int_arr)
implicit none
! user derived type passed to subroutine
type(person), intent(out) :: per1
integer, intent(in) :: age
real, intent(in) :: height
! 2-dimensional assumed-shape array.
integer, intent(inout), dimension(:,:) :: int_arr
! modify the person passed in.
per1%age = age
per1%height = height
! print out the assumed-shape array using high-level indexing.
print *, int_arr(:,1)
! modify the array with fortran-style indexing/slicing.
int_arr(::2,1) = -1000
end subroutine mod_sub
end module exOne would like to call the mod_sub subroutine, from within Cython with a minimum of hassle and maximally efficient.
The desired interface from Cython would be:
! file: wrapped_original.pxd
cdef extern:
ctypedef struct person:
int age
float height
void mod_sub(person*, int*, float*, object[int, ndim=2] arr)Notice that the int_arr in the original Fortran file is declared as a buffer object in Cython. That declaration could be done using the alternative buffer syntax (yet to be accepted):
void mod_sub(person*, int*, float*, int[:,:,mode='contiguous'] arr)The Cython code would pass a struct containing all relevant data pertaining to the buffer to the wrapped Fortran routine, which would convert the struct to an appropriate Fortran array, passing this array on to the original mod_sub subroutine.
A working wrapper (using gcc/gfortran version 4.3.4) in Fortran would be something like the following:
module wrap_ex
use ex
use iso_c_binding
implicit none
! example of C-interoperable wrapping of derived type
type, bind(c) :: wrap_person_type
integer(c_int) :: age
real(c_float) :: height
end type wrap_person_type
! example wrapping of a 2-dimensional array/buffer.
type, bind(c) :: int2Darr_wrapper
type(c_ptr) :: buf
integer(c_int) :: shape1, shape2
end type int2Darr_wrapper
contains
! subroutine will be callable from C with the name "mod_sub"
subroutine wrap_mod_sub(per1, age, height, int_arr) &
bind(C, name="mod_sub")
implicit none
type(wrap_person_type), intent(out) :: per1
integer(c_int), intent(in) :: age
real(c_float), intent(in) :: height
type(int2Darr_wrapper), intent(inout) :: int_arr
type(person) :: per2
integer(c_int), pointer, dimension(:,:) :: c_array
! associates the c_array assumed-shape array with the int_arr%buf
call c_f_pointer(int_arr%buf, c_array, (/int_arr%shape1,&
int_arr%shape2/))
! calls the wrapped subroutine
call mod_sub(per2, age, height, c_array)
! convert from the native person type to the wrapped person type
per1%age = per2%age
per1%height = per2%height
end subroutine wrap_mod_sub
end moduleThis wrapper code is callable from C as follows:
#include <stdio.h>
#include <stdlib.h>
#define NN 3
/* these typedef'd structs correspond to the derived Fortran types. */
typedef struct {
int age;
float height;
} person;
typedef struct {
int *buf;
int shape1, shape2;
} int2Darr_wrapper;
/* the signature for the wrapped Fortran code */
void mod_sub(person*, int*, float*, int2Darr_wrapper*);
int main(int argc, char **argv) {
/* create a person and a wrapped array */
person p;
int2Darr_wrapper arr_wrapper;
int age = 20;
float height = 30.0;
int i;
/* malloc memory for the arr_wrapper.buf buffer */
if(NULL == (arr_wrapper.buf = (int *)malloc(sizeof(int)*NN*NN))) {
return 1;
} else {
arr_wrapper.shape1 = arr_wrapper.shape2 = NN;
}
/* initialize the buffer */
for(i=0; i<NN*NN; i++) {
arr_wrapper.buf[i] = i;
}
/* call the wrapped Fortran subroutine */
mod_sub(&p, &age, &height, &arr_wrapper);
/* the Fortran call correctly modifies the person's fields */
/* result: "person: 20 30.00000" */
printf("person: %d %f\n", p.age, p.height);
/* the Fortran call also correctly handles a 2-dimensional array malloc'd in C */
for(i=0; i<NN*NN; i++) {
printf("%d ", arr_wrapper.buf[i]);
}
free(arr_wrapper.buf);
return 0;
}This example exposes many of the considerations when wrapping Fortran code. The f2py side of the project will make use of the fparser module from the G3 F2PY project (XXX link here) and add the ability to wrap Fortran code using the ISO C bindings seen above. The f2py utility will generate the Fortran wrapper, the pxd file and possibly a C header file for the Fortran code. The Cython side will take the pxd file with the buffer declarations and compile them into appropriate C wrappers that pass structs to the underlying Fortran wrappers.
(with input from Dag Sverre Seljebotn)
Thoroughly understand the ISO C bindings in the Fortran 2003 standard; compose test cases exercising all datatypes being passed from C to Fortran. (apparently making the Fortran character type play nicely with C chars is fraught with peril ;-) ).
Define a Cython language feature for passing numpy arrays / python buffers from Cython to an external subroutine or function; this will require input and consensus from Cython developers.
Map out necessary changes / updates to the fparser module from G3 F2PY. Add support for the iso_c_binding intrinsic module.
f2py -- enhanced with new backend that is able to wrap basic testcase subroutines and functions. Supports ints, floats, doubles, and complex arguments with 'in', 'inout', 'out' intent specifiers. Not required to pass tests for more complex testcases.
Cython -- implement basic support for CEP 402 against a mock testcase library written in C (or manually in Fortran), to get a feel for what is needed for array support. (This is not a big job, once the spec is in place 75% of the work is done.)
f2py -- correctly parses supported Fortran 77/90/95 function & subroutine interfaces with Fortran 90/95 assumed-shape arrays taking priority over full Fortran 77 support
f2py & Cython -- Add support for passing of assumed shape arrays of primitive types from end to end.
f2py & Cython -- Add support for structs/derived types, end to end (though without arrays)
Bonus goals: - String support - Support for "overloading" (those Fortran interface specs which selects different functions depending on argument types...) - Structs in arrays (if necesarry by copying)