Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Google Summer of Code 2009 Proposal -- Fortran integration with Cython

  • Author: Kurt W. Smith
  • Mentor: Dag Sverre Seljebotn and Cython developers

Abstract

Currently, calling a Fortran routine from Cython is difficult and error-prone if one desires, simultaneously: 1) efficient, direct calling of the Fortran code, 2) ease of wrapping the Fortran routines, 3) simple, seamless passing of numpy array objects or python buffer objects [1,2], 4) portability to different Fortran compilers, and 5) Fortran 90 code that uses assumed-shape arrays.

Our project proposes to address the above issues by improving the f2py-Cython integration. There are 2 aspects to the project: enhancing f2py (G3 F2PY specifically) to generate updated Fortran wrappers callable from C, and enhancing Cython to enable passing of python buffers using the buffer syntax. The Fortran wrappers will use the ISO C bindings that are a part of the Fortran 2003 standard. Provided the compiler supports the bindings (many do already, or will in the next version), they can be used to wrap any version of the language. These wrappers will make use of utility routines that can convert a numpy array or python buffer to an array passable to Fortran.

[1] http://docs.python.org/c-api/buffer.html

[2] wiki.cython.org/enhancements/buffer

The f2py Fortran-to-python interface generator serves an essential function, simplifying the interfacing of Fortran and python libraries. It has become, as of late, a bit long in the tooth, and could use some updating from its current distributed version (version 2). It is currently impossible for a programmer to pass an array from python to a Fortran routine without modifying the Fortran routine's signature to explicitly pass the array's dimensions. f2py wraps the compiled Fortran code inside a Fortran object, making all calls to the underlying routine pass through Python API calls; it would be nice to call the routine 'directly' from Cython, much as one would do when calling a Fortran routine from C code. Each Fortran compiler has a different 'name-mangling' scheme, but the ISO C bindings standardize the Fortran-C interoperability and allow portable wrapping of Fortran code, which would obviously be beneficial to support. Cython excels at producing optimized python extension modules that are Python 2 and Python 3 compliant, so leveraging Cython to make the extension module and f2py to make the Fortran wrappers is the best of both worlds.

Example

Suppose one has the following Fortran 90 code using assumed-shape arrays and a derived type:

! file original.f90
module ex
    implicit none

    ! contrived user derived type
    type person
        integer age
        real height
    end type person

    contains

    subroutine mod_sub(per1, age, height, int_arr)
        implicit none
        ! user derived type passed to subroutine
        type(person), intent(out) :: per1
        integer, intent(in) :: age
        real, intent(in) :: height
        ! 2-dimensional assumed-shape array.
        integer, intent(inout), dimension(:,:) :: int_arr

        ! modify the person passed in.
        per1%age = age
        per1%height = height

        ! print out the assumed-shape array using high-level indexing.
        print *, int_arr(:,1)

        ! modify the array with fortran-style indexing/slicing.
        int_arr(::2,1) = -1000

    end subroutine mod_sub
end module ex

One would like to call the mod_sub subroutine, from within Cython with a minimum of hassle and maximally efficient.

The desired interface from Cython would be:

! file: wrapped_original.pxd

cdef extern:
    ctypedef struct person:
        int age
        float height

    void mod_sub(person*, int*, float*, object[int, ndim=2] arr)

Notice that the int_arr in the original Fortran file is declared as a buffer object in Cython. That declaration could be done using the alternative buffer syntax (yet to be accepted):

void mod_sub(person*, int*, float*, int[:,:,mode='contiguous'] arr)

The Cython code would pass a struct containing all relevant data pertaining to the buffer to the wrapped Fortran routine, which would convert the struct to an appropriate Fortran array, passing this array on to the original mod_sub subroutine.

A working wrapper (using gcc/gfortran version 4.3.4) in Fortran would be something like the following:

module wrap_ex
      use ex
      use iso_c_binding
      implicit none

      ! example of C-interoperable wrapping of derived type
      type, bind(c) :: wrap_person_type
        integer(c_int) :: age
        real(c_float) :: height
      end type wrap_person_type

      ! example wrapping of a 2-dimensional array/buffer.
      type, bind(c) :: int2Darr_wrapper
        type(c_ptr) :: buf
        integer(c_int) :: shape1, shape2
      end type int2Darr_wrapper

      contains

      ! subroutine will be callable from C with the name "mod_sub"
      subroutine wrap_mod_sub(per1, age, height, int_arr) &
          bind(C, name="mod_sub")
        implicit none
        type(wrap_person_type), intent(out) :: per1
        integer(c_int), intent(in) :: age
        real(c_float), intent(in) :: height
        type(int2Darr_wrapper), intent(inout) :: int_arr

        type(person) :: per2
        integer(c_int), pointer, dimension(:,:) :: c_array

        ! associates the c_array assumed-shape array with the int_arr%buf
        call c_f_pointer(int_arr%buf, c_array, (/int_arr%shape1,&
            int_arr%shape2/))

        ! calls the wrapped subroutine
        call mod_sub(per2, age, height, c_array)

        ! convert from the native person type to the wrapped person type
        per1%age = per2%age
        per1%height = per2%height

      end subroutine wrap_mod_sub
end module

This wrapper code is callable from C as follows:

#include <stdio.h>
#include <stdlib.h>

#define NN 3

/* these typedef'd structs correspond to the derived Fortran types. */
typedef struct {
    int age;
    float height;
} person;

typedef struct {
    int *buf;
    int shape1, shape2;
} int2Darr_wrapper;

/* the signature for the wrapped Fortran code */
void mod_sub(person*, int*, float*, int2Darr_wrapper*);

int main(int argc, char **argv) {

    /* create a person and a wrapped array */
    person p;
    int2Darr_wrapper arr_wrapper;
    int age = 20;
    float height = 30.0;
    int i;

    /* malloc memory for the arr_wrapper.buf buffer */
    if(NULL == (arr_wrapper.buf = (int *)malloc(sizeof(int)*NN*NN))) {
        return 1;
    } else {
        arr_wrapper.shape1 = arr_wrapper.shape2 = NN;
    }

    /* initialize the buffer */
    for(i=0; i<NN*NN; i++) {
        arr_wrapper.buf[i] = i;
    }

    /* call the wrapped Fortran subroutine */
    mod_sub(&p, &age, &height, &arr_wrapper);

    /* the Fortran call correctly modifies the person's fields */
    /* result: "person: 20 30.00000" */
    printf("person: %d %f\n", p.age, p.height);


    /* the Fortran call also correctly handles a 2-dimensional array malloc'd in C */
    for(i=0; i<NN*NN; i++) {
        printf("%d ", arr_wrapper.buf[i]);
    }

    free(arr_wrapper.buf);

    return 0;
}

This example exposes many of the considerations when wrapping Fortran code. The f2py side of the project will make use of the fparser module from the G3 F2PY project (XXX link here) and add the ability to wrap Fortran code using the ISO C bindings seen above. The f2py utility will generate the Fortran wrapper, the pxd file and possibly a C header file for the Fortran code. The Cython side will take the pxd file with the buffer declarations and compile them into appropriate C wrappers that pass structs to the underlying Fortran wrappers.

Schedule

(with input from Dag Sverre Seljebotn)

Start of program

Thoroughly understand the ISO C bindings in the Fortran 2003 standard; compose test cases exercising all datatypes being passed from C to Fortran. (apparently making the Fortran character type play nicely with C chars is fraught with peril ;-) ).

Define a Cython language feature for passing numpy arrays / python buffers from Cython to an external subroutine or function; this will require input and consensus from Cython developers.

Map out necessary changes / updates to the fparser module from G3 F2PY. Add support for the iso_c_binding intrinsic module.

Midterm evaluation

f2py -- enhanced with new backend that is able to wrap basic testcase subroutines and functions. Supports ints, floats, doubles, and complex arguments with 'in', 'inout', 'out' intent specifiers. Not required to pass tests for more complex testcases.

Cython -- implement basic support for CEP 402 against a mock testcase library written in C (or manually in Fortran), to get a feel for what is needed for array support. (This is not a big job, once the spec is in place 75% of the work is done.)

Final evaluation

f2py -- correctly parses supported Fortran 77/90/95 function & subroutine interfaces with Fortran 90/95 assumed-shape arrays taking priority over full Fortran 77 support

f2py & Cython -- Add support for passing of assumed shape arrays of primitive types from end to end.

f2py & Cython -- Add support for structs/derived types, end to end (though without arrays)

Bonus goals: - String support - Support for "overloading" (those Fortran interface specs which selects different functions depending on argument types...) - Structs in arrays (if necesarry by copying)

Something went wrong with that request. Please try again.