ENH: Have f2py handle hybrid F77/F90 codebases #9928
Conversation
The f2py crackfortran parser (which reads declarations from Fortran files) assumed that all files with extension .f or .for is according to Fortran 77 standard. However, some legacy codebases use these extensions also for Fortran 90 code. This applies particularily to projects which started in Fortran 77 but migrated halfway to Fortran 90 when it arrived. This patch lets you set an option in the parser to disregard the file extension when trying to infer the language level of the file. (Such options are (unfortunately) passed to the compiler as global variables)
Allow a command-line option to f2py to specify that we want the internal parser to always regard files to be according to a certain Fortran standard. The command-line option is modeled after a similar option that is used for the compilers (which is also why both single and double dashes are accepted). An alternative to such an option would have been to look for a --std= switch sent to the compiler, but there is (currently) no settings for common flags to both F77 and F90 compilers. Also, using the standard options to the compiler may trigger a more stricter compliance mode, which may not be desirable with old codebases. To use this functionality, pass the (named) parameter f2py_options='-std=f90' to the Extension object in setup.py
The new -std= option must be recognized as an f2py option and passed to the code path that generates the extension module source.
Previous version would only check for module headers if the file extension was .f90 or .f95. However, hybrid F77/F90 codebases may have modules (for allocatable common variables, for instance) and these may not use .f90 as extension. This patch disables the assumption about file name, so that every Fortran source file is scanned for module headers regardless.
We don't want client modules to alter the _f90_mod_ext option flag willy-nilly; by introducing a context manager, it is more easily traced where this flag is set.
If we have declared that all our source is Fortran 90, then all of it should be scanned for module headers, so that they can be compiled first to generate module intermediate files.
Give a short overview of the switch to enable this functionality and an example of a case where you would use it.
|
I know very little about fortran, but if compilers are happy to ignore the extension and automatically do the right thing, then it seems like ideally f2py should do the same? Is there ever any circumstance where your |
|
@njsmith You mean: Why go through all the hoops and make the code more convoluted by having an extra option, if it is possible to instead just remove the extension check and always scan the code as Fortran 90? I agree it's a good question; sorry for not addressing this in the proposal. Honestly, I don't know the full impact of doing that. One snag I can think of is that .f/.for is supposed to be "fixed form", where which column text is in is significant, whereas in .f90 files it may be either "fixed form" or "free form"; it may be that there is code which can be interpreted differently between these two, but I don't have any examples to give. I don't have enough knowledge about the implementation of the 'crackfortran' internal parser, but just that there is a check on extension that toggles an internal flag makes me suspicious that there is someone out there depending on this (as usual xkcd has this covered). |
| a Fortran source file is according to Fortran 90 standard or not and | ||
| thus may contain a module (which must be compiled before other sources). | ||
| """ | ||
| def __init__(self, override = True): |
There was a problem hiding this comment.
PEP8, no spaces in keyword argument defaults, i.e., override=True.
|
Apparently the file extension is not part of the Fortran standard, merely convention. I suppose it could vary between compilers. Looking at ifort and gfortran, it seems that the extension can be overridden, although with different options in the two. So I'm curious as to what the mixed type files look like, are they just fixed-format files that make use of later features? Also, is the problem with f2py parsing of the files, or with compiling the files? |
|
If it is f2py that is the problem, and it sounds like it, anyreason not to always parse as newer fortran, or perhaps wait for an error and then try the newer version, using fixed-format in both cases. |
charris
changed the title
|
Just an observation, but someday we should replace the command line parser in f2py with |
Probably, since the original crackfortran code had
That is of course an option; buffering all print/outmess/errmess until it is determined if there was an error or not, if there was attempt to redo it in Fortran 90 mode, and it that case signal so to the build system. |
There is a problem here; f2py2e.run_compile calls setup, which indirectly calls f2py again, this time run_main which leads to callcrackfortran. I am having trouble to see how to cleanly get information from the latter (that we needed to use Fortran 90 scanning even though the file extension says otherwise) up to run_compile again, so that it can further instruct the module compilation to do modules first. Unless we of course ditch the whole file extension part from misc_utils also, and always scan the Fortran files for module headers and in that case compile them first. What do you think? |
|
Hello @rolk - is this PR something you are still interested in working on? In that case, would you add a test for it? Thanks! |
|
Automatically "doing the right thing" is a bad idea, for an explanation of why we should support |
|
@pearu do you have an opinion on whether this should be closed on can be picked up? |
|
Closing this for now ( @rolk thanks for working on this and for the valuable discussion. |
Some of the older codebases consists of a mix of files written in Fortran 77 and Fortran 90. Probably these started out as Fortran 77 long time ago and then Fortran 90 features were introduced when that became available. However, they are not necessarily careful in using the corresponding file extension for the source files, preferring to keep the existing extension for all files.
This lead into a problem when running them through f2py, as the parser chokes on newer constructs if it has been set in legacy mode. And this mode is determined by the file extension before parsing takes place.
This changeset introduce a new option,
--std=f90, which force the parser into Fortran 90 mode, even if the file extension is.for.for. The name of the option is based on a corresponding option that can be used for the compiler. I though about scanning the compiler options for such a flag, but in most cases, this triggers a strict standard compliance, which these older codebases are not ready for, hence an f2py option.The alternative to such a switch would be to scan through all files, looking for Fortran 90 constructs. I am open to such a solution if that would be deemed preferrable over a new command-line option, but it then incurs a cost of scanning through every file passed to f2py.