File system operations

[MarDiehl]

While some functionality for file system related operations exist in Fortran, some rather relevant operations are not standardized.
For example, figuring out whether a path is a directory:
https://stackoverflow.com/questions/9522933

A function like "isDirectory" can be based on the corresponding C-functionality. Would that be an appropriate solution? I would certainly require a bunch of #ifdefs in the C-side of the implementation

[certik]

This is related to #14. I think we agreed that file system operations are in the scope. The naming convention should probably be is_directory or is_dir, and we should also look at how Python, Julia and Matlab name such functions. We are always trying to be consistent with other languages where it makes sense.

The main goal of stdlib is to figure out and document (in a spec) the API. The underlying implementation is secondary --- stdlib will provide a reference implementation and then compiler vendors are free to provide their own, different or more optimized implementation if they want. The only requirement is that it must run on all platforms, but probably calling into C if we have to would do it.

[milancurcic]

#22 for POSIX systems is also related.

[MarDiehl]

I have thought a little bit about a possible contribution from my side, and I propose to add general path related functionality. Since the other language I am frequently using is python, I got my inspiration from

1. os.path
2. pathlib

In python, the object oriented approach of pathlib is often favored over os.path. However, unless I overlooked something, I believe that the os.path approach is better suited for Fortran because one can not chain function calls.

• Python: p = path.absolute(),is_file()
• Fortran: p = path_is_file(path_absolute(Str))

With Fortran variable length strings (allocatable), most operations are easily performed. However, there is one exception:
os.path.commonpath(paths) takes a list of strings, in Fortran this would be an array of strings/characters and all of them would need to have the same length. Alternatively, one can use an interface for a series of fpp generated functions with signatures of type path_commonpath(path1,path2), path_commonpath(path1,path2,path3), ...

While most functionality will be implemented in pure Fortran, certain operations require file system functions from C.

There is one more thing I need to mention: If Windows support is needed, someone else has to provide it. I've never compiled a Fortran program on windows and the path operations differ significantly. pathlib has essentially two implementations and I assume we are in the same situation. If someone volunteers, I would prefer that we work in parallel on both implementation. My time budget is about 4h/week and I would hope to finish the whole implementation in 3 months.

[epagone]

With Fortran variable length strings (allocatable), most operations are easily performed. However, there is one exception:
os.path.commonpath(paths) takes a list of strings, in Fortran this would be an array of strings/characters and all of them would need to have the same length. Alternatively, one can use an interface for a series of fpp generated functions with signatures of type path_commonpath(path1,path2), path_commonpath(path1,path2,path3), ...

Or it can be solved by an implementation of the iso_varying_string module or advanced libraries of string handling routines as summarised in this other issue.

[jvdp1]

However, there is one exception:
os.path.commonpath(paths) takes a list of strings, in Fortran this would be an array of strings/characters and all of them would need to have the same length. Alternatively, one can use an interface for a series of fpp generated functions with signatures of type path_commonpath(path1,path2), path_commonpath(path1,path2,path3), ...

fypp could be used for that too.
However, implementing and using iso_varying_string module would be the best option IMO.

[urbanjost]

FYI:

I have some related modules (M_path, M_io, and M_system) in the GPF
(General Purpose Fortran) site on github if you are looking for ideas
on how the API might look from a Fortran perspective.

https://github.com/urbanjost

Instead of the entire GPF, self-contained subsets of two of the modules
are available:

https://github.com/urbanjost/M_io
https://github.com/urbanjost/M_system

The M_path module description (actually all the GPF routines) can be
found in the manpage index:

GPF manpages

I do not have a stand-alone M_path.f90. It is currently only in the GPF
collection, as it uses a number of string routines from M_strings.f90.

[MarDiehl]

@urbanjost Nice code, thanks for the hint.

[urbanjost]

Thanks. No problem. Some is better than others. Seeded it with a lot of code I had around with the hopes of getting a development community started to expand it and clean it up but it did not catch on as I had hoped. stdlib(3f) seems to have far more momentum behind it. If you find anything useful in it feel free to use it for stdlib(3f).

[ivan-pi]

The Oracle Fortran Library seems to have covered some file system operations: https://docs.oracle.com/cd/E19957-01/805-4942/index.html

Based upon the function names, it looks like it is actually implemented in C. Perhaps it can serve as reference.

Edit: The Absoft Compiler also has similar compatibility libraries - https://www.absoft.com/wp-content/uploads/2015/08/Support-Libraries.pdf

Edit2: Compaq Fortran also had it's own library of C-like functions - http://h30266.www3.hpe.com/odl/unix/progtool/cf95au56/dfumroutines.htm#overview_lib_rout

[ivan-pi]

Perhaps the Fortyxima project (https://bitbucket.org/aradi/fortyxima/src/develop/fortyxima/filesys/) from @aradi could serve as a starting point?

[aradi]

If there is interest, I am happy to clean it up a bit, so that it meets the stdlib coding standards. 😉

[certik]

@urbanjost don't feel bad, a lot of us tried to do the same. Thanks for the pointer, I added it to #1, thanks for sharing the link. As you can see there, we list 10 such libraries that people did (mine is there too), and we all tried to get a community started around it. It's extremely hard. But I think we finally succeeded this time with stdlib and with fortran-lang.org. I should write a blog post about this --- Fortran is far from being saved, but just the fact that we managed to get the community together is the first necessary step, and it looked impossible to me just a year ago. And yet I think we succeeded at this first step at this point.

@ivan-pi thanks for the pointers, @aradi I think there will be interested, let's discuss the API.

[aradi]

Sure, I opened a separate issue for this (#220)

[MarDiehl]

I have a first prototype of a library that has functions from pythons os and os.path: https://github.com/MarDiehl/stdlib_os
Pure Fortran where possible, but most file system related operations rely on C routines.
Works on linux with GNU and Intel compilers.
Exceptions in python translate into error stop. Currently without a message, but that can be changed.

[jvdp1]

I have a first prototype of a library that has functions from pythons os and os.path: https://github.com/MarDiehl/stdlib_os
Pure Fortran where possible, but most file system related operations rely on C routines.
Works on linux with GNU and Intel compilers.
Exceptions in python translate into error stop. Currently without a message, but that can be changed.

@MarDiehl I looked to the code. There are already quite a bunch of procedures for Linux OS. Nicely done. Would it be an idea to submit a PR to discuss further the API?