Memory mapped features

[PierUgit]

Motivation

Provide some memory mapped file features to Fortran. These features are indeed accessible througt the C interoperability, but the idea here is to encapsulate them to get an easy to use interface, with additional runtime checks. Two meain usages:

• allocating arrays that are potentially bigger the RAM+swap size, and are which are backed in a temporary file (anonymous mapping)
• opening existing files or creating new files, and mapping them to arrays

Prior Art

Searching "memory mapped files Fortran" in Google returns results that all use the C interoperability of C/posix features, or Windows specific routines with the Intel compiler. A stdlib implementation should have the OS dependent code under a unique interface.

Additional Information

This was originally discussed on the Fortran Discourse, where the inclusion in stdlib has been suggested.

I have prepared a small demonstration code to show what it could look like (C/posix stuff only): https://github.com/PierUgit/fmmap/tree/main

[jvdp1]

Thank you @PierUgit for this suggestion.
The Discourse discussion is interesting and it seems that there is interest for such an implementation.
My main fear is that such an implementation will be OS-dependent, which may result in issues with portability. However, it should not stop such a projet. I think it would be a nice addition to stdlib.

[PierUgit]

It's definitely OS-dependent, and even worse maybe compiler-dependent. I assume that on Posix systems all compilers will provide the mmap features, but on Windows I don't know. The Intel compiler directly provides some routines in the kernel32 module, but I'm not sure how it goes with other compilers... I do not develop at all on Windows, so I'm a bit lost at that point.

Generally speaking, how are managed the OS dependent features in stdlib? Are there any in the existing code base? Are there available macros?

[jvdp1]

I do not develop at all on Windows, so I'm a bit lost at that point.

Me neither...

Generally speaking, how are managed the OS dependent features in stdlib? Are there any in the existing code base? Are there available macros?

Currently only stdlib_system is compiled depending on the OS (with a CPP macro _WIN32). But it gives some issues (e.g., #738).
fpm is probably more advanced regarding OS-dependencies.

[PierUgit]

I will try porting the demo code to Windows and see... I have to set up a dev environment on my multimedia mini-PC first (the one that sits below the TV, as this is the only Windows machine around :D)

[arjenmarkus]

I do develop on Windows and in a distant and hardly remembered past I dealt with memory-mapped files on Windows. I can dust off the code I had, if that helps or I can try and help out more directly :). I do remember that there were unexpected differences in behaviour between the two platforms. Op do 8 feb 2024 om 22:35 schreef PierUgit ***@***.***>:

…

I will try porting the demo code to Windows and see... I have to set up a dev environment on my multimedia mini-PC first (the one that sits below the TV, as this is the only Windows machine around :D) — Reply to this email directly, view it on GitHub <#765 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6YR4XOZUGCTDZFIF3VI3YSVAJHAVCNFSM6AAAAABCR53IP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZUHE3DSNZRG4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[PierUgit]

An example code (even with some dust :)) could definitely help. And you're also welcome if you want to help.

[arjenmarkus]

I had a quick look at the stuff - it has been many years since I last did. There is no particular documentation, I am afraid, so some code diving is the only way. The code also precedes Fortran 2003, so it is a bit messy when it comes to the Fortran-C interaction. Please find the zip file in the attachment (hoping that it is accepted) -- oops, that is not the case, well, I had to rename the two batch files, then it worked. Op vr 9 feb 2024 om 19:49 schreef PierUgit ***@***.***>:

…

An example code (even with some dust :)) could definitely help. And you're also welcome if you want to help. — Reply to this email directly, view it on GitHub <#765 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6YR5CVVO3CH5J354MX3LYSZVR7AVCNFSM6AAAAABCR53IP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZWGQZTOMRUGY> . You are receiving this because you commented.Message ID: ***@***.***>

[PierUgit]

I could install the MSYS2 environment on my office laptop, and start porting the code to Windows. I have a rough version of the anonymous mapping and it looks OK, so at the end everything should (hopefully) work.

Please find the zip file in the attachment (hoping that it is accepted) -- oops, that is not the case, well, I had to rename the two batch files, then it worked.

Actually there's no attachment in the GitHub interface, where I'm reading the posts. I am sending you a separate and personal email : could you send again the files in the answer?

[PierUgit]

The code now runs on Windows :)