Use native file I/O in local and overlay drive

[weirddan455]

Description

This PR replaces the use of C library I/O (fopen() and friends) with native file I/O. Windows uses Win32 API calls and everyone else uses POSIX.

I was also able to remove some hacky ftell() + fseek() nonsense that should not be needed since there's no library buffering. The OS itself will still buffer but that should be fine. All we care about is that a write followed by a read will return the new data.

I have some more work on the file I/O planned to better handle timestamps (which will depend on native file I/O) but I thought it would be best to put that in a separate PR and get this in first.

Manual testing

Tested on Windows (both MSYS2/GCC and MSVC/Clang) and Linux. Crystal Caves and Mortal Kombat saves work which had been problematic on Windows in the past.

Other games with save state tested: Secret of Monkey Island 2, Quake, Doom, Space Quest 1

Checklist

Please tick the items as you have addressed them. Don't remove items; leave the ones that are not applicable unchecked.

I have:

• followed the project's contributing guidelines and code of conduct.
• performed a self-review of my code.
• commented on the particularly hard-to-understand areas of my code.
• split my work into well-defined, bisectable commits, and I named my commits well.
• applied the appropriate labels (bug, enhancement, refactoring, documentation, etc.)
• checked that all my commits can be built.
• confirmed that my code does not cause performance regressions (e.g., by running the Quake benchmark).
• added unit tests where applicable to prove the correctness of my code and to avoid future regressions.
• made corresponding changes to the documentation or the website according to the documentation guidelines.
• locally verified my website or documentation changes.

[shermp]

Is there a case for making these file i/o wrappers more generic so that other parts of the code could use them later. I'm talking about seek, read, write etc.

Other code that could use this would be the disk image code, in which case large file support would definitely be appreciated.

[weirddan455]

Is there a case for making these file i/o wrappers more generic so that other parts of the code could use them later. I'm talking about seek, read, write etc.

Other code that could use this would be the disk image code, in which case large file support would definitely be appreciated.

They're already fairly generic and should be fine to use elsewhere. Some of the calls (especially create and open) may need additional flags depending on the use-case but I think we can cross that bridge when we come to it.

Maybe I should rename the functions to create_native_file read_native_file, etc. I'll bump the size types up to int64_t as @johnnovak suggested as well.

[FeralChild64]

Looks OK. I have no remarks.

So far tried some Windows 3.1 software, Colonization game. and MkS_Vir virus scanner, didn't find any problems/

[weirddan455]

Thanks for testing @FeralChild64

I pushed changes addressing the comments above. Mostly re-naming things and fixing formatting. I also changed to int64_t everywhere in these native I/O functions. The Win32 code needed some changes to accommodate 64 bit. SetFilePointer (used in seek_native_file) does a weird thing where it splits into two 32-bit words and I added a loop to ReadFile() and WriteFile() very similar to what I did with POSIX except the values get clamped to 32-bit max.

They should be suitable for use where-ever else in the code they might be needed. I renamed them as I said above to open_native_file and such (removing the local_drive prefix I had before). I also made the fat attributes on create_native_file a std::optional as that one doesn't make sense if we want to create a normal file on the host.

[johnnovak]

This is solid step in the right direction @weirddan455 🎉 Looks very good now, left one last comment about testing Blackthorne if you have the capacity. Great work!

[weirddan455]

I'll let you take a look at my last commit if you like @johnnovak addressing the Blackthrone issue.

[weirddan455]

Added a LOG_WARNING if the file seek fails. I wasn't quite sure what to do in this case because apparently the real DOS interrupt never fails. The most likely case is a game trying to seek before the beginning of the file so I'm just setting the file position to 0.

Blackthorn never hits this case. Using unsigned integer lets it succeed. I wanted this warning to be visible to end users rather than a debug log because it seems to be rare and I want users to put this in a bug report if my assumption proves to be incorrect.

[johnnovak]

Outstanding work @weirddan455, I'm glad you've managed to remove the Blackthorne hack as well 🚀 I agree with your last note about logging a warning, that's the best we can do.