Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
OSX fcntl(fd, F_NOCACHE, 1) not equivalent to O_DIRECT on Linux #48
This is probably already well known, but it bit me while planning to do some comparative benchmarks between Linux and OSX. I started with OSX and
So I looked up the ways to do direct I/O on OSX. On stack overflow and the Apple mailing lists,
I also commented on stack overflow: clear buffer cache on OSX with my observations. I'll copy it here as well:
I'm not advocating running
Another idea I had was to forcibly re-write the file each time for reading, while having the file open with
An interesting discussion involving an Apple dev on the Apple mailing list: http://lists.apple.com/archives/filesystem-dev/2007/Sep/msg00010.html. Specifically the part mentioning that files opened with F_NOCACHE will still uses already loaded pages if they're present.
EDIT: Some train of thought rambling: but perhaps mmap + MAP_NOCACHE on OSX is a possible avenue (+ telling uses to use that for direct IO testing). The man page doesn't give me a lot of hope though, as it appears that there's not guarantess and the OS might keep things in memory if it feels like it anyway. A small reference mentioning it: http://www.qtcentre.org/threads/24733-High-performance-large-file-reading-on-OSX
Thanks for doing a thorough investigation! And I agree with you, we should add some mention of having to run purge to evict any previous cache. Or at least be aware that caching effects may exist.
But it really is a horrible interface, seems strange that OSX doesn't give you finer control of what is cached and how.
No problem at all. I'm glad I learned something and found a workaround in short order. At least the difference was large enough (1200MB/s vs 22MB/s i pretty noticeable).
I agree. I've looked for a bit more but could find nothing at all. It seems
added a commit
Aug 27, 2017
Just for the record: I checked whether