Linux version is slow with files > 4GB in size

[crioux]

Not sure why yet, need more data, but the difference between a 1GB file and a 10GB file is huge on my test machine. Need different test machines and more data to determine if this is just that one box or what configuration this happens in. Doesn't affect Windows or OSX.

[crioux]

This has also only been reproduced on x86_64 so far. Haven't tried on 32-bit Linux

[philsmd]

Just a note: I can confirm this on a 64bit ubuntu system. The interesting thing is that it doesn't seem to happen (for me) when I use the same file (same system too) copied on a SSD. Even if I use a 150+ GB file on a SSD, tlc is still faster.

On the HDD instead (western digital 3TB), tlc starts to be almost always be slower than wc -l when the file is above 8GB and if it is over 10GB wc -l is always faster.

Could this somehow have to do with different kind/amount of disk caching (between ssd and HDD or is there a problem when the disk cache is not large enough?) or some (default) mount options etc? There must be an important difference between SSD and HDD (besides the speed) that makes the difference I see between the 2 types of disks.

(it doesn't seem to be connected at all to ram use or CPU load)
(the ssd and HDD are both connected to SATA ports, they do NOT have different connections like M.2 etc)