optimisation done by ksh93 file reading builtins break functionality and cause memory leak #15
Description
(tested with ksh93u from package on Debian amd64)
$ seq 5 > a; ksh93 -c 'read a; echo test > a; read b; echo "$a $b"' < a
1 2
The second "read" could not possibly have read "2" because we have replaced the content of the "a" file with "test\n". What happens (from examining strace output) is that on the first "read", ksh93 has read up to 64kB worth of data, put the first line in $a, lseek()ed back to the end of that first line and remembered the data that was after that. Upon the second read, it optimizes out the read() system call and uses the remembered data instead.
It does check that the position of the stdin cursor within the file has not changed to assert that the optimisation is valid, but here, stdin has not moved, but the optimisation is not valid for a different reason: the content of the file has changed.
That optimisation will probably not gain you much in most cases on most systems because the OS will keep the read data in cache already (and knows better how and when to invalidate the cache), so that ksh93 behaviour could be seen as wasting resources by keeping another copy of that data in memory.
It also seems like there's a memory leak in that that "remembered" data seems never to be freed even after a file descriptor has been reused on a different file:
$ ksh93 -c 'ps -o rss,comm -p "$$"; for f in /usr/*/*; do read -n1 a < $f; done; ps -o rss,comm -p "$$"; :'
RSS COMMAND
1472 ksh93
RSS COMMAND
134860 ksh93
It affects "read" and other builtin utilities that read data (which seem to share that same "remembered" data). I've verified it with cat and head.