Recursive directory traversal performance

[harendra-kumar]

The recursive directory listing APIs are inefficient as they list each directory and then use the produced paths to try to list the children. Instead we can use the inodes directly, also we can list the attributes at the same time to filter out non-dir children.

Currently the traversal using the ListDir example in streamly-examples is significantly slower than ls on Linux:

$ time ListDir > /dev/null

real    0m0.874s
user    0m0.657s
sys     0m0.228s

$ time ls -rRa . > /dev/null

real    0m0.201s
user    0m0.103s
sys     0m0.096s

$ time find . > /dev/null

real    0m0.229s
user    0m0.095s
sys     0m0.132s

Also need to see if the stream based recursive traversal is less efficient. Also, the use of String instead of OsPath might make a difference as we are heavily dealing with paths here.