Improve "directory cannot be listed" message to reflect underlying cause

[clintongormley]

In #7954 (comment), @drewr saw the following error message:

[2014-10-24 21:28:48,949][WARN ][index.engine.internal    ] [ops27-data04-A] [foo][31] failed engine [merge exception]
org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: directory '/d/es/data/ops27-data04-A/org.elasticsearch.test.ops27.data/nodes/0/indices/foo/31/index' exists and is a directory, but cannot be listed: list() returned null
        at org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)

Apparently, the underlying cause for this exception is not having enough file handles, but there is nothing in the message to reflect this.

Is too few file handles the only cause of the exception, or are there others? This is an exception thrown by Java. Is it possible to catch this particular exception and return a better more explanatory message?

[clintongormley]

@mikemccand not sure if this is still an issue but if so, would it be possible to improve this exception or is the real reason (not enough file handles) just not available?

[mikemccand]

@clintongormley I think in ES 2.0 this is already improved, because Lucene switched from Files.list in 4.10.x to Files.newDirectoryStream in 5.x, and the latter does a better job propagating errors from the OS up to Java, I think.