Redo #12707: Do not rely on isAlive() status of MemorySegment#Scope and make sure IndexInput#close() does not throw IllegalStateException and waits instead

[uschindler]

Unfortunately the solution in #12707 was not working well with concurrency. The is alive status of MemorySegment.Scope may be stale. In that case the IllegalStateException was catched, but the confirmation using isAlive() did not work, so the original IllegalStateException was rethrown instead of the required AlreadyClosedException.

This PR reverts #12707 and falls back to the naive approach by checking exception message for "closed". I am in contact with @mcimadamore to find a better solution to confirm the closed status without relying on Exception message.

[uschindler]

The bug in JDK is here: https://bugs.openjdk.org/browse/JDK-8319756

[uschindler]

After some discussion with @mcimadamore we figured out that there are more problems, so we need to rely on the exception message.

The following problem can occur and possibly happens here: When you close the main IndexInput from another thread this may fail with an IllegalStateException. While trying to do this also the clone may get the above ISE and should convert it to the AlreadyClosedException. But when the main thread got the ISE the close operation did not work and the previous state is restored. And that's waht we see here. The main thread in the test tries to close but gets an exception so the close did not succeed.

The main problem that the clsoed state is a 3state setting: OPEN, CLOSING, CLOSED. When the scope is in CLOSING state it may go back to OPEN. So after the IllegalStateException the status may go back to "OPEN" and this is why the Exception is not converted.

I will add another improvement to the test to "retry" closing on ISE. We should maybe also add a loop in the MemorySegmentIndexInput that it tries to close multiple times while yielding the threads.

[uschindler]

I committed another change to make the sequence of IndexInput#close() first try to close the segment and then set everything to null. In case if ISE, the IndexInput is not closed.

I will next check how to re-try closing (which may take very long time under high load on the other threads).

[uschindler]

I fixed the close() method to no longer throw IllegalStateException as this would violate the contract. When we close only IOException is allowed. As half-open index inputs are worse, we just retry to close the scope/session in a spinloop.

I checked this with Maurizio, until the bugs in JVM are fixed, this is the best thing to do.

Maurizio openend another issue in the JDK, describing our problem: https://bugs.openjdk.org/browse/JDK-8319782 (duplicate of https://bugs.openjdk.org/browse/JDK-8310644).

[uschindler]

I let TestMmapDirectory.testAceWithThreads run with gradlew :lucene:core:beast with many iterations and high multiplier: JDK 19, 20, 21 showed no problems.