Panicking in spawned Rayon tasks will abort the process by default.

[adamreichold]

C.f. https://docs.rs/rayon/latest/rayon/struct.ThreadPoolBuilder.html#method.panic_handler, i.e. spawn is fire-and-forget and hence cannot propagate the panics anywhere sensible. This leads to test flakiness like https://github.com/quickwit-oss/tantivy/actions/runs/9203311046/job/25314621091.

[fulmicoton]

@adamreichold thank you for the information! Maybe we should catch panics actually.

[adamreichold]

@adamreichold thank you for the information! Maybe we should catch panics actually.

We could just install a panic handler which logs and moves on, but that could leave the system in an inconsistent state. Debugging any issues resulting from that downstream might not be fun.

Alternatively, we could wrap all closure in catch_unwind and forward the panic payloads as errors, either as thread::Result<crate::Result<Vec<R>> or by adding a panic payload variant to tantivy::Result.

[fulmicoton]

I was more thinking of catch_unwind

[fulmicoton]

I am merging this. Let me know if you want to work on the catch_unwind stuff.

[PSeitz]

This wasn't merged