Deadlock in AnalysisSPILoader [LUCENE-10665]

[asfimport]

Loading different TokenFilter/Tokenizer/CharFilter from different threads is causing deadlock. To reproduce use the below code:

public static void main(String[] args) { 
  Thread thread1 = new Thread(UpperCaseFilterFactory::new);
  Thread thread2 = new Thread(LowerCaseFilterFactory::new);
  thread1.start();
  thread2.start();
}

Took the thread dump, and found that thread1 gets stuck while calling ensureClassInitialized(LowerCaseFilterFactory) possibly because thread2 is holding some lock on the same class.

---

Migrated from LUCENE-10665 by Jasir KT

[uschindler]

Hi,
this is exactly the same issue we have seen with Codecs and PostingsFormats. The problem is the following: The base class of all analysis factories (TokenizerFactory, TokenFilterFactory, and CharFilterFactory) has a static initializer that runs the SPI lookup. This causes all subclasses to be initialized. If another thread is doing the same it happens that the deadlock occurs.

A workaround was implemented for Codecs and other components in Lucene core, but not yet for analysis components:
https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/Codec.java#L40-L59

This must be done in the three base classes in same way. Actually as there is no "default analyzer" the extra checks can be ommitted. It is just important to move the static field into a separate class outside of analysis class hierarchy.

I will tkae care of this issue. In which verison did you see this? We may backport it to 8.x series but I would fix the issue for versions 9.x first.

[uschindler]

See #7541 and #10700 for more details.

[uschindler]

PR (main branch): #11718

[jasirkt]

In which verison did you see this?

9.1.0

Thanks for fixing. It works now!

[uschindler]

Thanks for feedback, well appreciated.