Elasticsearch instead of lucene as search engine

[ghiron]

Hi,

We're moving our current implementation which is based on fhir REST server to the fhir-jpa version.
We tried to update the hibernate search implementation to to use elasticsearch instead of lucene (we added hibernate-search-elasticsearch as a dependency).
It failed because some entity classes (like ca.uhn.fhir.jpa.entity.ResourceHistoryTable) have dependencies on lucene analyzer definition which are not compatible with elasticsearch.

Did we miss something in the fhir-jpa-server config or is using ES instead of lucene in need of some refactoring ?

We suggest to split all code specific to lucene package to another dependecy away from jpa-core module in the following way :

jpa-core -> jpa-core-lucene -> lucene-highlighter, lucene-analyzers-phonetic
         -> jpa-core-elasticsearch -> hibernate-search-elasticsearch, elasticsearch-analysis-phonetic

We plan to make 2 PRs in next days / weeks to :

• extract lucene specific classes in a dedicated module (without any other modification)
• elasticsearch equivalent implementation

We welcome any feedback on such an approach !

[jamesagnew]

Well now that's annoying though that the lucene anlyzers prevent hibernate-search-elasticsearch from starting. Is that the only problem? (i.e. if you remove those annotations, does it work?)

Does this seem like something that might be easier to solve via a PR to Hibernate Search itself? I'm certainly not opposed at all to the reorg you're suggesting, it sounds like a sensible way to go for sure.. I'm just worried that it's going to be a huge pain for you to decouple things to fix this, given that things like ResourceHistoryTable are about as central to the JPA server design as it gets, so refactoring them will be hard. I guess you could put an abstract implementation in jpa-core and then concrete implementations in the subprojects? It would really be nicer though if you could just configure hibernate-search-elasticsearch to ignore those annotations...

[JiajingLiang]

Hello James,
The main problem is that when we switch the search engine to ElasticSearch, hibernate-search-elasticsearch can't convert automatically some built-in Lucene factories definitions to the Elasticsearch equivalent, especially the PhoneticFilterFactory which is not defined in ElasticSearch core, but defined in ElasticSearch plugin. This prevent the application from starting.

So what I suggest is to remove those analyser definitions with @AnalyzerDef and to use Programmatic API of Hibernate Search to define the mapping. Here is the PR: #747

[JiajingLiang]

Hello James,
I've just provided an implementation of ElasticSearch analysis definition for Fhir Jpa Server which can resolve this issue about ElasticSearch integration. #798
Maybe I should also provide a demo application with my PR, but I'm a little bit confused about all those maven modules in the project. Please just tell me where should I put demo.

[jamesagnew]

Hi @JiajingLiang - Very cool!!

Probably here would be a good spot for that example: https://github.com/jamesagnew/hapi-fhir/tree/master/hapi-fhir-jpaserver-examples

[jamesagnew]

(oops, didn't mean to close the issue 😊 )

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.