Allow SOLR integration for vocabulary search.

[fdefalco]

Work initially started in Helios branch.

Much like security, SOLR integration will be disabled by default. A new property in the WebAPI configuration will allow you to toggle SOLR support. If SOLR support is enabled then the URL of the SOLR server must be configured. With SOLR support enabled the admin screen will provide a feature to build a SOLR index (core) off of an available CDM vocabulary version. As calls to the search API are received, if there is a SOLR index available for the version of the vocabulary, that search will be handled by SOLR.

A corresponding update to the UI should indicate that advanced capabilities are available, by providing a 'powered by SOLR' message beneath the search box along with a help pop up with some basic search capabilities provided by SOLR and a link to the SOLR documentation.

https://github.com/OHDSI/WebAPI/tree/helios

[pavgra]

Should be related to #505 and the resulting component must be shared between Atlas & Achilles

[anthonysena]

Agreed that this issue and #505 are related. To clarify:

the resulting component must be shared between Atlas & Achilles

@pavgra do you mean to say ATLAS & Athena instead of Achilles? Also, can you point us to the relevant code in the Athena code base that could be reviewed for thinking through a design that could be re-used between WebAPI and Athena?

[pavgra]

do you mean to say ATLAS & Athena instead of Achilles?

yep, sorry, misnamed. of course, Atlas & Athena

can you point us to the relevant code in the Athena code base
We could re-use DTOs, converters, services related to the concept search from
https://github.com/OHDSI/Athena

However, one of the most important things is Solr configuration, specifically tokenization, stemming and relevancy calculation (which is still not perfect in Athena, so we'll have a good reason to revisit the topic):
https://github.com/OHDSI/Athena/tree/master/src/main/resources/solr

[fdefalco]

@pavgra I know we've discussed integration possibilities in the past including having Athena include the SOLR Core for the vocabulary being distributed when someone downloads a vocabulary update. How does Athena manage the creation of the Core today for its search? Could that be the first component that is reusable?

[pavgra]

I don't think that sharing a Solr core together with vocabs bundle is the best option. Since Solr cores are built on top of original data and work like a "view", I strongly believe that Atlas should be able to generate those based on data in connected databases (sources). Athena does the same using Solr import handler (some docs, Athena code).

Anyway, as soon as we are done with Symposium release version and ready to talk about Atlas v3.0, I'll be happy to return back to the issue referenced in first response, and will provide design proposal for the list of shared component which we'd like to address first.

[konstjar]

@anthonysena Is there any documentation on how to configure this feature? I think it's good candidate for a wiki article

[anthonysena]

@konstjar - I had written up some instructions here which still need to go into the Wiki. If someone could review & port to the wiki, I'd be very grateful. Thanks!

[konstjar]

Great! We will review and validate in our environment