Solr Synonyms Don't Work

[PsukheDelos]

Scenario
I create three different pages.
On each of them I have put one of: gdp, jel, knp (unique words so I know these aren't anywhere else on the site)
I run a Solr_Reindex
I search for "gdp" and just that one page is returned in the results.
Searching any of the unique words returns the correct result.

Now, I'd like to make these words synonyms.
I go into the Fulltext Search tab in Settings of the CMS (which uses CWP's SynonymsSiteConfig)
This config modifies the synonyms.txt file.
I enter gdp, jel, knpas my Solr Synonyms. I run a Solr_Configure.

Result
Now when I search any of the unique words, I get no results.
I do a Reindex just to see if that has an effect. Still no results.

Expected Result
What I would expect, is that if I searched for any of those words that all 3 pages would be returned.

Am I misunderstanding the concept of Solr Synonyms?
Or do they not function correctly in the module?
An issue with version of Solr used in fulltextsearch-local?
Or something else?

Just thought I would get the convo started and see if anyone else is having this issue. 😄

[robbieaverill]

Hey @tractorcow or @chillu - do you have any context to provide here?

[tractorcow]

Can you check that you've run through all the steps in https://www.cwp.govt.nz/developer-docs/en/1.7/features/solr_search/tuning_the_search_results/#search-term-synonyms-2?

Does your index extend SolrSearchIndex?

[PsukheDelos]

Thank you Damian, I will have a look at this tomorrow and let you know! G

…

On 25/10/2017, at 4:46 PM, Damian Mooyman ***@***.***> wrote: Can you check that you've run through all the steps in https://www.cwp.govt.nz/developer-docs/en/1.7/features/solr_search/tuning_the_search_results/#search-term-synonyms-2 <https://www.cwp.govt.nz/developer-docs/en/1.7/features/solr_search/tuning_the_search_results/#search-term-synonyms-2>? Does your index extend SolrSearchIndex? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#153 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACTlMRk7NIie8lDT6PGsJJ3mh_vXki8Mks5svq75gaJpZM4P89rZ>.

[PsukheDelos]

Hey @tractorcow,

I can confirm that everything is set up as expected. I am able to make changes within the CMS to the synonyms config and see them reflected in the synonyms. txt file that is generated in .solr.

The custom index extends CwpSearchIndex, which I can see contains the relevant synonyms code. However, it does not extend SolrSearchIndex (which is itself an extension of CwpSearchIndex in this case).

I have also been able to replicate this by swapping out the custom index and using the SolrSearchIndex directly instead.

-Glen

[tractorcow]

Ok, the fact that synonyms.txt exists in that folder is a good sign; The issue is in the solr configuration, not in your code.

Next step is to debug the XML that solr is using... which requires a bit of docs reading and understanding to debug.

[tractorcow]

gdp, jel, knp

Could it be a min-character for synonyms isn't being met? Check that setting first. :)

[PsukheDelos]

I have done a fresh CWP 1.7.0 install to eliminate any further variables. I've created three pages with their page name, navigation label and content as "tortoise", "kangaroo", and "stingray". This uses the standard SolrSearchIndex.

When I search for these terms without synonyms, the relevant individual pages are returned as expected.

However, when I add synonyms to the config (and run Solr_Configure to see them reflected in the synonyms.txt) no search results appear for any of those words now. Instead I get Sorry, your search query did not return any results.

[tractorcow]

However, when I add synonyms to the config (and run Solr_Configure to see them reflected in the synonyms.txt) no search results appear for any of those words now. Instead I get Sorry, your search query did not return any results.

Depending on your config you will ALSO need to run Solr_Reindex. Synonyms can be configured either against the query, or against the source dataset; The latter will require a re-index for it to activate.

[PsukheDelos]

Hi Damian,

I have run Solr_Reindex. I usually do just in case. I'm using a standard CWP 1.7.0 installation and am getting no results.

You might be onto something however. Maybe the instructions for setting up synonyms or the implementation of synonyms with the CwpSearchIndex don't go far enough to actually implement query time synonym use. From the CWP set up instructions it says It's not necessary to run Solr_Reindex in order for changes in synonyms to take effect., meaning that it must be designed to use synonyms at query time, although perhaps that is not actually implemented. Do you know where in cwp, cwp-core, or fulltextsearch that it is actually set up to use synonyms at query time or is does SOLR simply do this itself when synonyms.txt is present?

[tractorcow]

Do you know where in cwp, cwp-core, or fulltextsearch that it is actually set up to use synonyms at query time or is does SOLR simply do this itself when synonyms.txt is present?

Here is the query config that declares query-time synonym interpolation.

https://github.com/silverstripe/silverstripe-fulltextsearch/blob/master/conf/solr/4/templates/types.ss#L140-L154

It looks like this is uploaded directly to CWP, so you could possibly force it to index-time if you wanted to.

[robbieaverill]

Fixed in #156

[dhensby]

wow - good job figuring this out!