Different behaviour of embedded Solr vs Solr container?

[icarito]

Hi,
We are PublicLab.Org appreciate Sunspot and are in the process of deploying it to production.

Our testing and development was done with the embedded Solr engine found in sunspot_solr (2.2.6) gem.

Now for production we are using a Solr 5.3 container (as found in https://hub.docker.com/_/solr/). In this case, the default Sunspot query is not recognized by the Solr service , specifically, the parameters qf=title_text+body_text+comment_text and defType=edismax cause Solr to not return any results.

Adjusting these parameters for production, causes results to break in development (the embedded Solr).

Thanks for your guidance!

[jywarren]

Quick edit; those parameters cause Solr to return only some results and not others. -- we get some hits, but very spotty, and miss a lot of content. If either of those parameters are removed (in a manual query from the Solr web console) -- such as qf='' or actually fully removing defType=edismax -- the search succeeds and we get full expected results.

[jywarren]

I've added more detail on this Stack Overflow post: https://stackoverflow.com/questions/44617017/difficulty-getting-more-than-exact-field-match-results-on-sunspot-solr-with-qf

[jywarren]

It seems that the default sunspot gem query only returns an exact, case-sensitive match of the title field. For example:

• https://publiclab.org/searches/test/test?q=Moderation
• https://publiclab.org/searches/test/test?q=moderation

So, either we need to

a) figure out why the seemingly well-formed sunspot-generated query only matches exact title text and change that, OR
b) figure out why our manual edits to the query return nothing at all in embedded (sunspot gem) Solr

[jywarren]

When using edismax, qf= seems not to be behaving as we'd expect it to based on the edismax docs or common Solr queries docs.

The sunspot source seems to track those pretty well:

sunspot/sunspot/lib/sunspot/query/join.rb

Line 27 in 49904ac

params[:qf] = @fulltext_fields.values.map { |field| field.to_boosted_field }.join(' ')

And I am attempting (in Solr console) the same difference without sunspot involved at all:

/solr/default/select?wt=json&indent=true&fq=type:Node&q=balloon&fl=*%20score&qf=title_text%20body_text%20comment_text&defType=edismax

returns nothing, while

/solr/default/select?wt=json&indent=true&fq=type:Node&q=balloon&fl=*%20score&qf=title_text%20body_text%20comment_text

returns full results.

[jywarren]

Just a follow-up; we've ended up switching away from the embedded Solr provided in the sunspot_solr gem on Rails, and using a containerized (and more up-to-date) Solr. This has bypassed the issue, though not really solved it.

[icarito]

Actually, @jywarren we did figure this out! It turns out that the Solr engine needs the "Sunspot configsets" which, in the case of the container, need to be loaded as a volume.
In this commit, we included the configset and setup the container to pre-build the core from this configuration. After doing this, tests of both container and gem-embedded Solr coincide. It was hard to figure out, so worth leaving a comment for posterity here.
This one reference helped figure it out: https://www.norconex.com/use-solr-5-with-docker/
Regards!