Solr6 template upgrade #1504
Solr6 template upgrade #1504
Conversation
…lr spinup to create a default core using predefined config in server/confdir
…on and configuration a little
…check to select requestHandler and fixed parsing changes needed in core on our side
…han one 'suggestion' but keep current behavior. Update schema definition to get rid of _text_
…pt to try to ping correct location. Adding ping handler
… tests to come soon
…pt kwargs (search mlt etc). get_spelling_suggestions() will need to be updated
…ness of assertStatements
|
I was hoping to avoid committing anything which we don't maintain and
instead copying them from the unpacked tarball.
…On Sat, Apr 29, 2017 at 12:01 AM Kyle Stapp ***@***.***> wrote:
Boo...removing the stopwords actually caused problems...I'll look at it
tomorrow. Maybe I can config to not need it. I could just have them create
using example stuff then drop our configs in. But then it's possible we'll
have a mis-mash of old/new stuff. Any opinion? We config completely from
scratch (what i was doing with the tons of stopword files) or I change
something to not require those stopwords (possible -- but not investigated)
or I create from an example and drop solrconfig.xml and schema.xml into
place. Idealy configuring to not need those stop words would be great:).
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
<#1504 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAC15XmLSqe84z1XMx4LkpgsOmRzMnq4ks5r0rYigaJpZM4NBNHJ>
.
|
…age....but I also am not worried that random lines will get inserted into alien future configs
|
Alrighty. I think we are in a good place now. I removed all but our two per-configured files (solrconfig.xml and schema.xml), and we generate from solr. then overlay our changed files. |
|
Let me know if there's anything else. I believe everything is in order. |
|
Eh? :) |
| raise CommandError('Could not write {}: {}'.format(solrconfig_path, exc)) | ||
|
|
||
| if reload_core: | ||
| core = settings.HAYSTACK_CONNECTIONS['solr']['URL'].rsplit('/', 1)[-1] |
There was a problem hiding this comment.
This line throws an exception if the core is labeled as 'default' instead of 'solr' in HAYSTACK_CONNECTIONS. Is this the desired behavior?
| raise ImproperlyConfigured("'URL' to the core must be specified in the HAYSTACK_CONNECTIONS settings for the backend") | ||
| if 'ADMIN_URL' not in settings.HAYSTACK_CONNECTIONS[using]: |
There was a problem hiding this comment.
I don't think I can commit on this branch, so I wanted to point out the minor syntax error here!
There was a problem hiding this comment.
Thanks - that was correct when authored but it looks like I screwed up the merge
There was a problem hiding this comment.
Yeah, just a sloppy split staging those lines. I corrected the history in 03209da so both commits are valid standalone but the end state of the tree is unchanged
acdha
force-pushed
the
solr6_template_upgrade
branch
from
9c9c154
to
5608db1
Compare
|
Thanks guys for the input and editing. I just got out from under a giant project so I'll have more free time soon. Please let me know if there's any more work needed here. |
|
I migrated our project from the fork on #1315 to this one and haven't encountered any issues beyond the minor one I mentioned above (now fixed). I'll be doing more extensive testing this week and next, and will report any problems I find here. One thing worth noting is that #1315 had a |
|
I'm catching some errors that I believe are related to the new In the past, the following two queries were equivalent When I use DebugQuery, I'm seeing what might be the problem, but I'm not sure. In the old version, it returns And the new version returns Note the added |
| <defaultSearchField>{{ content_field_name }}</defaultSearchField> | ||
|
|
||
| <!-- SolrQueryParser configuration: defaultOperator="AND|OR" --> | ||
| <solrQueryParser defaultOperator="{{ default_operator }}"/> |
There was a problem hiding this comment.
This line is what's causing the problem. Without it, the error I described does not occur.
| <str name="fmap.content">{{ content_field_name }}</str> | ||
| </lst> | ||
| </requestHandler> | ||
|
|
There was a problem hiding this comment.
The analysis tool in the Solr admin also doesn't work because some RequestHanders were missing.
<requestHandler name="/analysis/field"
startup="lazy"
class="solr.FieldAnalysisRequestHandler" />
<requestHandler name="/analysis/document"
class="solr.DocumentAnalysisRequestHandler"
startup="lazy" />
|
I'll look at this today a little bit. Thanks for the messages. |
|
Good news for |
This is the Major upgrade addressing #1502 and started in #1488 to handle Solr6 (specifically 6.4 and 6.5). It adds new command parameters to build_solr_schema. The two new options allow you to specify a directory to put the config files in, and to automatically reload the core. This involved putting a new HAYSTACK_CONNECTIONS variable in place 'ADMIN_URL'.
One major change was that the template got renamed to schema.xml in the template directory instead of solr,xml
Along the way spelling suggestions format parsing was updated and handled (though the legacy style of returning only one suggestion was maintained -- we now have the plumbing to supply multiple). Upgrades were made to the tests to test spelling suggestion without collation to improve code coverage.
Upgrades were also made to handle django 11 by fixing the context passing to be native dicts in the main code base and the tests.
PEP8 changes were made, though not exhaustively.
Updated the Travis config to only use Trusty images as the others didn't have access to Oracle8 which was also required for newer Solr.
Expanded test matrix to handle Django 1.11
TBD:
Upgrade haystack to handle multiple spelling suggestions for Solr
Something about ExtendedCollation from @acdha's other comments
Expand matrix to handle multiple Solr versions (don't know what a reasonable anchor version is)
probably many more:)