Troubleshooting Solr and Quepid
Clone this wiki locally
Connecting Solr to Quepid
Here's a blog written for Quepid beginners, explaining how to set up a connection to a Solr index (ignore the bits about the free trial and single Case, this was written before Quepid was free to use): https://www.flax.co.uk/index.html@p=3316.html
Quepid talks to Solr using the JSONP method, which basically means sending a big JSON request wrapped in some Ajax. It's not as reliable as using CORS, but avoids having to enable CORS in Solr. Your Solr should be ready out of the box!
Note: As of Solr 8.4.1, and the latest browser security enhancements, JSONP may also require additional configuration. Please review the Connectivity section for more information.
Demo Solr Indexes with the TMDB dataset
We now have a number of different versions of Solr deployed with the same TMDB dataset, as well as some others, which is useful if you have a new feature that you want to make sure is backwards compatible. solr:SolrRocks
- https://quepid-solr.dev.o19s.com/solr/tmdb/select?q=dune is Solr 8.11 with CORS setup, and HTTPS and LetsEncrypt support
- https://quepid-solr.dev.o19s.com:8986/solr/tmdb/select?q=dune is Solr 8.11 with HTTPS support
- http://quepid-solr.dev.o19s.com:8985/solr/tmdb/select?q=goonies is Solr 8.11
- http://quepid-solr.dev.o19s.com:8985/solr/statedecoded/select?q=legal is Solr 8.11
- http://quepid-solr.dev.o19s.com:8985/solr/url_for_primary_key/select?q=: is Solr 8.11 w/ URL as the id for a doc.
- http://solr:SolrRocks@quepid-solr.dev.o19s.com:8985/solr/media/select?q=Example is Solr 8.11 w/ images and Basic Auth and http links
- http://quepid-solr.dev.o19s.com:8987/solr/tmdb/select?q=legal is Solr 9.0
Highlighting of Results in Quepid Results Page
Quepid by default tries to be smart with how it handles the fields returned. If you are searching on a field, then it automatically creates a snippet of the field using highlighting. If you aren't searching on the field, then it returns the entire field. You can pass in
hl parameters as part of your query to tweak the behaviors.
As of Solr 8.4.1, Solr ships with more restrictive security options by default. This, along with a early 2020 change by all the browser vendors has tightened up the rules for browser CORS interaction. The new default of
X-Content-Type-Options appears to be breaking this functionality, which interferes with outside websites accessing a Solr instance directly. The default configuration that ships with 8.4.1 now only allows such requests to originate from the Solr host itself.
HTTPS Only Mode
If your browser is set up with HTTPS Only mode, and you are connecting to a non HTTPS Solr, then of course it won't work ;-)
Solr w/ Basic Auth
If you are using Basic Authentication, ie a URL in the format of
[http|https]://[username:password]@[host][:port]/solr/[collectionName]/select then you will NOT be prompted by your browser to authenticate.
Otherwise, if you skip setting the username and password as part of the URL, then the browser will prompt you to login.
Be aware of the fact that browsers behave differently. Using Chrome may block Quepid queries with embedded credentials without any details. See https://chromestatus.com/feature/5669008342777856 for reference. On the other hand, Firefox may work without issues.
The following workarounds are recommended to establish connectivity between Quepid and your Solr instances.
Compatibility with nosniff
solrconfig.xml example files we overrode the output format of json to make it easy to read by a human, but this triggers the nosniff issue. If you delete that override, or explicitly configure the
json response writer to use a
Assuming you are using SolrCloud, you can actually make this change via the Config API:
Confirm the change by issuing a curl command similar to
curl "http://localhost:8983/solr/gettingstarted/select?q=*:*&rows=0&wt=json" -v -i to confirm you are getting
This will satisfy the
nosniff header as the Content-Type will match the expected content for the request.
Run a private proxy that sets custom CORS/CSP headers
There are numerous proxy solutions that can customize the headers to be compatible with Quepid. We are working to supply such an option with Quepid itself.
Allow CORS requests
This is a tricky option because the CORS specification provides for allowing all hosts via a wildcard or only allowing a specific host. Specifying multiple hosts is not supported within the header itself, however it can be supported by custom code that generates the header on the fly by checking a allow list of domains.
access your Solr instances. If you're able to whitelist a domain via configuration, try adding the
app.quepid.com domain. If you need to use the wildcard you can follow the process documented here.
etc/jetty.xml <!-- security-related headers --> <Call name="addRule"> <Arg> <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule"> <Set name="pattern">*</Set> <Set name="name">Content-Security-Policy</Set> <Set name="value">default-src 'none'; base-uri 'none'; connect-src 'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 'self'; media-src 'self'; style-src 'self' 'unsafe- inline'; script-src 'self'; worker-src 'self';</Set> </New> </Arg> </Call> <Call name="addRule"> <Arg> <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule"> <Set name="pattern">*</Set> <Set name="name">X-Content-Type-Options</Set> <Set name="value">nosniff</Set> </New> </Arg> </Call> <Call name="addRule"> <Arg> <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule"> <Set name="pattern">*</Set> <Set name="name">X-Frame-Options</Set> <Set name="value">SAMEORIGIN</Set> </New> </Arg> </Call> <Call name="addRule"> <Arg> <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule"> <Set name="pattern">*</Set> <Set name="name">X-XSS-Protection</Set> <Set name="value">1; mode=block</Set> </New> </Arg> </Call>
Web browsers and best security practices are always evolving. With Quepid, we strive to keep up with these security policies while maintaining compatibility with the application.
It is never recommended to have a Solr instance exposed to the public web. It is also never recommended to "loosen" security on private instances unless it is a last resort. Practicing safe practices with your sandbox instances will lead to better practices with your production instances.
As always, Happy (and safe) searching!