9635 solr improvements #10050

landreev · 2023-10-26T14:57:55Z

What this PR does / why we need it:

The following application-side improvements/workarounds are being added as part of this PR:

Adding app-side support for the Solr-side circuit breakers. Enabling those makes Solr start dropping requests with 503s (instead of dying) under heavy load. As currently implemented however, the page does not detect that condition and doesn't handle it in any special way, and simply shows zero search results/empty collection page. It needs to show something intelligent, along the lines of "search engine is experiencing heavy load, please try again later".
Optimizing the second search that the page repeats solely for the purpose of populating the type counts facets for the unchecked types. There is absolutely no need to repeat the same exact search, with all its super expensive parts, like all the other facets. There's no need to search for anything other than these one or two total counts; or to even run this second search, if all 3 type facets happen to be checked.
Adding nofollow to all the facet links and such, that we don't want crawlers/bots to touch. This will obviously only help with the nicer bots that take hints and follow rules, but better than nothing.
An off switch for facets support on the dataverse page. Again, facets are super expensive; it should be far better to be able to display the page with just the search cards and "facets are temporarily unavailable because of load issues" in the left column, than to have an empty page/have a "solr down" error page/have the page load take an hour.

Which issue(s) this PR closes:

Closes #9635

Special notes for your reviewer:

Suggestions on how to test this:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?:

Additional documentation:

…hes, unless specifically requested. (#9635)

…nd pass query sent in order to populate the unchecked type count facets. (#9635)

…enerating facets. #9635

…guide as well. #9635

landreev · 2023-10-30T14:22:17Z

I just added a "closes" tag for #9635 to the PR. Once the pr is merged, I want to hot-patch our prod. with the improvements here, and then close the issue, at least for now. As I said before, there's definitely more to research and investigate and I want to get back to it, but there may be more pressing things r/n.

…he dataset page #9635)

landreev · 2023-10-30T18:42:42Z

OK, one test is legitimately failing in this PR, will fix.

…reindexing the parent dataverse when adding guestbooks, stops the update dataverse command from unnecessarily reindexing the underlying datasets in some cases, makes that reindex truly async. in the onSuccess part of the command. (#9635)

…minates reindexing the" (some things got checked in by mistake) This reverts commit 18cd587.

…rk: eliminates reindexing the parent dataverse when adding guestbooks, stops the update dataverse command from unnecessarily reindexing the underlying datasets in some cases, makes that reindex truly async. in the onSuccess part of the command. (#9635)

qqmyers · 2023-11-03T15:52:20Z

src/main/java/edu/harvard/iq/dataverse/DatasetPage.java

+        if (settingsWrapper.isTrueForKey(SettingsServiceBean.Key.DisableSolrFacets, false)) {
+            return false;
+        }
+


re: the following code - is it worth it to query solr just to see if you can query solr? Would a check of the last indexed time be a reasonable alternative? (Not sure how much this would save given it's a simple query).

Yeah, this looks like it's yet another thing we can save/economize on.
We are already checking that this is the latest released version... so, yes, comparing the version published date against the indexed time stamp should be reliable enough.
These queries that just search for the version id cannot be that expensive; but there's definitely a lot of them in the logs.

qqmyers · 2023-11-03T15:53:30Z

src/main/java/edu/harvard/iq/dataverse/GuestbookPage.java

                guestbook.setEnabled(true);
                dataverse.getGuestbooks().add(guestbook);
-                cmd = new UpdateDataverseCommand(dataverse, null, null, dvRequestService.getDataverseRequest(), null);                
-                commandEngine.submit(cmd);
+                logger.info("Returned from command");


I added it just for debugging the async part - probably not needed, but at least should be fine()

I actually noticed it, and meant to drop it. 😄

qqmyers

Looks good. I haven't tested everything, but I have cherry-picked the nofollow links to QDR (and have the changes to the Guestbook page running there as well. The rest of the changes make sense.

landreev · 2023-11-03T16:08:44Z

Looks good. I haven't tested everything, but I have cherry-picked the nofollow links to QDR (and have the changes to the Guestbook page running there as well. The rest of the changes make sense.

Cool, thank you. I will refactor the "version searchable" test above, and drop the .info log line quickly.

sekmiller · 2023-11-15T15:47:23Z

Seeing some weird results on the counts for dvobject facets in child Dataverses when those types are unselected.

sekmiller · 2023-11-15T16:34:34Z

You can see the above on Dataverse internal at https://dataverse-internal.iq.harvard.edu/dataverse/sek11-15

(I added a couple of objects while facets were unavailable and it worked fine)

landreev · 2023-11-16T18:22:40Z

@sekmiller
Just to confirm that yes, this is clearly a bug. The counts shown for the unchecked types are the totals for the root collection, not the current sub-collection.
... But I'm still working on a fix.

…ry. #9635

landreev · 2023-11-20T12:42:23Z

@sekmiller
Sorry for the delay. The fix was quite simple actually.

pdurbin · 2023-11-20T19:50:36Z

@sekmiller and I talked and I just kicked this off for a second opinion on API tests: https://github.com/gdcc/api-test-runner/actions/runs/6935079103

Update: it failed with Error: DataversesIT.testMalformedFacetQueryString:330 expected: <true> but was: <false>

landreev · 2023-11-27T14:40:20Z

I still need to fix the API test above.

… ways to improve detecting bad facet queries early on (and trying to decide if it's out of scope or not). #9635

…SS/dataverse into 9635-solr-search-improvements

github-actions · 2023-11-28T19:39:58Z

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:9635-solr-search-improvements

ghcr.io/gdcc/configbaker:9635-solr-search-improvements

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

landreev · 2023-11-28T20:35:43Z

The tests are passing now.

landreev added 11 commits October 16, 2023 17:00

adding extra logic to skip things like facets and highlights in searc…

48144a2

…hes, unless specifically requested. (#9635)

more fixes/cleanup #9635

6307292

more fixes (#9635)

74eb7c5

finally working as it should; much simplified/way less expensive seco…

f1e37ae

…nd pass query sent in order to populate the unchecked type count facets. (#9635)

a stub for interecepting a "circuit breaker" 503 from the server (#9635)

c1a1929

minor incremental changes (#9635)

f056d6c

Merge branch 'develop' into 9635-solr-search-improvements

55e8227

adding rel="nofollow" to the links throughout the collection page. #9635

92e5917

an off switch for the facets on the collection page. (#9635)

643c555

a couple of places in the api where we can save solr the trouble of g…

59bc5ad

…enerating facets. #9635

guide entry, etc. #9635

25c180a

landreev added this to Ready for Review ⏩ in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) via automation Oct 26, 2023

This comment has been minimized.

Sign in to view

documented the "disable facets" setting in the config section of the …

abd0754

…guide as well. #9635

This comment has been minimized.

Sign in to view

(the "disable facets" switch should also cover the search facets on t…

bec806a

…he dataset page #9635)

This comment has been minimized.

Sign in to view

landreev added 2 commits November 2, 2023 12:38

Revert "A few more improvements, borrowed from @qqmyers QDR fork: eli…

5b1c800

…minates reindexing the" (some things got checked in by mistake) This reverts commit 18cd587.

This comment has been minimized.

Sign in to view

qqmyers reviewed Nov 3, 2023

View reviewed changes

qqmyers approved these changes Nov 3, 2023

View reviewed changes

IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) automation moved this from Ready for Review ⏩ to Ready for QA ⏩ Nov 3, 2023

This comment has been minimized.

Sign in to view

sekmiller assigned landreev Nov 15, 2023

landreev added 4 commits November 19, 2023 21:09

extra logging

8766932

extra logging

e4aea93

A fix for the missing subtree filter query in the 2nd pass search que…

235b1b0

…ry. #9635

typo. #9635

ceeeaec

This comment has been minimized.

Sign in to view

landreev removed their assignment Nov 20, 2023

Merge branch 'develop' into 9635-solr-search-improvements

b1ceecd

This comment has been minimized.

Sign in to view

landreev self-assigned this Nov 27, 2023

landreev added 2 commits November 28, 2023 13:32

Checking in the trivial fix for the failing tests. Still looking into…

908a624

… ways to improve detecting bad facet queries early on (and trying to decide if it's out of scope or not). #9635

Merge branch '9635-solr-search-improvements' of https://github.com/IQ…

f65d571

…SS/dataverse into 9635-solr-search-improvements

This comment has been minimized.

Sign in to view

one extra weird edge case (restores the existing behavior). #9635

8403a59

cmbz added the Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) label Nov 29, 2023

sekmiller merged commit 960751f into develop Nov 29, 2023
24 checks passed

IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) automation moved this from QA ✅ to Done 🚀 Nov 29, 2023

sekmiller deleted the 9635-solr-search-improvements branch November 29, 2023 20:16

pdurbin added this to the 6.1 milestone Nov 29, 2023

jp-tosca mentioned this pull request Mar 21, 2024

Move publication status facet from bottom to where it used to be #10338

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

9635 solr improvements #10050

9635 solr improvements #10050

landreev commented Oct 26, 2023 •

edited

This comment has been minimized.

This comment has been minimized.

landreev commented Oct 30, 2023

This comment has been minimized.

landreev commented Oct 30, 2023

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

qqmyers Nov 3, 2023

landreev Nov 3, 2023

qqmyers Nov 3, 2023

landreev Nov 3, 2023

qqmyers left a comment

landreev commented Nov 3, 2023

This comment has been minimized.

This comment has been minimized.

sekmiller commented Nov 15, 2023

sekmiller commented Nov 15, 2023

landreev commented Nov 16, 2023

This comment has been minimized.

landreev commented Nov 20, 2023

This comment has been minimized.

pdurbin commented Nov 20, 2023 •

edited

landreev commented Nov 27, 2023

This comment has been minimized.

github-actions bot commented Nov 28, 2023

landreev commented Nov 28, 2023

9635 solr improvements #10050

9635 solr improvements #10050

Conversation

landreev commented Oct 26, 2023 • edited

This comment has been minimized.

This comment has been minimized.

landreev commented Oct 30, 2023

This comment has been minimized.

landreev commented Oct 30, 2023

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

qqmyers Nov 3, 2023

Choose a reason for hiding this comment

landreev Nov 3, 2023

Choose a reason for hiding this comment

qqmyers Nov 3, 2023

Choose a reason for hiding this comment

landreev Nov 3, 2023

Choose a reason for hiding this comment

qqmyers left a comment

Choose a reason for hiding this comment

landreev commented Nov 3, 2023

This comment has been minimized.

This comment has been minimized.

sekmiller commented Nov 15, 2023

sekmiller commented Nov 15, 2023

landreev commented Nov 16, 2023

This comment has been minimized.

landreev commented Nov 20, 2023

This comment has been minimized.

pdurbin commented Nov 20, 2023 • edited

landreev commented Nov 27, 2023

This comment has been minimized.

github-actions bot commented Nov 28, 2023

landreev commented Nov 28, 2023

landreev commented Oct 26, 2023 •

edited

pdurbin commented Nov 20, 2023 •

edited