SOLR-17066: Switch ConcurrentSolrClients away from core URLs #2254
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SOLR-17066 introduced a withDefaultCollection builder method that can be used in lieu of providing a core URL as a "base". (Core URLs are not ideal as they prevent clients from making core-agnostic requests.) However, Solr's own client usage still needs modified to use this new builder method. This commit migrates all ConcurrentUpdateSolrClient usage away from core URLs and towards using `withDefaultCollection` instead.
SOLR-17066 introduced a withDefaultCollection builder method that can be used in lieu of providing a core URL as a "base". (Core URLs are not ideal as they prevent clients from making core-agnostic requests.) However, Solr's own client usage still needs modified to use this new builder method. This commit migrates all LBHttpSolrClient usage away from core URLs and towards using `withDefaultCollection` instead.
SOLR-17066 introduced a withDefaultCollection builder method that can be used in lieu of providing a core URL as a "base". (Core URLs are not ideal as they prevent clients from making core-agnostic requests.) However, Solr's own client usage still needs modified to use this new builder method. This commit migrates all ConcurrentUpdateHttp2SolrClient usage away from core URLs and towards using `withDefaultCollection` instead.
SOLR-17066 introduced a withDefaultCollection builder method that can be used in lieu of providing a core URL as a "base". (Core URLs are not ideal as they prevent clients from making core-agnostic requests.) However, Solr's own client usage still needs modified to use this new builder method. This commit migrates all LBHttp2SolrClient usage away from core URLs and towards using `withDefaultCollection` instead. Has a test failure in TestLBHttp2SolrClient that still needs resolved.
epugh
approved these changes
Feb 8, 2024
| @@ -83,7 +83,8 @@ private SolrClient buildClient(CloseableHttpClient httpClient, URL url) { | |||
| switch (random().nextInt(3)) { | |||
| case 0: | |||
| // currently, only testing with 1 thread | |||
| return new ConcurrentUpdateSolrClient.Builder(url.toString() + "/" + COLLECTION) | |||
| return new ConcurrentUpdateSolrClient.Builder(url.toString()) | |||
There was a problem hiding this comment.
sigh... if only a builder new what a url was and we didn't .tostring everywhere. ;-) #beatdeadhorse
| @@ -156,7 +158,7 @@ public void testSimple() throws Exception { | |||
| // Start the killed server once again | |||
| solr[1].startJetty(); | |||
| // Wait for the alive check to complete | |||
| Thread.sleep(1200); | |||
| Thread.sleep(1200 * 5); | |||
There was a problem hiding this comment.
I have no memory of this 🤔 - will remove.
| @@ -167,17 +169,15 @@ public void testSimple() throws Exception { | |||
| } | |||
| } | |||
|
|
|||
| private LBHttp2SolrClient getLBHttp2SolrClient(Http2SolrClient httpClient, String... s) { | |||
| @@ -31,6 +31,8 @@ | |||
| /** Unit tests for {@link Builder}. */ | |||
| public class ConcurrentUpdateSolrClientBuilderTest extends SolrTestCase { | |||
|
|
|||
| private static final String ANY_BASE_URL = "http://localhost:8983/solr"; | |||
5 tasks
Zombie-resurrection was broken in LBHttp2SolrClient because the default collection wasn't being taken into account on these requests. The root of the problem is the way that LBHttp2 takes in an already-constructed SolrClient, making it difficult to set the default collection. (In contrast, LBHttpSolrClient creates a different client for each base URL using a common Builder - this allows it to property set the default collection.) This commit fixes this by created a 'DelegatingSolrClient' that LBHttp2 can use internally around its single Http2SolrClient. The default collection is set on this 'Delegating' wrapper, so that it gets picked up by the zombie-resurrection logic.
epugh
reviewed
Feb 9, 2024
| @@ -158,7 +158,7 @@ public void testSimple() throws Exception { | |||
| // Start the killed server once again | |||
| solr[1].startJetty(); | |||
| // Wait for the alive check to complete | |||
| Thread.sleep(1200 * 5); | |||
| Thread.sleep(1200); | |||
There was a problem hiding this comment.
The most important code change ;-)
|
Turns out doing this for our 'LB' clients is a little more involved than I initially thought, due to some of the additional public methods they have (e.g. I'll open a separate PR for tackling the LB clients. |
gerlowskija
changed the title
gerlowskija
added a commit
that referenced
this pull request
Feb 14, 2024
Providing a core URL as a SolrClient's "base URL" prevents it from communicating with other cores or making core-agnostic API requests (e.g. node healthcheck, list cores, etc.) This commit migrates all usage of both "concurrent update" clients away from core URLs.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/SOLR-17066
Description
SOLR-17066 introduced a
withDefaultCollectionbuilder method that can be used in lieu of providing a core URL as a "base". (Core URLs are not ideal as they prevent clients from making core-agnostic requests.)However, Solr's own client usage still needs modified to use this new builder method.
Solution
This commit migrates Solr's "concurrent update" Solr client implementations away from core URLs and towards using
withDefaultCollection. Following this PR, the only client that still remains to be migrated are the two "load balancing" clients.)Tests
Existing tests continue to pass. No new tests needed, as this is "just" a refactor.
Checklist
Please review the following and check all that apply:
mainbranch../gradlew check.