SOLR-12217: Support shards.preference for individual shard requests

[HoustonPutman]

Description

Use shard preference rules when choosing replicas to send individual shard requests. This includes requests from SolrJ as well as requests sent via streaming expressions.

Solution

How routing preferences are determined:

For Streaming Expressions, it will use the preferences from the following sources, in order of preference (defaults last):

1. The params given in the stream source
2. The URL params sent in the streaming request
3. The DEFAULT_SHARD_PREFERENCES set in cluster properties

For SolrJ, it will use the preferences from the params of the request it is sending.

Tests

Current tests pass.

TODO Add tests for routing functionality

Checklist

Please review the following and check all that apply:

• I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
• I have created a Jira issue and added the issue ID to my pull request title.
• I am authorized to contribute this code to the ASF and have removed any code I do not have a license to distribute.
• I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
• I have developed this patch against the master branch.
• I have run ant precommit and the appropriate test suite.
• I have added tests for my changes.
• I have added documentation for the Ref Guide (for Solr changes only).

[tflobbe]

Looks good. I don't know much about the streaming part, maybe @joel-bernstein can take a look, but in general, LGTM other than the comments.

[tflobbe]

Is this necessary for updates? are you trying to keep consistency only or do you have some use case in mind?

[HoustonPutman]

Mainly for consistency. I guess it's mostly a no-op, since the leader is always going to be first anyways (if a leader exists).

[tflobbe]

LGTM. I'm running tests and there are some failures, most probably unrelated but want to make sure

[HoustonPutman]

Want to hear your opinion @tflobbe .

So I changed the logic for non-update/admin/v2 request routing to use core URLs instead of collection URLs, unless the request is querying multiple collections. In this way, if there are certain replicas on a host that you want to target you can make that distinction. However that does change the default behavior in a non-trivial way.

The only place where I see this breaks any test is weirdly enough in the Hadoop auth tests for sending a commit request. It seems that since the commit request is sent directly to a core, instead of the collection url, it is missing an additional count of unauthenticated requests.

[HoustonPutman]

I've decided to be conservative and revert that logic. So for non-update requests, SolrJ will send the request to the collection URL, not the replica URL. All tests pass on my end now.

[tflobbe]

Changes LGTM @HoustonPutman. You don't need me for merging now :)