[serve] Add basic session affinity via shard key

[edoakes]

Why are these changes needed?

When splitting traffic among backends, users may want a single user/session/client's traffic to deterministically map to a given backend to avoid poor UX/increase signal for A/B tests. This is a basic/limited solution, allowing users to specify a "shard key" with each request that will be hashed to map the request to a backend.

This solution is limited in that if the user changes the traffic policy, the mapping for some or all users may change.

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/latest/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failure rates at https://ray-travis-tracker.herokuapp.com/.
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested (please justify below)

[AmplabJenkins]

Can one of the admins verify this patch?

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25909/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25910/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25914/
Test FAILed.

[simon-mo]

LGTM