Implement hash-based routing as a per-route load balancing Algorithm in gorouter

[Soha-Albaghdady]

Proposed Change

As a CF app developer,
I want to leverage hash-based routing in my application scenarios,
So that incoming traffic can be efficiently distributed across various instances based on predefined hash values, ensuring optimal performance and reliability.

In n RFC-0042 "Implementing a Hash-Based Load Balancing Algorithm for CF Routing" we decided to introduce hash-based routing as a load balancing algorithm to be used on a per-route basis. This approach involves the Gorouter using the hash of a request header to make routing decisions. The focus is on distributing users across instances instead of individual requests.

Acceptance criteria

• Gorouter can parse the new fields containing the per-route options related to hash-routing
• Gorouter can calculate a hash based on the header
• Gorouter can apply the new load-balancing algorithm to make routing decisions for routes that provide fields related to the hash-routing
• Gorouter can assess the current number of in-flight requests across all application instances and ensure the balance factor is not exceeded before routing to an instance
• Gorouter should not incur any performance impact when no application use hash-routing

Related links

No response

[plowin]

Discussion around performance-testing and feature-flags is taking place via Slack:
https://cloudfoundry.slack.com/archives/C02HNDJB31R/p1757068276784729

The decisions will be documented here.

[hoffmaen]

We would like to discuss the following edge cases. I added our proposals for each case.

1. X-CF-App-Instance vs Hash-Based Routing: The X-CF-App-Instance header's purpose is to route requests to a specific app instance for debugging. Thus, the header should have precedence.
2. Sticky Sessions (Session Affinity) vs. Hash-Based Routing: The sticky session cookie should have precedence and the request should be routed to the instance stored in the cookie. This is the case when an application sets the JSESSIONID cookie, which is then sent by the client, together with Gorouter's __VCAP_ID__ cookie. This allows a consistent behavior across load balancing algorithms.
3. Requests do not have the header set that is configured for the hash-based route option. In this case, we should fall back to the global default load balancing algorithm (Round Robin or Least Connection).
4. locallyOptimistic AZ_PREF_LOCAL and localAvailabilityZone vs Hash-Based Routing: As both locallyOptimistic and localAvailabilityZone are global properties, they should be ignored when using Hash-Based Routing. Otherwise, hash-based routing couldn't be used in setups where these properties are enabled/set.
5. Retries vs Hash-Based Routing: If the initial instance fails, subsequent retries should be routed to other app instances.