Make routing_nodes an independent metric option in cluster state api

[lmenezes]

closes #10352

[javanna]

I like it @lmenezes , thanks! Makes sense to keep bw comp in 1.x and still return routing_nodes too when asking for routing_table, I would change that in master probably as a follow-up. @clintongormley want to double check?

[clintongormley]

+1 to this change, and yes i'd make it 2.0 only

[lmenezes]

@clintongormley great! but for 2.0 only you mean making routing_table routing_table only, right? the routing_nodes as an independent metric could go on 1.X, right? :)

[clintongormley]

@lmenezes thinking about it again... it's such an easy change for users to make in 1.6, that i'd be tempted to just go ahead and do it, ie make the change completely in 1.6.

[lmenezes]

@clintongormley perfect for me. just thought it could be useful to have a smooth transition, where you could upgrade your nodes and have old behaviour, and then adapt your code on a cluster that already has this change.

anyway, any of the alternatives work for me :)

[javanna]

Hi @lmenezes I gave a final look at this, I wonder if we should add the same options to the Java api as well. We would need to add a routingNodes flag to ClusterStateRequest and handle bw comp on the wire format using version checks. Thoughts?

[lmenezes]

@javanna makes sense. Will update the PR with this :)

[lmenezes]

@javanna so, just had time to look into this... Does it really make sense to separate that for the Java API?

On the REST API, routing_nodes and routing_table are 2 completely different entities(even though both are rendered from the same data) and returning them or not might make difference(bytes sent through the wire, and the process of iterating the routing table to built the json representation for each of them).

On the Java API, the RoutingTable object has to be sent over the wire anyway, and the RoutingNodes is actually volatile and built on demand from the RoutingTable(https://github.com/elastic/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/ClusterState.java#L134 and https://github.com/elastic/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/ClusterState.java#L226-L232).

So, unless I'm missing something, it would make no difference at all(except from offering the same options on both Java/REST API) being able to control that individually on the Java API, and likely also make it confusing since you cannot really offer RoutingNodes without offering RoutingTable.

Makes sense?

[javanna]

hey @lmenezes you are right, this makes perfect sense to me. The routing table is what gets serialized over the wire, routing nodes are built on demand when requested first based on the routing table. I will merge this as-is, thanks a lot for looking into this!

[lmenezes]

@javanna thank you for taking the time to merge this 👍

[javanna]

Merged, thanks a lot @lmenezes !