Skip to content

DCAwareRoundRobinPolicy and RackAwareRoundRobinPolicy populate does no register all the nodes in some cases #576

@dkropachev

Description

@dkropachev

populate is using groupby in both policies:

def populate(self, cluster, hosts):
for dc, dc_hosts in groupby(hosts, lambda h: self._dc(h)):
self._dc_live_hosts[dc] = tuple(set(dc_hosts))

def populate(self, cluster, hosts):
for (dc, rack), rack_hosts in groupby(hosts, lambda host: (self._dc(host), self._rack(host))):
self._live_hosts[(dc, rack)] = tuple(set(rack_hosts))
for dc, dc_hosts in groupby(hosts, lambda host: self._dc(host)):
self._dc_live_hosts[dc] = tuple(set(dc_hosts))

groupby will produce another key every time it encounter next group, example:

for dc, dc_hosts in groupby([{'k': 1}, {'k': 1}, {'k': 2}, {'k': 2}], lambda v: v['k'] ):
    print(dc, list(dc_hosts))
# 1 [{'k': 1}, {'k': 1}]
# 2 [{'k': 2}, {'k': 2}] 

for dc, dc_hosts in groupby([{'k': 1}, {'k': 2}, {'k': 1}, {'k': 2}], lambda v: v['k'] ):
    print(dc, list(dc_hosts))
# 1 [{'k': 1}]
# 2 [{'k': 2}]
# 1 [{'k': 1}]
# 2 [{'k': 2}]

But code in both policies does not consider that and overwrite original value, as result, there is a case when policy will loos some of the hosts, which results in weird routing behavior.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions