Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

One slow broker can slow fetch responses from all brokers #218

Open
thomaslee opened this issue Jan 30, 2018 · 0 comments
Open

One slow broker can slow fetch responses from all brokers #218

thomaslee opened this issue Jan 30, 2018 · 0 comments

Comments

@thomaslee
Copy link

The call to Client.fetchRequest in BaseConsumer_fetch wraps requests to all brokers in a Promise.all(), meaning that one slow/misbehaving broker can leave us idle in spite of data being returned by all other brokers.

Returning a Promise.map(...) might be better, but would probably require changes to what is returned by fetchRequest (from a single topics object to individual topic objects per leader) & I'm not entirely sure what guarantees we're making about API stability.

Locally I've tinkered with splitting up calls to fetchRequest by leader in BaseConsumer order to fully decouple broker requests from one another & avoid this issue without tweaking Client semantics, but that seems a bit dirty.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant