Skip to content

fluvio consume stalls if produced batch size is too large #2259

@tjtelan

Description

@tjtelan

This came out of testing #2258

I produced 1 million JSON records (~1GB) with a batch size of 1MB, no compression.

When I tried consume from this topic (with fluvio consume -B -d my-topic) the consumer just sat there and never returned any data.

This worked fine when I used the -T option (instead of -B) to pull the last 10 records.

I expect there to be some output (even if I have to wait a while) but even after a few minutes I never received data.


I deleted the topic and produced the records again, but with a batch of 500K, no compression. And fluvio consumer -B -d my-topic worked

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions