Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new buffered_sub_entry_messages in Producer #128

Closed
garvenlee opened this issue Sep 21, 2023 · 4 comments
Closed

Add a new buffered_sub_entry_messages in Producer #128

garvenlee opened this issue Sep 21, 2023 · 4 comments
Assignees
Labels
performance refactoring refactoring of code

Comments

@garvenlee
Copy link

The messages in send and send_sub_entry can be handled separately in the _timer task like this:

async def _timer(self):
    while True:
        await sleep(delay)
        for stream, messages in buffered_messages.items():
            await _send_batch()

        for stream, messages in buffered_sub_entry_messages.items():
            await _send_sub_entry_batch()

And redesign the _send_batch: the arg sync is only used in function send_wait, so it can be removed from _send_batch.

@Gsantomaggio
Copy link
Collaborator

Thank you, feel free to open a PR !

@DanielePalaia
Copy link
Collaborator

Ah yes this is also a known issue: #55

There was a plan to redesign it but with lower priority.

But yes feel free to open a PR!

@DanielePalaia
Copy link
Collaborator

I take this issue also to decouple send_batch from send/send_sub_batch in order to avoid the initial loop:
See: #148

@DanielePalaia
Copy link
Collaborator

I will close this issue as it is not a top priority at the moment.
We already made some decouples of the API in the past here:
#55

At the moment even if it's possible to run the client in parallel on the two different modes: normal, and with sub-entry-batch enabled I don't think it will really be a common case so at the moment we will focus on other improvements.

We will come back to it in a next iteration if needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance refactoring refactoring of code
Projects
None yet
Development

No branches or pull requests

3 participants