Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send multiple event in batches #3

Merged
merged 3 commits into from
Mar 18, 2023
Merged

Conversation

zmoog
Copy link
Owner

@zmoog zmoog commented Mar 18, 2023

Motivation

We want to send more than one event with a single command.

For example, given the multiline.txt file contains multiple lines:

$ cat multiline.txt
line 1
line 2
line 3

We want to send all of these lines to event hub in a single batch using one of the following options:

# reading the lines from a file:
$ eh eventdata send --lines-from-text-file multiline.txt
line 1
line 2
line 3

# or using the standard input:
$ cat multiline.txt | eh eventdata send
line 1
line 2
line 3

Change description

Switch from using send_event(event) to send_batch(batch) on EventHubProducerClient.

We currently support str-based inputs only, so no bytes content at the moment.

Additional Notes

Reviewer checklist

  • PR address a single concern.
  • PR title and description are appropriately filled.
  • Changes will be merged in main.
  • Changes are covered by tests.
  • Logging is meaningful in case of troubleshooting.
  • Docs are updated (at least the README.md, if needed).
  • History is clean, commit messages are meaningful (see CONTRIBUTING.md) and are well-formatted.

We now allow sending multiple events at once in addition to sending a
single one.

Users can specify multiple events using one of the following:

- the existing `--text` option multiple times
- the new `--lines-from-text-file` option
- the `stdin`

Using batches instead of single events makes the data transfer more
efficient.

refs: #2
@zmoog zmoog self-assigned this Mar 18, 2023
@zmoog zmoog added the enhancement New feature or request label Mar 18, 2023
@zmoog zmoog marked this pull request as ready for review March 18, 2023 08:17
@zmoog zmoog force-pushed the zmoog/send-event-using-batches branch from 1293f7b to 4232175 Compare March 18, 2023 08:26
@zmoog zmoog merged commit 353e07d into main Mar 18, 2023
@zmoog zmoog deleted the zmoog/send-event-using-batches branch March 18, 2023 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Send eventdata using batches
1 participant