Skip to content

Unroll the Zeek TSV header parsing loop #3291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Jul 4, 2023
Merged

Unroll the Zeek TSV header parsing loop #3291

merged 8 commits into from
Jul 4, 2023

Conversation

Dakostu
Copy link

@Dakostu Dakostu commented Jul 3, 2023

This change unrolls the header parsing loop in the zeek-tsv parser to allow early yielding during header parsing.

@Dakostu Dakostu added refactoring Restructuring of existing code bug Incorrect behavior format Parser and printer labels Jul 3, 2023
@Dakostu Dakostu requested a review from a team July 3, 2023 12:07
@Dakostu Dakostu force-pushed the topic/unroll-zeek-tsv-header-parsing-loop branch from f006d23 to 98a3fc1 Compare July 3, 2023 13:24
dominiklohmann
dominiklohmann reviewed Jul 3, 2023
View reviewed changes
dominiklohmann
dominiklohmann approved these changes Jul 3, 2023
View reviewed changes
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this so quickly.

I confirmed locally that this works as expected by making the file loader return chunks of size 16 rather than 16,384, which would guarantee that I run into the error all the time. I don't anymore, so this is good to go.

dominiklohmann
dominiklohmann reviewed Jul 3, 2023
View reviewed changes
dominiklohmann
dominiklohmann reviewed Jul 3, 2023
View reviewed changes
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to actually submit my local changes before as part of the approving review, these two must get in.

Daniel Kostuj and others added 4 commits July 4, 2023 08:55
@Dakostu Dakostu enabled auto-merge July 4, 2023 07:03
@Dakostu Dakostu merged commit 1db55ed into main Jul 4, 2023
@Dakostu Dakostu deleted the topic/unroll-zeek-tsv-header-parsing-loop branch July 4, 2023 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dominiklohmann dominiklohmann dominiklohmann approved these changes

Assignees
No one assigned
Labels
bug Incorrect behavior format Parser and printer refactoring Restructuring of existing code
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Dakostu @dominiklohmann