Skip to content
This repository has been archived by the owner on Oct 12, 2023. It is now read-only.

Use a buffered writer to improve performance #358

Merged
merged 1 commit into from
Dec 15, 2020

Conversation

theunrepentantgeek
Copy link
Member

@theunrepentantgeek theunrepentantgeek commented Dec 15, 2020

Improve the performance of the export files stage by buffering file writes using buffio.

Rudimentary/naïve benchmark:

Unbuffered - 193.7s
Buffered - 123.1s

Saved time ~70s (~35%)

(Benchmark method: one run under each condition without isolating the effect of other processes on the machine.)

@coveralls
Copy link

coveralls commented Dec 15, 2020

Coverage Status

Coverage decreased (-1.7%) to 65.478% when pulling e0132b6 on improve/write-buffers into 7c7cd93 on master.

Copy link
Member

@babbageclunk babbageclunk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

@theunrepentantgeek theunrepentantgeek merged commit c3c8fde into master Dec 15, 2020
CRD Code Generation automation moved this from In progress to Done Dec 15, 2020
@theunrepentantgeek theunrepentantgeek deleted the improve/write-buffers branch December 15, 2020 05:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

None yet

3 participants