-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CsvWriter performance optimization #2248
Conversation
- removed duplicate type resolving - optimized Type HashCodes - reduced memory allocation during writes
Eliminating all remaining |
I'm working on a new parser and serializer that uses spans and has 0 allocations, outside the initial buffer. I'm not sure how I'm going to fit it into the existing codebase yet. |
@JoshClose Are there any chances that this PR will be concidered / reviewd / merged and then published in near future? We would really like to fix a performance issue in our project that is fixed in this PR. |
I'll try and get to it soon. |
Does this same issue happen when reading? |
No, when reading it's much better, and It allocates only a few bytes per operation. |
This has been merged. I made other changes before you put in your second commit, that aren't in there, but I had done the same thing myself. Please look over everything and try it out. I will create a release once you do. |
@JoshClose I've tried the same benchmarks, and the results are almost the same, except with P.S. My |
Maybe not, but there is definitely less computations. Plus, it's consistent with the writer now. I'll push a new build soon. Just need to update the change log. |
I changed expando to use any I added a fast dynamic object that is created instead of |
This will be in the next release 32.0.0 on NuGet. |
I noticed that
CsvWriter
allocated huge amount of memory during serialization, much more than the size of the data, up to 3 orders of magnitude. It doesn't have memory leaks in my scenario, however such memory traffic createsGC
pressure and has noticeable performance impact on my application.There are some breaking changes to the public types though. But they can be easily fixed if needed. I'd also consider to make them internal in further versions to lower the probability of someone uses them directly.
So that's what is implemented here:
GetType()
is called too often.Type
HashCodes. No need to getAssemblyQualifiedName
to get it'sHashCode
, we can get it directly from theType
. However both approaches don't guarantee no collisions.Before
After