Skip Convert.ChangeType on the same-type object read path#338
Merged
Conversation
NumericVectorDataReader.GetValue(offset, targetType) backs the non-generic read surface (GetValue(ordinal), this[ordinal], GetValues). It always ran the boxed value through Convert.ChangeType, even when targetType already matched the column's natural type — the common case, since those APIs read using the column's ClrType. Convert.ChangeType does not early-out on same-type for IConvertible values: it dispatches through ToXxx() and re-boxes the result, so every value was boxed twice. Add a value.GetType() == targetType fast path that returns the already boxed value directly. Benchmark (read 1,000,000 rows x 3 numeric columns via GetValue, .NET 10, M4 Pro): Before: 54.50 ms, 137.3 MB allocated After: 22.73 ms, 68.7 MB allocated -58% time and -50% allocations on the object-returning path (the remaining allocation is the unavoidable single box of the object API). The typed GetFieldValue<T> path is unaffected. Behavior is unchanged: ChangeType returned the same value for same-type, and BigInteger (not IConvertible) already used this same early-out inside ChangeType.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #338 +/- ##
========================================
Coverage 87.58% 87.59%
========================================
Files 77 77
Lines 3134 3136 +2
Branches 463 464 +1
========================================
+ Hits 2745 2747 +2
Misses 275 275
Partials 114 114 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
Author
|
Just flagging that the macOS red X isn't from this change — it's the Coveralls upload step, which dies trying to Happy to send a small CI fix for it separately if that's helpful. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's Changed
NumericVectorDataReader.GetValue(offset, targetType)backs the non-generic readsurface —
GetValue(ordinal),this[ordinal],GetValues(object[]). It alwaysran the boxed value through
Convert.ChangeType, even whentargetTypealreadymatched the column's natural type. That is the common case, because those APIs
read using the column's
ClrType.Convert.ChangeTypedoes not early-out on same-type forIConvertiblevalues: it dispatches through
ToXxx()and re-boxes the result, so every valuewas boxed twice. This adds a
value.GetType() == targetTypefast path thatreturns the already-boxed value directly.
Benchmark
Reading 1,000,000 rows × 3 numeric columns (
INTEGER, BIGINT, DOUBLE) viaGetValue(ordinal). BenchmarkDotNet, .NET 10, Apple M4 Pro:GetValuepathThe typed
GetFieldValue<T>path is unaffected (~16 ms baseline). The remaining~69 MB is the unavoidable single box of the
object-returning API.Correctness
Behavior is unchanged: for same-type,
ChangeTypereturned the same valueanyway, and
BigInteger(which is notIConvertible) already hit this exactearly-out inside
ChangeType. Full existing suite passes locally (6,895 tests).Third in a small series of allocation-focused changes (#336, #337). Source-only;
the benchmark harness is the
DuckDB.NET.Benchmarksproject from #336.