chore: Improve decimal benchmark coverage - #10954
Open
neilconway wants to merge 3 commits into
Open
Conversation
arrow/benches/csv_reader.rs exercised integer, float and string columns only, so the CSV decimal path (arrow-csv build_decimal_array -> arrow_cast::parse::parse_decimal) had no benchmark coverage. The new cases follow the pattern of the existing ones: build an array of the target type with random values, write it to CSV and read it back. create_decimal_array draws each value digit by digit so that it works for every decimal width, and takes the number of significant digits so that short and full-precision values can be benchmarked separately. Cases cover Decimal32/64/128/256 and a three-column batch. Shape-specific parser costs (values with fewer or more fractional digits than the scale) are left to arrow-cast/benches/parse_decimal.rs.
arrow-json/benches/json_reader.rs had no decimal cases, so the JSON decimal path (arrow-json DecimalArrayDecoder -> arrow_cast::parse::parse_decimal) had no benchmark coverage. The new case decodes newline-delimited objects with a single Decimal128 number field of seven integer and two fractional digits, following the existing build_*_json / bench_decode_* structure.
Every case in arrow-cast/benches/parse_decimal.rs parsed the same string on every iteration, so the branch predictor learned the input and data-dependent costs were invisible. In particular the digit that decides rounding is unpredictable for real data, and a branch on it costs about 20% on inputs with more fractional digits than the scale, which the "rounded scale" case did not show. Each case is now a shape rather than a literal: bench_parse generates 1024 strings with the shape's sign, decimal point, exponent and leading zeros but random digits, parses them all per iteration, and reports the throughput in elements. The case list is unchanged apart from a short Decimal128 shape matching typical CSV data, so each case still pins the syntactic path it did before.
Jefffrey
approved these changes
Sep 2, 2026
Rich-T-kid
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
decimaldid not have benchmark coverage for end-to-end CSV or JSON parsing; also, theparse_decimalmicrobenchmark had unrepresentative branch predictor behavior.What changes are included in this PR?
parse_decimalmicrobenchmark to generate a set of random strings to parse, rather than repeatedly parsing the same string. Repeatedly parsing the same string is not representative of real-world workloads; in particular, it gives the branch predictor an artificial boost, which can hide constructs that will poorly poorly in more realistic scenarios due to poor branch prediction.Are these changes tested?
Yes.
Are there any user-facing changes?
No.
AI usage
Developed with Claude Code, Fable 5.1. I revised and understand the resulting code.