ARROW-11035: [Rust] Improved performance of casting to utf8 #9014
Conversation
jorgecarleitao
commented
Dec 26, 2020
•
jorgecarleitao
commented
Codecov Report
@@ Coverage Diff @@
## master #9014 +/- ##
==========================================
- Coverage 82.61% 82.61% -0.01%
==========================================
Files 202 202
Lines 50048 50027 -21
==========================================
- Hits 41347 41328 -19
+ Misses 8701 8699 -2
Continue to review full report at Codecov.
|
|
|
||
| Ok(b.finish()) | ||
| from.iter() | ||
| .map(|maybe_value| maybe_value.map(|value| value.to_string())) |
There was a problem hiding this comment.
Note, we probably can use lexical like here later #9010
| Ok(Arc::new( | ||
| array | ||
| .iter() | ||
| .map(|value| value.map(|value| if value { "1" } else { "0" })) |
There was a problem hiding this comment.
Does the speed up come here from not using the string builder or is using this iterator also faster?
It looks at least better, so if no difference this is better 👍
There was a problem hiding this comment.
Good question. I suspect the builder, because the iterator does the same thing as before atm (i.e. same bound checks).
The builders IMO are inefficient atm. Since IMO they are less idiomatic, I do not see any issue in replacing them whenever we can ^_^
github-actions
bot
added
the
needs-rebase
There was a problem hiding this comment.
The code looks cleaner and it is faster. Double win. Nice work @jorgecarleitao
|
The full set of Rust CI tests did not run on this PR :( Can you please rebase this PR against apache/master to pick up the changes in #9056 so that they do? I apologize for the inconvenience. |
jorgecarleitao
removed
the
needs-rebase
```
cast i64 to string 512 time: [92.618 us 92.839 us 93.097 us]
change: [-14.915% -14.287% -13.743%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
```
Closes apache#9014 from jorgecarleitao/speed_cast
Authored-by: Jorge C. Leitao <jorgecarleitao@gmail.com>
Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>
