Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Decimal support to pretty-print display utility (#230) #273

Merged
merged 4 commits into from
May 14, 2021

Conversation

mgill25
Copy link
Contributor

@mgill25 mgill25 commented May 9, 2021

Which issue does this PR close?

Closes #230 .

What changes are included in this PR?

Added support for Decimal column datatype during conversion of value to String. Removes the previous error.

@alamb I made a tiny change and added a couple of tests (copied similar test data from other tests I found) within the same file just to ensure things are working correctly.

I wasn't sure what the definition of "pretty" in the pretty-printing context here would be, so I made no assumptions and just propagated whatever was being printed by default. Still just in the learning mode :) If more improvements are needed, please do let me know!

@codecov-commenter
Copy link

Codecov Report

Merging #273 (a23138f) into master (4dfbca6) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #273      +/-   ##
==========================================
+ Coverage   82.52%   82.54%   +0.02%     
==========================================
  Files         162      162              
  Lines       43672    43819     +147     
==========================================
+ Hits        36040    36172     +132     
- Misses       7632     7647      +15     
Impacted Files Coverage Δ
arrow/src/util/display.rs 47.36% <100.00%> (+16.81%) ⬆️
arrow/src/array/transform/fixed_binary.rs 78.94% <0.00%> (-5.27%) ⬇️
parquet/src/arrow/levels.rs 81.25% <0.00%> (-0.44%) ⬇️
parquet/src/column/writer.rs 93.76% <0.00%> (-0.27%) ⬇️
arrow/src/compute/kernels/regexp.rs 97.56% <0.00%> (-0.03%) ⬇️
parquet/src/arrow/schema.rs 88.93% <0.00%> (-0.02%) ⬇️
arrow/src/json/reader.rs 83.43% <0.00%> (-0.02%) ⬇️
arrow/src/array/data.rs 72.07% <0.00%> (ø)
arrow/src/array/null.rs 86.66% <0.00%> (ø)
arrow/src/array/array.rs 76.07% <0.00%> (ø)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4dfbca6...a23138f. Read the comment docs.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mgill25 this is a great start.

FYI here is some more information about what the different fields on DecimalArray mean: https://github.com/apache/arrow/blob/master/format/Schema.fbs#L180-L190

FYI @ovr as I think you had been working on a more full featured DecimalArray

I think the output should show the decimal point . being added, which I am not sure it does quite yet

arrow/src/util/display.rs Outdated Show resolved Hide resolved
arrow/src/util/display.rs Outdated Show resolved Hide resolved
arrow/src/util/display.rs Outdated Show resolved Hide resolved
arrow/src/util/display.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me @mgill25 . Thank you!

@jorgecarleitao or @paddyhoran can someone double check that this looks good?

Copy link
Contributor

@paddyhoran paddyhoran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @mgill25. Just one minor question.

@@ -217,6 +231,9 @@ pub fn array_value_to_string(column: &array::ArrayRef, row: usize) -> Result<Str
DataType::Float16 => make_string!(array::Float32Array, column, row),
DataType::Float32 => make_string!(array::Float32Array, column, row),
DataType::Float64 => make_string!(array::Float64Array, column, row),
DataType::Decimal(_, scale) => {
make_string_from_decimal!(array::DecimalArray, column, row, scale)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be a macro? I think you can inline this as it's the only usage, right?

Copy link
Contributor

@alamb alamb May 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it can be a function (rather than a macro) but that can always be done as a follow on PR. I think this PR is a good step forward. 👍

@alamb alamb merged commit 4449ee9 into apache:master May 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for pretty-printing Decimal numbers
5 participants