Skip to content

ARROW-4244: [Format] Clarify padding/alignment rationale/recommendation.#3388

Closed
emkornfield wants to merge 4 commits into
apache:masterfrom
emkornfield:clarify_padding_spec
Closed

ARROW-4244: [Format] Clarify padding/alignment rationale/recommendation.#3388
emkornfield wants to merge 4 commits into
apache:masterfrom
emkornfield:clarify_padding_spec

Conversation

@emkornfield
Copy link
Copy Markdown
Contributor

I think this makes it the spec more consistent (but I might have misunderstood the intent).

The flow of text was a little bit strange.  I think this was
because of change from 64 bytes required padding/alignment
to a recommendation and using 8 byte padding/alignment as the
requirement.
Comment thread docs/source/format/Layout.rst Outdated
an aligned 8-byte offset. Additionally, each buffer should be padded to a multiple
of 8 bytes.
* The general recommendation is to align buffers to a 64-byte boundary and pad
to a multiple of 64 bytes, but this is not absolutely necessary.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we emphasis on highly recommended?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly, I couldn't find the thread that was the impetus to the change in requirement, so I'm not sure what the impetus was.
It looks like C++ and Rust implementations currently honor 64-bytes padding and alignment. C# has 64 byte alignment and 8 byte padding. I couldn't determine what the Java implementation was using (and I didn't look at javascript).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added bold and a note on performance reasons, and put the recommendation in bold. Let me know if you would like further changes.

@codecov-io
Copy link
Copy Markdown

codecov-io commented Jan 18, 2019

Codecov Report

Merging #3388 into master will increase coverage by 1.67%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #3388      +/-   ##
=========================================
+ Coverage   87.62%   89.3%   +1.67%     
=========================================
  Files         464     502      +38     
  Lines       54471   70343   +15872     
=========================================
+ Hits        47729   62817   +15088     
- Misses       6633    7526     +893     
+ Partials      109       0     -109
Impacted Files Coverage Δ
cpp/src/plasma/common.cc 93.33% <0%> (-0.15%) ⬇️
cpp/src/parquet/column_writer.cc 97.57% <0%> (-0.02%) ⬇️
cpp/src/plasma/common.h 100% <0%> (ø) ⬆️
go/arrow/array/table.go
go/arrow/math/uint64_amd64.go
go/arrow/internal/testing/tools/bool.go
go/arrow/internal/bitutil/bitutil.go
go/arrow/memory/memory_avx2_amd64.go
go/arrow/array/null.go
go/arrow/datatype_nested.go
... and 207 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a553b7...10c74f1. Read the comment docs.

Copy link
Copy Markdown
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Thank you, this makes things clearer / more consistent

@wesm wesm changed the title ARROW-4244: [DOCUMENTATION] Clarify padding/alignment rationale/recommendation. ARROW-4244: [Format] Clarify padding/alignment rationale/recommendation. Jan 23, 2019
@wesm wesm closed this in a226e88 Jan 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants