New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARROW-648: [C++] Support multiarch on Debian #398
Conversation
On multiarch enabled Debian, we need to install libraries into ${CMAKE_INSTALL_PREFIX}/lib/${ARCH}/ instead of ${CMAKE_INSTALL_PREFIX}/lib/.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM
Sorry... The original changes have a bug... I've added a commit that fixes the bug. |
@kou do you know how other projects handle this issue? We need to be able to install these libraries in a deterministic location -- when the user passes
This does not look right to me https://issues.apache.org/jira/browse/ARROW-680 |
Umm... I couldn't reproduce it. I'm using I'm on Debian GNU/Linux. |
From a clean out of source build, using gcc 4.9.2 not obtained from apt:
|
this is on Ubuntu 14.04 LTS |
also, in case it's relevant:
|
Looks like the problem is that my GNUInstallDirs.cmake is kind of old, and I have i386 enabled on my x86_64 machine, see the commit: I can install a newer cmake to solve this issue, but it would be unfortunate to have this failure mode out of the box on 14.04 LTS |
I've created a pull request as #419 for Ubuntu 14.04. |
…ed data, refactor parquet/arrow/reader We did not have very consistent logic around reading values from leaf nodes versus reading semantic records where the repetition level is greater than zero. This introduces a reader class that reads from column chunks until it identifies the end of records. It also reads values (with spaces, if required by the schema) into internal buffers. This permitted a substantial refactoring and simplification of the code in parquet::arrow where we were handling the interpretation of batch reads as records manually. As follow up patch, we should be able to take a collection of record readers from the same "tree" in a nested type and reassemble the intermediate Arrow structure and dealing with any redundant structure information in repetition and definition levels. This should a allow a unification of our nested data read code path so that we can read arbitrary nested structures. Author: Wes McKinney <wes.mckinney@twosigma.com> Closes apache#398 from wesm/PARQUET-1100 and squashes the following commits: 9ea85d9 [Wes McKinney] Revert to const args f4dc0fe [Wes McKinney] Make parquet::schema::Node non-copyable. Use const-refs instead of const-ptr for non-nullable argument 0d859cc [Wes McKinney] Code review comments, scrubbing some flakes 1368415 [Wes McKinney] Fix more MSVC warnings eccb84c [Wes McKinney] Give macro more accurate name 0eaada0 [Wes McKinney] Use int64_t instead of int for batch sizes 79c3709 [Wes McKinney] Add documentation. Remove RecordReader from public API 8fa619b [Wes McKinney] Initialize memory in DecodeSpaced to avoid undefined behavior 5a0c860 [Wes McKinney] Remove non-repeated branch from DelimitRecords c754e6e [Wes McKinney] Refactor to skip record delimiting for non-repeated data ed2a03f [Wes McKinney] Move more code into TypedRecordReader 2e934e9 [Wes McKinney] Set some integers as const 58d3a0f [Wes McKinney] Do not index into levels arrays b766371 [Wes McKinney] Add RecordReader::Reserve to preallocate, fixing perf regression. cpplint 1bf3e8f [Wes McKinney] Refactor to create stateful parquet::RecordReader class to better support nested data. Shift value buffering logic from parquet/arrow/reader into RecordReader. Fix bug described in PARQUET-1100
…ed data, refactor parquet/arrow/reader We did not have very consistent logic around reading values from leaf nodes versus reading semantic records where the repetition level is greater than zero. This introduces a reader class that reads from column chunks until it identifies the end of records. It also reads values (with spaces, if required by the schema) into internal buffers. This permitted a substantial refactoring and simplification of the code in parquet::arrow where we were handling the interpretation of batch reads as records manually. As follow up patch, we should be able to take a collection of record readers from the same "tree" in a nested type and reassemble the intermediate Arrow structure and dealing with any redundant structure information in repetition and definition levels. This should a allow a unification of our nested data read code path so that we can read arbitrary nested structures. Author: Wes McKinney <wes.mckinney@twosigma.com> Closes apache#398 from wesm/PARQUET-1100 and squashes the following commits: 9ea85d9 [Wes McKinney] Revert to const args f4dc0fe [Wes McKinney] Make parquet::schema::Node non-copyable. Use const-refs instead of const-ptr for non-nullable argument 0d859cc [Wes McKinney] Code review comments, scrubbing some flakes 1368415 [Wes McKinney] Fix more MSVC warnings eccb84c [Wes McKinney] Give macro more accurate name 0eaada0 [Wes McKinney] Use int64_t instead of int for batch sizes 79c3709 [Wes McKinney] Add documentation. Remove RecordReader from public API 8fa619b [Wes McKinney] Initialize memory in DecodeSpaced to avoid undefined behavior 5a0c860 [Wes McKinney] Remove non-repeated branch from DelimitRecords c754e6e [Wes McKinney] Refactor to skip record delimiting for non-repeated data ed2a03f [Wes McKinney] Move more code into TypedRecordReader 2e934e9 [Wes McKinney] Set some integers as const 58d3a0f [Wes McKinney] Do not index into levels arrays b766371 [Wes McKinney] Add RecordReader::Reserve to preallocate, fixing perf regression. cpplint 1bf3e8f [Wes McKinney] Refactor to create stateful parquet::RecordReader class to better support nested data. Shift value buffering logic from parquet/arrow/reader into RecordReader. Fix bug described in PARQUET-1100 Change-Id: I94c1dd3f7936e30036e9c42eef689e86fb5fe7fa
…ed data, refactor parquet/arrow/reader We did not have very consistent logic around reading values from leaf nodes versus reading semantic records where the repetition level is greater than zero. This introduces a reader class that reads from column chunks until it identifies the end of records. It also reads values (with spaces, if required by the schema) into internal buffers. This permitted a substantial refactoring and simplification of the code in parquet::arrow where we were handling the interpretation of batch reads as records manually. As follow up patch, we should be able to take a collection of record readers from the same "tree" in a nested type and reassemble the intermediate Arrow structure and dealing with any redundant structure information in repetition and definition levels. This should a allow a unification of our nested data read code path so that we can read arbitrary nested structures. Author: Wes McKinney <wes.mckinney@twosigma.com> Closes apache#398 from wesm/PARQUET-1100 and squashes the following commits: 9ea85d9 [Wes McKinney] Revert to const args f4dc0fe [Wes McKinney] Make parquet::schema::Node non-copyable. Use const-refs instead of const-ptr for non-nullable argument 0d859cc [Wes McKinney] Code review comments, scrubbing some flakes 1368415 [Wes McKinney] Fix more MSVC warnings eccb84c [Wes McKinney] Give macro more accurate name 0eaada0 [Wes McKinney] Use int64_t instead of int for batch sizes 79c3709 [Wes McKinney] Add documentation. Remove RecordReader from public API 8fa619b [Wes McKinney] Initialize memory in DecodeSpaced to avoid undefined behavior 5a0c860 [Wes McKinney] Remove non-repeated branch from DelimitRecords c754e6e [Wes McKinney] Refactor to skip record delimiting for non-repeated data ed2a03f [Wes McKinney] Move more code into TypedRecordReader 2e934e9 [Wes McKinney] Set some integers as const 58d3a0f [Wes McKinney] Do not index into levels arrays b766371 [Wes McKinney] Add RecordReader::Reserve to preallocate, fixing perf regression. cpplint 1bf3e8f [Wes McKinney] Refactor to create stateful parquet::RecordReader class to better support nested data. Shift value buffering logic from parquet/arrow/reader into RecordReader. Fix bug described in PARQUET-1100 Change-Id: I94c1dd3f7936e30036e9c42eef689e86fb5fe7fa
…ed data, refactor parquet/arrow/reader We did not have very consistent logic around reading values from leaf nodes versus reading semantic records where the repetition level is greater than zero. This introduces a reader class that reads from column chunks until it identifies the end of records. It also reads values (with spaces, if required by the schema) into internal buffers. This permitted a substantial refactoring and simplification of the code in parquet::arrow where we were handling the interpretation of batch reads as records manually. As follow up patch, we should be able to take a collection of record readers from the same "tree" in a nested type and reassemble the intermediate Arrow structure and dealing with any redundant structure information in repetition and definition levels. This should a allow a unification of our nested data read code path so that we can read arbitrary nested structures. Author: Wes McKinney <wes.mckinney@twosigma.com> Closes apache#398 from wesm/PARQUET-1100 and squashes the following commits: 9ea85d9 [Wes McKinney] Revert to const args f4dc0fe [Wes McKinney] Make parquet::schema::Node non-copyable. Use const-refs instead of const-ptr for non-nullable argument 0d859cc [Wes McKinney] Code review comments, scrubbing some flakes 1368415 [Wes McKinney] Fix more MSVC warnings eccb84c [Wes McKinney] Give macro more accurate name 0eaada0 [Wes McKinney] Use int64_t instead of int for batch sizes 79c3709 [Wes McKinney] Add documentation. Remove RecordReader from public API 8fa619b [Wes McKinney] Initialize memory in DecodeSpaced to avoid undefined behavior 5a0c860 [Wes McKinney] Remove non-repeated branch from DelimitRecords c754e6e [Wes McKinney] Refactor to skip record delimiting for non-repeated data ed2a03f [Wes McKinney] Move more code into TypedRecordReader 2e934e9 [Wes McKinney] Set some integers as const 58d3a0f [Wes McKinney] Do not index into levels arrays b766371 [Wes McKinney] Add RecordReader::Reserve to preallocate, fixing perf regression. cpplint 1bf3e8f [Wes McKinney] Refactor to create stateful parquet::RecordReader class to better support nested data. Shift value buffering logic from parquet/arrow/reader into RecordReader. Fix bug described in PARQUET-1100 Change-Id: I94c1dd3f7936e30036e9c42eef689e86fb5fe7fa
…ed data, refactor parquet/arrow/reader We did not have very consistent logic around reading values from leaf nodes versus reading semantic records where the repetition level is greater than zero. This introduces a reader class that reads from column chunks until it identifies the end of records. It also reads values (with spaces, if required by the schema) into internal buffers. This permitted a substantial refactoring and simplification of the code in parquet::arrow where we were handling the interpretation of batch reads as records manually. As follow up patch, we should be able to take a collection of record readers from the same "tree" in a nested type and reassemble the intermediate Arrow structure and dealing with any redundant structure information in repetition and definition levels. This should a allow a unification of our nested data read code path so that we can read arbitrary nested structures. Author: Wes McKinney <wes.mckinney@twosigma.com> Closes apache#398 from wesm/PARQUET-1100 and squashes the following commits: 9ea85d9 [Wes McKinney] Revert to const args f4dc0fe [Wes McKinney] Make parquet::schema::Node non-copyable. Use const-refs instead of const-ptr for non-nullable argument 0d859cc [Wes McKinney] Code review comments, scrubbing some flakes 1368415 [Wes McKinney] Fix more MSVC warnings eccb84c [Wes McKinney] Give macro more accurate name 0eaada0 [Wes McKinney] Use int64_t instead of int for batch sizes 79c3709 [Wes McKinney] Add documentation. Remove RecordReader from public API 8fa619b [Wes McKinney] Initialize memory in DecodeSpaced to avoid undefined behavior 5a0c860 [Wes McKinney] Remove non-repeated branch from DelimitRecords c754e6e [Wes McKinney] Refactor to skip record delimiting for non-repeated data ed2a03f [Wes McKinney] Move more code into TypedRecordReader 2e934e9 [Wes McKinney] Set some integers as const 58d3a0f [Wes McKinney] Do not index into levels arrays b766371 [Wes McKinney] Add RecordReader::Reserve to preallocate, fixing perf regression. cpplint 1bf3e8f [Wes McKinney] Refactor to create stateful parquet::RecordReader class to better support nested data. Shift value buffering logic from parquet/arrow/reader into RecordReader. Fix bug described in PARQUET-1100 Change-Id: I94c1dd3f7936e30036e9c42eef689e86fb5fe7fa
On multiarch enabled Debian, we need to install libraries into
${CMAKE_INSTALL_PREFIX}/lib/${ARCH}/ instead of
${CMAKE_INSTALL_PREFIX}/lib/.
See also: https://wiki.debian.org/Multiarch/HOWTO