Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libawkward export tuning #316

Merged
merged 3 commits into from
Aug 19, 2020
Merged

libawkward export tuning #316

merged 3 commits into from
Aug 19, 2020

Conversation

veprbl
Copy link
Contributor

@veprbl veprbl commented Jul 3, 2020

Would like to test these two changes.

@jpivarski
Copy link
Member

Hi! Thanks for looking into this again. Since I work on Linux, I've been seeing the warnings for a while now, thinking I should look into it and figure out how to fix it. Maybe that's still true because the warnings are still present in the Linux build (below is from Azure, the Linux + Python 3.8 build). I also tried compiling it on my computer, and see the same warnings.

2020-07-03T08:21:31.1652947Z   -- The CXX compiler identification is GNU 5.4.0
2020-07-03T08:21:31.1724229Z   -- Check for working CXX compiler: /usr/bin/g++
2020-07-03T08:21:31.2408078Z   -- Check for working CXX compiler: /usr/bin/g++ - works
2020-07-03T08:21:31.2419326Z   -- Detecting CXX compiler ABI info
2020-07-03T08:21:31.3086722Z   -- Detecting CXX compiler ABI info - done
2020-07-03T08:21:31.3288860Z   -- Detecting CXX compile features
2020-07-03T08:21:31.3299107Z   -- Detecting CXX compile features - done
2020-07-03T08:21:31.3309768Z   -- CMake version 3.17.3
2020-07-03T08:21:31.3310419Z   -- Release
2020-07-03T08:21:31.3544007Z   -- Found PythonInterp: /opt/hostedtoolcache/Python/3.8.3/x64/bin/python (found version "3.8.3")
2020-07-03T08:21:31.3915784Z   -- Found PythonLibs: /opt/hostedtoolcache/Python/3.8.3/x64/lib/libpython3.8.so
2020-07-03T08:21:31.3997094Z   -- pybind11 v2.4.3
2020-07-03T08:21:31.4071238Z   -- Performing Test HAS_FLTO
2020-07-03T08:21:33.4117854Z   -- Performing Test HAS_FLTO - Success
2020-07-03T08:21:33.4118841Z   -- LTO enabled
2020-07-03T08:21:33.4136840Z   -- Configuring done
2020-07-03T08:21:33.4460654Z   -- Generating done
2020-07-03T08:21:33.4475968Z   -- Build files have been written to: /tmp/pip-req-build-5pd9zbqc/build/temp.linux-x86_64-3.8
2020-07-03T08:21:33.5158355Z   Scanning dependencies of target awkward-cpu-kernels-objects
2020-07-03T08:21:33.5194153Z   [  1%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/getitem.cpp.o
2020-07-03T08:21:33.5588086Z   Scanning dependencies of target awkward-objects
2020-07-03T08:21:33.5684033Z   [  2%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Content.cpp.o
2020-07-03T08:21:35.1728746Z   [  4%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/identities.cpp.o
2020-07-03T08:21:36.1183319Z   [  5%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/operations.cpp.o
2020-07-03T08:21:38.1010411Z   [  7%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/reducers.cpp.o
2020-07-03T08:21:39.4957864Z   [  8%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Identities.cpp.o
2020-07-03T08:21:39.5703725Z   [  9%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/sorting.cpp.o
2020-07-03T08:21:39.9355493Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/Identities.cpp:334:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:39.9357268Z      template class EXPORT_SYMBOL IdentitiesOf<int64_t>;
2020-07-03T08:21:39.9357994Z                                   ^
2020-07-03T08:21:40.9654520Z   [ 11%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Index.cpp.o
2020-07-03T08:21:41.3524168Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/Index.cpp:310:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:41.3526811Z      template class EXPORT_SYMBOL IndexOf<int8_t>;
2020-07-03T08:21:41.3527716Z                                   ^
2020-07-03T08:21:41.3538329Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/Index.cpp:311:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:41.3540096Z      template class EXPORT_SYMBOL IndexOf<uint8_t>;
2020-07-03T08:21:41.3540874Z                                   ^
2020-07-03T08:21:41.3549097Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/Index.cpp:312:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:41.3550069Z      template class EXPORT_SYMBOL IndexOf<int32_t>;
2020-07-03T08:21:41.3550429Z                                   ^
2020-07-03T08:21:41.3558440Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/Index.cpp:313:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:41.3559305Z      template class EXPORT_SYMBOL IndexOf<uint32_t>;
2020-07-03T08:21:41.3559910Z                                   ^
2020-07-03T08:21:41.3566180Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/Index.cpp:314:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:41.3567098Z      template class EXPORT_SYMBOL IndexOf<int64_t>;
2020-07-03T08:21:41.3567567Z                                   ^
2020-07-03T08:21:42.5986890Z   [ 12%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Iterator.cpp.o
2020-07-03T08:21:43.1330863Z   [ 14%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Reducer.cpp.o
2020-07-03T08:21:45.5040215Z   [ 15%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Slice.cpp.o
2020-07-03T08:21:48.1318014Z   [ 16%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/BitMaskedArray.cpp.o
2020-07-03T08:21:50.0815141Z   [ 18%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/ByteMaskedArray.cpp.o
2020-07-03T08:21:52.1530494Z   [ 18%] Built target awkward-cpu-kernels-objects
2020-07-03T08:21:52.1557889Z   [ 19%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/EmptyArray.cpp.o
2020-07-03T08:21:52.3740618Z   [ 21%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/IndexedArray.cpp.o
2020-07-03T08:21:52.8726228Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/IndexedArray.cpp:2343:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:52.8727771Z      template class EXPORT_SYMBOL IndexedArrayOf<int64_t, true>;
2020-07-03T08:21:52.8728501Z                                   ^
2020-07-03T08:21:52.8733472Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/IndexedArray.cpp:2345:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:52.8735042Z      template class EXPORT_SYMBOL IndexedArrayOf<int32_t, false>;
2020-07-03T08:21:52.8735671Z                                   ^
2020-07-03T08:21:52.8741512Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/IndexedArray.cpp:2346:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:52.8742725Z      template class EXPORT_SYMBOL IndexedArrayOf<uint32_t, false>;
2020-07-03T08:21:52.8743480Z                                   ^
2020-07-03T08:21:52.8752177Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/IndexedArray.cpp:2347:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:52.8753361Z      template class EXPORT_SYMBOL IndexedArrayOf<int64_t, false>;
2020-07-03T08:21:52.8754052Z                                   ^
2020-07-03T08:21:52.8762495Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/IndexedArray.cpp:2348:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:52.8763626Z      template class EXPORT_SYMBOL IndexedArrayOf<int32_t, true>;
2020-07-03T08:21:52.8764337Z                                   ^
2020-07-03T08:21:53.7145523Z   Scanning dependencies of target awkward-cpu-kernels-static
2020-07-03T08:21:53.7215045Z   [ 22%] Linking CXX static library libawkward-cpu-kernels-static.a
2020-07-03T08:21:53.7430320Z   [ 22%] Built target awkward-cpu-kernels-static
2020-07-03T08:21:53.7476553Z   Scanning dependencies of target awkward-cpu-kernels
2020-07-03T08:21:53.7519323Z   [ 23%] Linking CXX shared library libawkward-cpu-kernels.so
2020-07-03T08:21:53.8496698Z   [ 23%] Built target awkward-cpu-kernels
2020-07-03T08:21:53.8521192Z   [ 25%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/ListArray.cpp.o
2020-07-03T08:21:54.3415440Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/ListArray.cpp:1838:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:54.3417082Z      template class EXPORT_SYMBOL ListArrayOf<int32_t>;
2020-07-03T08:21:54.3417823Z                                   ^
2020-07-03T08:21:54.3421383Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/ListArray.cpp:1839:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:54.3422609Z      template class EXPORT_SYMBOL ListArrayOf<uint32_t>;
2020-07-03T08:21:54.3423288Z                                   ^
2020-07-03T08:21:54.3427824Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/ListArray.cpp:1840:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:21:54.3428895Z      template class EXPORT_SYMBOL ListArrayOf<int64_t>;
2020-07-03T08:21:54.3430126Z                                   ^
2020-07-03T08:22:00.2964004Z   [ 26%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/ListOffsetArray.cpp.o
2020-07-03T08:22:01.1959129Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/ListOffsetArray.cpp:2238:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:01.1960776Z      template class EXPORT_SYMBOL ListOffsetArrayOf<int32_t>;
2020-07-03T08:22:01.1961317Z                                   ^
2020-07-03T08:22:01.1970443Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/ListOffsetArray.cpp:2239:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:01.1971111Z      template class EXPORT_SYMBOL ListOffsetArrayOf<uint32_t>;
2020-07-03T08:22:01.1971460Z                                   ^
2020-07-03T08:22:01.1979734Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/ListOffsetArray.cpp:2240:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:01.1980605Z      template class EXPORT_SYMBOL ListOffsetArrayOf<int64_t>;
2020-07-03T08:22:01.1981392Z                                   ^
2020-07-03T08:22:02.0206316Z   [ 28%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/None.cpp.o
2020-07-03T08:22:02.7726291Z   [ 29%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/NumpyArray.cpp.o
2020-07-03T08:22:07.3269262Z   [ 30%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/Record.cpp.o
2020-07-03T08:22:08.8669878Z   [ 32%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/RecordArray.cpp.o
2020-07-03T08:22:10.5547958Z   [ 33%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/RegularArray.cpp.o
2020-07-03T08:22:12.7854226Z   [ 35%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/UnionArray.cpp.o
2020-07-03T08:22:13.0500683Z   [ 36%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/UnmaskedArray.cpp.o
2020-07-03T08:22:13.2559503Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/UnionArray.cpp:1951:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:13.2561126Z      template class EXPORT_SYMBOL UnionArrayOf<int8_t, int32_t>;
2020-07-03T08:22:13.2561790Z                                   ^
2020-07-03T08:22:13.2566778Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/UnionArray.cpp:1952:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:13.2567996Z      template class EXPORT_SYMBOL UnionArrayOf<int8_t, uint32_t>;
2020-07-03T08:22:13.2568588Z                                   ^
2020-07-03T08:22:13.2574003Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/array/UnionArray.cpp:1953:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:13.2574947Z      template class EXPORT_SYMBOL UnionArrayOf<int8_t, int64_t>;
2020-07-03T08:22:13.2575482Z                                   ^
2020-07-03T08:22:15.1107321Z   [ 38%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/VirtualArray.cpp.o
2020-07-03T08:22:17.0628329Z   [ 39%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/ArrayBuilder.cpp.o
2020-07-03T08:22:18.0753468Z   [ 40%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/ArrayBuilderOptions.cpp.o
2020-07-03T08:22:18.3200918Z   [ 42%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/BoolBuilder.cpp.o
2020-07-03T08:22:19.0651391Z   [ 43%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/Builder.cpp.o
2020-07-03T08:22:19.4772321Z   [ 45%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/Float64Builder.cpp.o
2020-07-03T08:22:19.7852527Z   [ 46%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/GrowableBuffer.cpp.o
2020-07-03T08:22:20.2426660Z   [ 47%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/IndexedBuilder.cpp.o
2020-07-03T08:22:20.6340355Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/builder/IndexedBuilder.cpp:151:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:20.6341886Z      template class EXPORT_SYMBOL IndexedBuilder<Content>;
2020-07-03T08:22:20.6342603Z                                   ^
2020-07-03T08:22:20.6639559Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/builder/IndexedBuilder.cpp:257:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:20.6640786Z      template class EXPORT_SYMBOL IndexedBuilder<IndexedArray32>;
2020-07-03T08:22:20.6641421Z                                   ^
2020-07-03T08:22:20.6658377Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/builder/IndexedBuilder.cpp:305:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:20.6664875Z      template class EXPORT_SYMBOL IndexedBuilder<IndexedArrayU32>;
2020-07-03T08:22:20.6665578Z                                   ^
2020-07-03T08:22:20.6683028Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/builder/IndexedBuilder.cpp:353:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:20.6684556Z      template class EXPORT_SYMBOL IndexedBuilder<IndexedArray64>;
2020-07-03T08:22:20.6685198Z                                   ^
2020-07-03T08:22:20.6703463Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/builder/IndexedBuilder.cpp:401:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:20.6704540Z      template class EXPORT_SYMBOL IndexedBuilder<IndexedOptionArray32>;
2020-07-03T08:22:20.6705217Z                                   ^
2020-07-03T08:22:20.6721330Z   /tmp/pip-req-build-5pd9zbqc/src/libawkward/builder/IndexedBuilder.cpp:443:32: warning: type attributes ignored after type is already defined [-Wattributes]
2020-07-03T08:22:20.6722458Z      template class EXPORT_SYMBOL IndexedBuilder<IndexedOptionArray64>;
2020-07-03T08:22:20.6723142Z                                   ^
2020-07-03T08:22:20.7522929Z   [ 49%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/Int64Builder.cpp.o
2020-07-03T08:22:21.4767574Z   [ 50%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/ListBuilder.cpp.o
2020-07-03T08:22:22.1767590Z   [ 52%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/OptionBuilder.cpp.o
2020-07-03T08:22:22.3336520Z   [ 53%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/RecordBuilder.cpp.o
2020-07-03T08:22:23.0794748Z   [ 54%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/StringBuilder.cpp.o
2020-07-03T08:22:23.9560825Z   [ 56%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/TupleBuilder.cpp.o
2020-07-03T08:22:24.1407071Z   [ 57%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/UnionBuilder.cpp.o
2020-07-03T08:22:25.1869663Z   [ 59%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/UnknownBuilder.cpp.o
2020-07-03T08:22:25.6792238Z   [ 60%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/io/json.cpp.o
2020-07-03T08:22:26.0103487Z   [ 61%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/io/root.cpp.o
2020-07-03T08:22:26.8226578Z   [ 63%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/kernel.cpp.o
2020-07-03T08:22:28.1675153Z   [ 64%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/partition/IrregularlyPartitionedArray.cpp.o
2020-07-03T08:22:28.1999245Z   [ 66%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/partition/PartitionedArray.cpp.o
2020-07-03T08:22:29.0644783Z   [ 67%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/ArrayType.cpp.o
2020-07-03T08:22:29.1566643Z   [ 69%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/ListType.cpp.o
2020-07-03T08:22:29.6688337Z   [ 70%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/OptionType.cpp.o
2020-07-03T08:22:29.8892538Z   [ 71%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/PrimitiveType.cpp.o
2020-07-03T08:22:30.4484092Z   [ 73%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/RecordType.cpp.o
2020-07-03T08:22:30.6788134Z   [ 74%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/RegularType.cpp.o
2020-07-03T08:22:31.3991430Z   [ 76%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/Type.cpp.o
2020-07-03T08:22:32.1843294Z   [ 77%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/UnionType.cpp.o
2020-07-03T08:22:32.2190904Z   [ 78%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/UnknownType.cpp.o
2020-07-03T08:22:32.8619623Z   [ 80%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/util.cpp.o
2020-07-03T08:22:33.0534420Z   [ 81%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/virtual/ArrayCache.cpp.o
2020-07-03T08:22:33.5305867Z   [ 83%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/virtual/ArrayGenerator.cpp.o
2020-07-03T08:22:35.2345315Z   [ 83%] Built target awkward-objects
2020-07-03T08:22:35.2391601Z   Scanning dependencies of target awkward
2020-07-03T08:22:35.2400573Z   Scanning dependencies of target awkward-static
2020-07-03T08:22:35.2443795Z   [ 84%] Linking CXX shared library libawkward.so
2020-07-03T08:22:35.2450753Z   [ 85%] Linking CXX static library libawkward-static.a
2020-07-03T08:22:35.4142619Z   [ 85%] Built target awkward-static
2020-07-03T08:22:36.2452515Z   [ 85%] Built target awkward
2020-07-03T08:22:36.2453210Z   Scanning dependencies of target _ext
2020-07-03T08:22:36.2453728Z   [ 87%] Building CXX object CMakeFiles/_ext.dir/src/python/_ext.cpp.o
2020-07-03T08:22:36.2454246Z   [ 88%] Building CXX object CMakeFiles/_ext.dir/src/python/content.cpp.o
2020-07-03T08:22:39.6684871Z   [ 90%] Building CXX object CMakeFiles/_ext.dir/src/python/forms.cpp.o
2020-07-03T08:22:50.1727161Z   [ 91%] Building CXX object CMakeFiles/_ext.dir/src/python/identities.cpp.o
2020-07-03T08:22:54.9663911Z   [ 92%] Building CXX object CMakeFiles/_ext.dir/src/python/index.cpp.o
2020-07-03T08:22:59.9279459Z   [ 94%] Building CXX object CMakeFiles/_ext.dir/src/python/io.cpp.o
2020-07-03T08:23:02.5410636Z   [ 95%] Building CXX object CMakeFiles/_ext.dir/src/python/partition.cpp.o
2020-07-03T08:23:06.7429349Z   [ 97%] Building CXX object CMakeFiles/_ext.dir/src/python/types.cpp.o
2020-07-03T08:23:16.2611171Z   [ 98%] Building CXX object CMakeFiles/_ext.dir/src/python/virtual.cpp.o
2020-07-03T08:23:26.1310310Z   [100%] Linking CXX shared module _ext.cpython-38-x86_64-linux-gnu.so
2020-07-03T08:24:28.2186095Z   [100%] Built target _ext
2020-07-03T08:24:28.2998750Z   [  7%] Built target awkward-cpu-kernels-objects
2020-07-03T08:24:28.3066723Z   [  8%] Built target awkward-cpu-kernels-static
2020-07-03T08:24:28.3134632Z   [  9%] Built target awkward-cpu-kernels
2020-07-03T08:24:28.3340981Z   [ 83%] Built target awkward-objects
2020-07-03T08:24:28.3417603Z   [ 84%] Built target awkward-static
2020-07-03T08:24:28.3590699Z   [ 98%] Built target _ext
2020-07-03T08:24:28.3666552Z   [100%] Built target awkward

@veprbl
Copy link
Contributor Author

veprbl commented Jul 3, 2020

@jpivarski Yes, there is a third change coming to fix those.

@jpivarski
Copy link
Member

Hi, @veprbl! I'm checking in on your PR because it has been a long time. Is this something you intend to finish? If not, then I'll close it.

Meanwhile, I'm going to have to do something about the warnings when building on Linux. Doing so may affect the downstream linking on MacOS that you got working. Do you have a minimal example that I'd be able to include as a test, so that when I fix the compile-time warnings on Linux I don't break downstream linking on MacOS? Would the dependent project, as it is currently defined, be an appropriate test? If so, I'll include this or whatever test you give me into CI so that I don't break things for you. Since I don't have a Mac, CI is the only way I have to know that I'm not breaking your use-case.

Thanks!

@veprbl
Copy link
Contributor Author

veprbl commented Aug 19, 2020 •
edited
Loading

@jpivarski There was a delay, but I'm back at it now. This was already ready, all that was missing was some extra explanations as for what I'm doing and why. I will rebase and update this now.

@jpivarski
Copy link
Member

@jpivarski There was a delay, but I'm back at it now. This was already ready, all that was missing was some extra explanations as for what I'm doing and why. I will rebase and update this now.

In that case, now is a better time than an hour ago—I just put in a big change. I think that nothing has touched the linking process, and maybe even CMakeLists.txt is unchanged since you were last working on this, but there have been some substantial rearrangements of the code. (Moved header files and several search-and-replacements, but I don't think the lines you changed—the first line of each class declaration—have changed.)

@veprbl
Copy link
Contributor Author

veprbl commented Aug 19, 2020

@jpivarski I see. I will look into the next rebase tomorrow.

@veprbl
libawkward: avoid marking extern template classes as dllexport
aa16177 
MRE https://godbolt.org/z/imCwCW :

template <typename T>
class V c {};

extern template class c<int>;

gives

warning C4910: 'c<int>': '__declspec(dllexport)' and 'extern' are incompatible on an explicit instantiation
@veprbl
libawkward: introduce LIBAWKWARD_EXPORT_SYMBOL
3e372c3 
The consumers of libawkward's headers are not supposed to export the
symbols of libawkward. Those should be either instantiated privately
or imported from libawkward.
@veprbl
libawkward: Silence a gcc warning: type attributes ignored after type…
f9aeec7 
… is already defined
@jpivarski
Copy link
Member

No warnings in Linux; that's great!

2020-08-19T19:14:45.1054802Z   -- The CXX compiler identification is GNU 5.5.0
2020-08-19T19:14:45.1187165Z   -- Detecting CXX compiler ABI info
2020-08-19T19:14:45.1919706Z   -- Detecting CXX compiler ABI info - done
2020-08-19T19:14:45.2074959Z   -- Check for working CXX compiler: /usr/bin/g++ - skipped
2020-08-19T19:14:45.2081196Z   -- Detecting CXX compile features
2020-08-19T19:14:45.2088946Z   -- Detecting CXX compile features - done
2020-08-19T19:14:45.2098324Z   -- CMake version 3.18.0
2020-08-19T19:14:45.2099660Z   -- Release
2020-08-19T19:14:45.2396353Z   -- Found PythonInterp: /opt/hostedtoolcache/Python/3.8.5/x64/bin/python (found version "3.8.5")
2020-08-19T19:14:45.2791750Z   -- Found PythonLibs: /opt/hostedtoolcache/Python/3.8.5/x64/lib/libpython3.8.so
2020-08-19T19:14:45.2880831Z   -- pybind11 v2.4.3
2020-08-19T19:14:45.2956758Z   -- Performing Test HAS_FLTO
2020-08-19T19:14:46.8389906Z   -- Performing Test HAS_FLTO - Success
2020-08-19T19:14:46.8391963Z   -- LTO enabled
2020-08-19T19:14:46.8410807Z   -- Configuring done
2020-08-19T19:14:46.8711309Z   -- Generating done
2020-08-19T19:14:46.8723205Z   -- Build files have been written to: /tmp/pip-req-build-aoz21nbp/build/temp.linux-x86_64-3.8
2020-08-19T19:14:46.9380571Z   Scanning dependencies of target awkward-cpu-kernels-objects
2020-08-19T19:14:46.9420221Z   [  1%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/allocators.cpp.o
2020-08-19T19:14:46.9728551Z   Scanning dependencies of target awkward-objects
2020-08-19T19:14:46.9815510Z   [  2%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Content.cpp.o
2020-08-19T19:14:47.6928207Z   [  4%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/getitem.cpp.o
2020-08-19T19:14:49.5412868Z   [  5%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/identities.cpp.o
2020-08-19T19:14:50.8293456Z   [  6%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/operations.cpp.o
2020-08-19T19:14:53.4226719Z   [  8%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Identities.cpp.o
2020-08-19T19:14:54.4722199Z   [  9%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/reducers.cpp.o
2020-08-19T19:14:55.3011364Z   [ 10%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Index.cpp.o
2020-08-19T19:14:56.2080454Z   [ 12%] Building CXX object CMakeFiles/awkward-cpu-kernels-objects.dir/src/cpu-kernels/sorting.cpp.o
2020-08-19T19:14:57.6801792Z   [ 13%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Iterator.cpp.o
2020-08-19T19:14:58.5246321Z   [ 14%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Reducer.cpp.o
2020-08-19T19:15:00.9736606Z   [ 16%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/Slice.cpp.o
2020-08-19T19:15:03.5345843Z   [ 17%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/BitMaskedArray.cpp.o
2020-08-19T19:15:05.7606761Z   [ 18%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/ByteMaskedArray.cpp.o
2020-08-19T19:15:08.3710223Z   [ 18%] Built target awkward-cpu-kernels-objects
2020-08-19T19:15:08.3712645Z   [ 20%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/EmptyArray.cpp.o
2020-08-19T19:15:08.3754733Z   Scanning dependencies of target awkward-cpu-kernels-static
2020-08-19T19:15:08.3805892Z   [ 21%] Linking CXX static library libawkward-cpu-kernels-static.a
2020-08-19T19:15:08.4235451Z   [ 21%] Built target awkward-cpu-kernels-static
2020-08-19T19:15:08.4278036Z   Scanning dependencies of target awkward-cpu-kernels
2020-08-19T19:15:08.4315904Z   [ 22%] Linking CXX shared library libawkward-cpu-kernels.so
2020-08-19T19:15:08.5192336Z   [ 22%] Built target awkward-cpu-kernels
2020-08-19T19:15:08.5233595Z   [ 24%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/IndexedArray.cpp.o
2020-08-19T19:15:10.3912498Z   [ 25%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/ListArray.cpp.o
2020-08-19T19:15:16.0986720Z   [ 27%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/ListOffsetArray.cpp.o
2020-08-19T19:15:17.1552240Z   [ 28%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/None.cpp.o
2020-08-19T19:15:18.8691461Z   [ 29%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/NumpyArray.cpp.o
2020-08-19T19:15:22.1449772Z   [ 31%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/Record.cpp.o
2020-08-19T19:15:24.2402100Z   [ 32%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/RecordArray.cpp.o
2020-08-19T19:15:27.9121497Z   [ 33%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/RegularArray.cpp.o
2020-08-19T19:15:28.6143662Z   [ 35%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/UnionArray.cpp.o
2020-08-19T19:15:30.5188530Z   [ 36%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/UnmaskedArray.cpp.o
2020-08-19T19:15:32.7344610Z   [ 37%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/array/VirtualArray.cpp.o
2020-08-19T19:15:35.0320944Z   [ 39%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/ArrayBuilder.cpp.o
2020-08-19T19:15:35.3571102Z   [ 40%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/ArrayBuilderOptions.cpp.o
2020-08-19T19:15:35.9810443Z   [ 41%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/BoolBuilder.cpp.o
2020-08-19T19:15:36.3162094Z   [ 43%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/Builder.cpp.o
2020-08-19T19:15:37.0200205Z   [ 44%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/Float64Builder.cpp.o
2020-08-19T19:15:37.0541115Z   [ 45%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/GrowableBuffer.cpp.o
2020-08-19T19:15:38.1035318Z   [ 47%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/IndexedBuilder.cpp.o
2020-08-19T19:15:38.2570744Z   [ 48%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/Int64Builder.cpp.o
2020-08-19T19:15:39.3562197Z   [ 50%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/ListBuilder.cpp.o
2020-08-19T19:15:40.2749146Z   [ 51%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/OptionBuilder.cpp.o
2020-08-19T19:15:40.5009011Z   [ 52%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/RecordBuilder.cpp.o
2020-08-19T19:15:41.4692837Z   [ 54%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/StringBuilder.cpp.o
2020-08-19T19:15:42.4348423Z   [ 55%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/TupleBuilder.cpp.o
2020-08-19T19:15:42.8285905Z   [ 56%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/UnionBuilder.cpp.o
2020-08-19T19:15:43.9593032Z   [ 58%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/builder/UnknownBuilder.cpp.o
2020-08-19T19:15:44.5793545Z   [ 59%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/io/json.cpp.o
2020-08-19T19:15:45.1346585Z   [ 60%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/io/root.cpp.o
2020-08-19T19:15:46.2517426Z   [ 62%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/kernel-dispatch.cpp.o
2020-08-19T19:15:47.1853462Z   [ 63%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/partition/IrregularlyPartitionedArray.cpp.o
2020-08-19T19:15:48.3569100Z   [ 64%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/partition/PartitionedArray.cpp.o
2020-08-19T19:15:49.5885255Z   [ 66%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/ArrayType.cpp.o
2020-08-19T19:15:50.4555943Z   [ 67%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/ListType.cpp.o
2020-08-19T19:15:51.4554676Z   [ 68%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/OptionType.cpp.o
2020-08-19T19:15:52.4846162Z   [ 70%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/PrimitiveType.cpp.o
2020-08-19T19:15:53.5493829Z   [ 71%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/RecordType.cpp.o
2020-08-19T19:15:55.1024851Z   [ 72%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/RegularType.cpp.o
2020-08-19T19:15:56.0873635Z   [ 74%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/Type.cpp.o
2020-08-19T19:15:56.8950184Z   [ 75%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/UnionType.cpp.o
2020-08-19T19:15:57.1878032Z   [ 77%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/type/UnknownType.cpp.o
2020-08-19T19:15:58.1085863Z   [ 78%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/util.cpp.o
2020-08-19T19:15:58.1553223Z   [ 79%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/virtual/ArrayCache.cpp.o
2020-08-19T19:15:58.9023077Z   [ 81%] Building CXX object CMakeFiles/awkward-objects.dir/src/libawkward/virtual/ArrayGenerator.cpp.o
2020-08-19T19:16:00.4990977Z   [ 81%] Built target awkward-objects
2020-08-19T19:16:00.5037571Z   Scanning dependencies of target awkward-static
2020-08-19T19:16:00.5041292Z   Scanning dependencies of target awkward
2020-08-19T19:16:00.5076761Z   [ 82%] Linking CXX shared library libawkward.so
2020-08-19T19:16:00.5084547Z   [ 83%] Linking CXX static library libawkward-static.a
2020-08-19T19:16:00.6843908Z   [ 83%] Built target awkward-static
2020-08-19T19:16:00.8603036Z   [ 83%] Built target awkward
2020-08-19T19:16:01.1198388Z   Scanning dependencies of target _ext
2020-08-19T19:16:01.1300583Z   [ 85%] Building CXX object CMakeFiles/_ext.dir/src/python/content.cpp.o
2020-08-19T19:16:01.1302148Z   [ 86%] Building CXX object CMakeFiles/_ext.dir/src/python/_ext.cpp.o
2020-08-19T19:16:04.6530258Z   [ 87%] Building CXX object CMakeFiles/_ext.dir/src/python/forms.cpp.o
2020-08-19T19:16:14.6254245Z   [ 89%] Building CXX object CMakeFiles/_ext.dir/src/python/identities.cpp.o
2020-08-19T19:16:18.9241076Z   [ 90%] Building CXX object CMakeFiles/_ext.dir/src/python/index.cpp.o
2020-08-19T19:16:23.4868145Z   [ 91%] Building CXX object CMakeFiles/_ext.dir/src/python/io.cpp.o
2020-08-19T19:16:26.1226559Z   [ 93%] Building CXX object CMakeFiles/_ext.dir/src/python/kernel_utils.cpp.o
2020-08-19T19:16:29.1618402Z   [ 94%] Building CXX object CMakeFiles/_ext.dir/src/python/partition.cpp.o
2020-08-19T19:16:32.9059236Z   [ 95%] Building CXX object CMakeFiles/_ext.dir/src/python/startup.cpp.o
2020-08-19T19:16:35.2361023Z   [ 97%] Building CXX object CMakeFiles/_ext.dir/src/python/types.cpp.o
2020-08-19T19:16:42.7497944Z   [ 98%] Building CXX object CMakeFiles/_ext.dir/src/python/virtual.cpp.o
2020-08-19T19:16:47.4964151Z   [100%] Linking CXX shared module _ext.cpython-38-x86_64-linux-gnu.so
2020-08-19T19:17:49.4909551Z   [100%] Built target _ext
2020-08-19T19:17:49.6406137Z   [  8%] Built target awkward-cpu-kernels-objects
2020-08-19T19:17:49.6475348Z   [  9%] Built target awkward-cpu-kernels-static
2020-08-19T19:17:49.6555194Z   [ 10%] Built target awkward-cpu-kernels
2020-08-19T19:17:49.6751747Z   [ 81%] Built target awkward-objects
2020-08-19T19:17:49.6825492Z   [ 82%] Built target awkward-static
2020-08-19T19:17:49.7012804Z   [ 98%] Built target _ext
2020-08-19T19:17:49.7086201Z   [100%] Built target awkward
2020-08-19T19:17:49.7143186Z   Install the project...
2020-08-19T19:17:49.7163861Z   -- Install configuration: "Release"
2020-08-19T19:17:49.7167640Z   -- Installing: /tmp/pip-req-build-aoz21nbp/build/lib.linux-x86_64-3.8/awkward1/libawkward-static.a
2020-08-19T19:17:49.7287833Z   -- Installing: /tmp/pip-req-build-aoz21nbp/build/lib.linux-x86_64-3.8/awkward1/libawkward.so
2020-08-19T19:17:49.7352491Z   -- Installing: /tmp/pip-req-build-aoz21nbp/build/lib.linux-x86_64-3.8/awkward1/libawkward-cpu-kernels.so
2020-08-19T19:17:49.7363241Z   -- Installing: /tmp/pip-req-build-aoz21nbp/build/lib.linux-x86_64-3.8/awkward1/libawkward-cpu-kernels-static.a
2020-08-19T19:17:49.7380442Z   -- Installing: /tmp/pip-req-build-aoz21nbp/build/lib.linux-x86_64-3.8/awkward1/_ext.cpython-38-x86_64-linux-gnu.so
2020-08-19T19:17:49.7538727Z   installing to build/bdist.linux-x86_64/wheel

@jpivarski
Copy link
Member

jpivarski commented Aug 19, 2020 •
edited
Loading

MacOS/Cling has found some implicit integer sign changes, but that's my problem. I'll deal with it later.

Edit: the above was fixed in #407, specifically 941aafc .

Windows/Visual Studio has no warnings (32-bit and 64-bit).

@veprbl veprbl marked this pull request as ready for review August 19, 2020 22:52
@veprbl
Copy link
Contributor Author

veprbl commented Aug 19, 2020

This PR is mostly aimed at silencing warnings in a reasonable way. The only visible change should be that the dependent libraries should not re-export any libawkward symbols, before it was pointless to set the default visibility to hidden.

@jpivarski
Copy link
Member

Okay, thanks for all of your help with this. The differences look good to me.

I do need to get the dependent project into the unit tests; something I'll tackle soon. The thing that was preventing it was not substantial—it was something like setting the PATH on Azure (I couldn't find where it had installed into).

@jpivarski jpivarski merged commit c78e7db into scikit-hep:master Aug 19, 2020
@veprbl veprbl deleted the pr/libawkward_export_tuning branch August 20, 2020 00:05
@veprbl
Copy link
Contributor Author

veprbl commented Aug 20, 2020

@jpivarski Sorry, I don't quite follow you on the dependent project thread. I haven't looked at this yet.

On another note, I must admit I still haven't tried applying awkward1 for my analysis, so I had some questions for you. Is it somewhat ready for use? Is parquet file reading supported?

@jpivarski
Copy link
Member

https://github.com/scikit-hep/awkward-1.0/tree/master/dependent-project

is an example of a C++ project that links to (depends on) Awkward. It used to be part of CI, then I had to drop it when something broke and I didn't know exactly what, now I'd like to reinstate it, since I think you've fixed linking issues, at least on MacOS.

Other thread: it is ready to use, and some analysis groups have been using it. Generally speaking, it's more complete and self-consistent than Awkward 0 was, and is better documented, too. Most of my work recently has been responding to bug reports and feature requests of users applying it to data analyses (not all of which are in HEP).

Parquet reading is supported. There are issues like not supporting partitioned datasets (#368), but I'm working through them. In fact, the thing I'm working on right now (#403) is adding a categorical type because generic IndexedArrays (in which "categories" are not necessarily unique) is hitting an upstream bug in Arrow (which assumes that they are unique). Declaring categorical types explicitly will make it clear which things I should translate to and from Parquet dictionaries—so this is second-order stuff. To first-order, it works.

@veprbl
Copy link
Contributor Author

veprbl commented Aug 20, 2020

@jpivarski This sounds very encouraging. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@veprbl @jpivarski