Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segfault in SlabsPolygonIndex::find on high longitude polygons #58612

Closed
cpg314 opened this issue Jan 8, 2024 · 8 comments · Fixed by #58771
Closed

Segfault in SlabsPolygonIndex::find on high longitude polygons #58612

cpg314 opened this issue Jan 8, 2024 · 8 comments · Fixed by #58771
Assignees
@yariks5s
Labels
potential bug To be reviewed by developers and confirmed/rejected.

Comments

@cpg314
Copy link

cpg314 commented Jan 8, 2024 •
edited

Describe what's wrong

A polygonal dictionary is created with longitudes and latitudes. The keys are MultiPolygons with a single polygon each.

When some of the polygon points contain coordinate with longitude close to +-180°, a segfault appears in SlabsPolygonIndex::find and makes the entire server crash.

Does it reproduce on recent release?

This is present on version 23.12.2.59 (official build), build id: 7F4C1A822F9C67A4D137A58F9A95BD4B0F1B6A8A, git hash: 17ab210)

How to reproduce

  • Create a polygon dictionary that contains points having longitude up to 179°.
  • Run a query that uses the dictionary, e.g. with dictGet.
  • The server crashes with a segfault.
  • Clamp the longitudes to 170°: no segfault.
  • Clamp the longitudes to 175°: still a segfault.

I have not yet tried creating a minimal example reproducing the bug.
Maybe it is clear to someone faimilar with SlabsPolygonIndex::find what line could be causing an invalid memory access.

Expected behavior

Even if some polygons are invalid, the server should not segfault

Error message and/or stacktrace

[533a9f31784d] 2024.01.08 22:28:56.775037 [ 765 ] <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
[533a9f31784d] 2024.01.08 22:28:56.775050 [ 765 ] <Fatal> BaseDaemon: Stack trace: 0x000000000e83760b 0x000000000e80ac04 0x000000000e83079c 0x000000000e82d470 0x000000000760de2c 0x000000000716cbce 0x000000000fc09ade 0x000000000fc0a65e 0x000000000fc0bc09 0x0000000010a7d4f9 0x0000000010a7e7a6 0x00000000122bd1bd 0x00000000122ba0ed 0x00000000122c28b6 0x0000000012b3080f 0x00000000122affca 0x0000000012b5d218 0x00000000126d2de3 0x00000000126ebd7a 0x00000000126e27b0 0x00000000126e3881 0x000000000c7bda84 0x000000000c7c12dc 0x000000000c7c00d8 0x00007f5b1e90bea7 0x00007f5b1e82ba2f
[533a9f31784d] 2024.01.08 22:28:56.775089 [ 765 ] <Fatal> BaseDaemon: 2. DB::SlabsPolygonIndex::find(boost::geometry::model::d2::point_xy<float, boost::geometry::cs::cartesian> const&, unsigned long&) const @ 0x000000000e83760b in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775102 [ 765 ] <Fatal> BaseDaemon: 3. DB::PolygonDictionaryIndexCell::find(boost::geometry::model::d2::point_xy<float, boost::geometry::cs::cartesian> const&, unsigned long&) const @ 0x000000000e80ac04 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775157 [ 765 ] <Fatal> BaseDaemon: 4. void detail::static_for<DB::AttributeUnderlyingType, void DB::callOnDictionaryAttributeType<DB::IPolygonDictionary::getColumn(String const&, std::shared_ptr<DB::IDataType const> const&, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&>(DB::AttributeUnderlyingType, DB::IPolygonDictionary::getColumn(String const&, std::shared_ptr<DB::IDataType const> const&, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)::'lambda'(DB::IPolygonDictionary::getColumn(String const&, std::shared_ptr<DB::IDataType const> const&, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&), 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul, 8ul, 9ul, 10ul, 11ul, 12ul, 13ul, 14ul, 15ul, 16ul, 17ul, 18ul, 19ul, 20ul, 21ul, 22ul, 23ul>(void DB::callOnDictionaryAttributeType<DB::IPolygonDictionary::getColumn(String const&, std::shared_ptr<DB::IDataType const> const&, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&>(DB::AttributeUnderlyingType, DB::IPolygonDictionary::getColumn(String const&, std::shared_ptr<DB::IDataType const> const&, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)::'lambda'(DB::IPolygonDictionary::getColumn(String const&, std::shared_ptr<DB::IDataType const> const&, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)&&, std::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul, 8ul, 9ul, 10ul, 11ul, 12ul, 13ul, 14ul, 15ul, 16ul, 17ul, 18ul, 19ul, 20ul, 21ul, 22ul, 23ul>) @ 0x000000000e83079c in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775174 [ 765 ] <Fatal> BaseDaemon: 5. DB::IPolygonDictionary::getColumn(String const&, std::shared_ptr<DB::IDataType const> const&, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x000000000e82d470 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775197 [ 765 ] <Fatal> BaseDaemon: 6. DB::FunctionDictGetNoType<(DB::DictionaryGetFunctionType)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x000000000760de2c in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775224 [ 765 ] <Fatal> BaseDaemon: 7. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x000000000716cbce in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775257 [ 765 ] <Fatal> BaseDaemon: 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000000fc09ade in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775276 [ 765 ] <Fatal> BaseDaemon: 9. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000000fc0a65e in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775297 [ 765 ] <Fatal> BaseDaemon: 10. DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000000fc0bc09 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775323 [ 765 ] <Fatal> BaseDaemon: 11. DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool) const @ 0x0000000010a7d4f9 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775347 [ 765 ] <Fatal> BaseDaemon: 12. DB::ExpressionActions::execute(DB::Block&, bool) const @ 0x0000000010a7e7a6 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775384 [ 765 ] <Fatal> BaseDaemon: 13. DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x00000000122bd1bd in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775401 [ 765 ] <Fatal> BaseDaemon: 14. DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x00000000122ba0ed in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775419 [ 765 ] <Fatal> BaseDaemon: 15. DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x00000000122c28b6 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775442 [ 765 ] <Fatal> BaseDaemon: 16. DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x0000000012b3080f in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775466 [ 765 ] <Fatal> BaseDaemon: 17. DB::MergeTreeSelectProcessor::read() @ 0x00000000122affca in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775485 [ 765 ] <Fatal> BaseDaemon: 18. DB::MergeTreeSource::tryGenerate() @ 0x0000000012b5d218 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775501 [ 765 ] <Fatal> BaseDaemon: 19. DB::ISource::work() @ 0x00000000126d2de3 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775520 [ 765 ] <Fatal> BaseDaemon: 20. DB::ExecutionThreadContext::executeTask() @ 0x00000000126ebd7a in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775553 [ 765 ] <Fatal> BaseDaemon: 21. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x00000000126e27b0 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775579 [ 765 ] <Fatal> BaseDaemon: 22. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x00000000126e3881 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775604 [ 765 ] <Fatal> BaseDaemon: 23. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000c7bda84 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775629 [ 765 ] <Fatal> BaseDaemon: 24. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c7c12dc in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775657 [ 765 ] <Fatal> BaseDaemon: 25. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7c00d8 in /usr/bin/clickhouse
[533a9f31784d] 2024.01.08 22:28:56.775688 [ 765 ] <Fatal> BaseDaemon: 26. start_thread @ 0x0000000000007ea7 in /lib/x86_64-linux-gnu/[libpthread-2.31.so](http://libpthread-2.31.so/)
[533a9f31784d] 2024.01.08 22:28:56.775714 [ 765 ] <Fatal> BaseDaemon: 27. ? @ 0x00000000000fba2f in /lib/x86_64-linux-gnu/[libc-2.31.so](http://libc-2.31.so/)
[533a9f31784d] 2024.01.08 22:28:56.786388 [ 763 ] <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: AC32B04C37311E5E413148969FBCE1EA)
[533a9f31784d] 2024.01.08 22:28:56.786737 [ 763 ] <Fatal> BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues
[533a9f31784d] 2024.01.08 22:28:56.786880 [ 763 ] <Fatal> BaseDaemon: No settings were changed
@cpg314 cpg314 added the potential bug To be reviewed by developers and confirmed/rejected. label Jan 8, 2024
@KochetovNicolai
Copy link
Member 

$ llvm-addr2line-13 -afiCe ./23.12.2.58/usr/lib/debug/usr/bin/clickhouse.debug 0x000000000e83760b 0x000000000e80ac04 0x000000000e83079c 0x000000000e82d470 0x000000000760de2c 0x000000000716cbce 0x000000000fc09ade 0x000000000fc0a65e 0x000000000fc0bc09 0x0000000010a7d4f9 0x0000000010a7e7a6 0x00000000122bd1bd 0x00000000122ba0ed 0x00000000122c28b6 0x0000000012b3080f 0x00000000122affca 0x0000000012b5d218 0x00000000126d2de3 0x00000000126ebd7a 0x00000000126e27b0 0x00000000126e3881 0x000000000c7bda84 0x000000000c7c12dc 0x000000000c7c00d8 0x00007f5b1e90bea7 0x00007f5b1e82ba2f
0xe83760b
DB::SlabsPolygonIndex::find(boost::geometry::model::d2::point_xy<float, boost::geometry::cs::cartesian> const&, unsigned long&) const
./build_docker/./build_docker/./src/Dictionaries/PolygonDictionaryUtils.cpp:270
0xe80ac04
DB::PolygonDictionaryIndexCell::find(boost::geometry::model::d2::point_xy<float, boost::geometry::cs::cartesian> const&, unsigned long&) const
./build_docker/./build_docker/./src/Dictionaries/PolygonDictionaryImplementations.cpp:146
0xe83079c
void DB::IPolygonDictionary::getItemsImpl<StringRef, auto DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0::operator()<DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> >(DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> const&) const::'lambda'(unsigned long), auto DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0::operator()<DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> >(DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> const&) const::'lambda'(StringRef), DB::DefaultValueProvider>(std::__1::vector<boost::geometry::model::d2::point_xy<float, boost::geometry::cs::cartesian>, std::__1::allocator<boost::geometry::model::d2::point_xy<float, boost::geometry::cs::cartesian> > > const&, auto DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0::operator()<DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> >(DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> const&) const::'lambda'(unsigned long)&&, auto DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0::operator()<DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> >(DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> const&) const::'lambda'(StringRef)&&, DB::DefaultValueProvider&) const
./build_docker/./build_docker/./src/Dictionaries/PolygonDictionary.cpp:358
auto DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0::operator()<DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> >(DB::DictionaryAttributeType<(DB::AttributeUnderlyingType)19> const&) const
./build_docker/./build_docker/./src/Dictionaries/PolygonDictionary.cpp:123
auto void DB::callOnDictionaryAttributeType<DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&>(DB::AttributeUnderlyingType, DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)::'lambda'(DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)::operator()<std::__1::integral_constant<DB::AttributeUnderlyingType, (DB::AttributeUnderlyingType)19> >(DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&) const
./build_docker/./src/Dictionaries/DictionaryStructure.h:88
void detail::static_for<DB::AttributeUnderlyingType, void DB::callOnDictionaryAttributeType<DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&>(DB::AttributeUnderlyingType, DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)::'lambda'(DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&), 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul, 8ul, 9ul, 10ul, 11ul, 12ul, 13ul, 14ul, 15ul, 16ul, 17ul, 18ul, 19ul, 20ul, 21ul, 22ul, 23ul>(void DB::callOnDictionaryAttributeType<DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&>(DB::AttributeUnderlyingType, DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)::'lambda'(DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const::$_0&)&&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul, 8ul, 9ul, 10ul, 11ul, 12ul, 13ul, 14ul, 15ul, 16ul, 17ul, 18ul, 19ul, 20ul, 21ul, 22ul, 23ul>)
./build_docker/./base/base/../base/EnumReflection.h:14
0xe82d470
boost::intrusive_ptr<DB::IColumn const>::intrusive_ptr<DB::IColumn>(boost::intrusive_ptr<DB::IColumn>&&, boost::detail::sp_enable_if_convertible<DB::IColumn, DB::IColumn const>::type)
./build_docker/./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:140
COW<DB::IColumn>::immutable_ptr<DB::IColumn>::immutable_ptr<DB::IColumn>(COW<DB::IColumn>::mutable_ptr<DB::IColumn>&&)
./build_docker/./src/Common/COW.h:144
DB::IPolygonDictionary::getColumn(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, std::__1::vector<std::__1::shared_ptr<DB::IDataType const>, std::__1::allocator<std::__1::shared_ptr<DB::IDataType const> > > const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const
./build_docker/./build_docker/./src/Dictionaries/PolygonDictionary.cpp:145
0x760de2c
DB::FunctionDictGetNoType<(DB::DictionaryGetFunctionType)0>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const
FunctionsExternalDictionaries.cpp:0
0x716cbce
DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const
CRC.cpp:0
0xfc09ade
boost::intrusive_ptr<DB::IColumn const>::intrusive_ptr(boost::intrusive_ptr<DB::IColumn const>&&)
./build_docker/./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117
boost::intrusive_ptr<DB::IColumn const>::operator=(boost::intrusive_ptr<DB::IColumn const>&&)
./build_docker/./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:124
COW<DB::IColumn>::immutable_ptr<DB::IColumn>::operator=(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&&)
./build_docker/./src/Common/COW.h:136
DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const
./build_docker/./build_docker/./src/Functions/IFunction.cpp:0
0xfc0a65e
boost::intrusive_ptr<DB::IColumn const>::intrusive_ptr(boost::intrusive_ptr<DB::IColumn const>&&)
./build_docker/./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117
boost::intrusive_ptr<DB::IColumn const>::operator=(boost::intrusive_ptr<DB::IColumn const>&&)
./build_docker/./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:124
COW<DB::IColumn>::immutable_ptr<DB::IColumn>::operator=(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&&)
./build_docker/./src/Common/COW.h:136
DB::IExecutableFunction::executeWithoutSparseColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const
./build_docker/./build_docker/./src/Functions/IFunction.cpp:303
0xfc0bc09
std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> >::~vector[abi:v15000]()
./build_docker/./contrib/llvm-project/libcxx/include/vector:434
DB::IExecutableFunction::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const
./build_docker/./build_docker/./src/Functions/IFunction.cpp:379
0x10a7d4f9
boost::intrusive_ptr<DB::IColumn const>::intrusive_ptr(boost::intrusive_ptr<DB::IColumn const>&&)
./build_docker/./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117
boost::intrusive_ptr<DB::IColumn const>::operator=(boost::intrusive_ptr<DB::IColumn const>&&)
./build_docker/./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:124
COW<DB::IColumn>::immutable_ptr<DB::IColumn>::operator=(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&&)
./build_docker/./src/Common/COW.h:136
DB::executeAction(DB::ExpressionActions::Action const&, DB::(anonymous namespace)::ExecutionContext&, bool)
./build_docker/./build_docker/./src/Interpreters/ExpressionActions.cpp:613
DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool) const
./build_docker/./build_docker/./src/Interpreters/ExpressionActions.cpp:730
0x10a7e7a6
std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> >::size[abi:v15000]() const
./build_docker/./contrib/llvm-project/libcxx/include/vector:537
DB::Block::columns() const
./build_docker/./src/Core/Block.h:106
DB::Block::operator bool() const
./build_docker/./src/Core/Block.h:117
DB::Block::operator!() const
./build_docker/./src/Core/Block.h:118
DB::ExpressionActions::execute(DB::Block&, bool) const
./build_docker/./build_docker/./src/Interpreters/ExpressionActions.cpp:776
0x122bd1bd
std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long[abi:v15000]() const
./build_docker/./contrib/llvm-project/libcxx/include/string:1499
std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::size[abi:v15000]() const
./build_docker/./contrib/llvm-project/libcxx/include/string:968
std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::empty[abi:v15000]() const
./build_docker/./contrib/llvm-project/libcxx/include/string:996
DB::MergeTreeRangeReader::executePrewhereActionsAndFilterColumns(DB::MergeTreeRangeReader::ReadResult&) const
./build_docker/./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:1393
DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&)
./build_docker/./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:1084
0x122ba0ed
DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&)
./build_docker/./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0
0x122c28b6
DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&)
./build_docker/./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:163
0x12b3080f
DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&)
./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38
0x122affca
DB::MergeTreeSelectProcessor::read()
./build_docker/./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:162
0x12b5d218
DB::MergeTreeSource::processReadResult(DB::ChunkAndProgress)
./build_docker/./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181
DB::MergeTreeSource::tryGenerate()
./build_docker/./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:226
0x126d2de3
std::__1::__optional_storage_base<DB::Chunk, false>::has_value[abi:v15000]() const
./build_docker/./contrib/llvm-project/libcxx/include/optional:344
std::__1::optional<DB::Chunk>::operator bool[abi:v15000]() const
./build_docker/./contrib/llvm-project/libcxx/include/optional:998
DB::ISource::work()
./build_docker/./build_docker/./src/Processors/ISource.cpp:108
0x126ebd7a
DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*)
./build_docker/./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0
DB::ExecutionThreadContext::executeTask()
./build_docker/./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:95
0x126e27b0
DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*)
./build_docker/./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273
0x126e3881
std::__1::shared_ptr<DB::ThreadGroup>::operator bool[abi:v15000]() const
./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:833
DB::PipelineExecutor::spawnThreads()::$_0::operator()() const::'lambda'()::operator()() const
./build_docker/./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:362
BasicScopeGuard<DB::PipelineExecutor::spawnThreads()::$_0::operator()() const::'lambda'()>::invoke()
./build_docker/./base/base/../base/scope_guard.h:99
BasicScopeGuard<DB::PipelineExecutor::spawnThreads()::$_0::operator()() const::'lambda'()>::~BasicScopeGuard()
./build_docker/./base/base/../base/scope_guard.h:48
DB::PipelineExecutor::spawnThreads()::$_0::operator()() const
./build_docker/./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381
decltype(std::declval<DB::PipelineExecutor::spawnThreads()::$_0&>()()) std::__1::__invoke[abi:v15000]<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&)
./build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394
void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::PipelineExecutor::spawnThreads()::$_0&>(DB::PipelineExecutor::spawnThreads()::$_0&)
./build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479
std::__1::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>::operator()[abi:v15000]()
./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235
void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()> >(std::__1::__function::__policy_storage const*)
./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716
0xc7bda84
bool wide::integer<128ul, unsigned int>::_impl::operator_eq<wide::integer<128ul, unsigned int> >(wide::integer<128ul, unsigned int> const&, wide::integer<128ul, unsigned int> const&)
./build_docker/./base/base/../base/wide_integer_impl.h:809
bool wide::operator==<128ul, unsigned int, 128ul, unsigned int>(wide::integer<128ul, unsigned int> const&, wide::integer<128ul, unsigned int> const&)
./build_docker/./base/base/../base/wide_integer_impl.h:1482
StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag>::operator==(StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag> const&) const
./build_docker/./base/base/../base/strong_typedef.h:42
DB::OpenTelemetry::Span::isTraceEnabled() const
./build_docker/./src/Common/OpenTelemetryTraceContext.h:65
ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::worker(std::__1::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>)
./build_docker/./build_docker/./src/Common/ThreadPool.cpp:457
0xc7c12dc
ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()()
./build_docker/./build_docker/./src/Common/ThreadPool.cpp:0
decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&)
./build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394
void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&)
./build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479
std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]()
./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235
void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*)
./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716
0xc7c00d8
bool wide::integer<128ul, unsigned int>::_impl::operator_eq<wide::integer<128ul, unsigned int> >(wide::integer<128ul, unsigned int> const&, wide::integer<128ul, unsigned int> const&)
./build_docker/./base/base/../base/wide_integer_impl.h:809
bool wide::operator==<128ul, unsigned int, 128ul, unsigned int>(wide::integer<128ul, unsigned int> const&, wide::integer<128ul, unsigned int> const&)
./build_docker/./base/base/../base/wide_integer_impl.h:1482
StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag>::operator==(StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag> const&) const
./build_docker/./base/base/../base/strong_typedef.h:42
DB::OpenTelemetry::Span::isTraceEnabled() const
./build_docker/./src/Common/OpenTelemetryTraceContext.h:65
ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>)
./build_docker/./build_docker/./src/Common/ThreadPool.cpp:457
void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const
./build_docker/./build_docker/./src/Common/ThreadPool.cpp:217
decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)
./build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394
void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>)
./build_docker/./contrib/llvm-project/libcxx/include/thread:284
void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()> >(void*)
./build_docker/./contrib/llvm-project/libcxx/include/thread:295
0x7f5b1e90bea7
??
??:0
0x7f5b1e82ba2f
??
??:0

@yariks5s yariks5s self-assigned this Jan 9, 2024
@cpg314 cpg314 mentioned this issue Jan 9, 2024
Open
@yariks5s
Copy link
Member

Hi, @cpg314 !😀 Thanks for reporting an issue! Can you please provide an example, of how you got this error?

@cpg314
Copy link
Author

cpg314 commented Jan 10, 2024 •
edited

I've been trying to create a repro, but this is fairly difficult:
This works:
SELECT sum(dictHas(dict, position)) FROM source
This segfaults
SELECT sum(dictHas(dict, position)) FROM other_big_table
Probably some points in big_table are triggering the segfault, but I have not been able to identify them. It doesn't segfault with the min/max points.

Do you have an idea how I could easily find the offending point(s)? I could write something that performs binary search among the points until a segfault occurs, but the big tables has 100 million rows.

@yariks5s
Copy link
Member

yariks5s commented Jan 11, 2024 •
edited

I've been trying to create a repro, but this is fairly difficult: This works: SELECT sum(dictHas(dict, position)) FROM source This segfaults SELECT sum(dictHas(dict, position)) FROM other_big_table Probably some points in big_table are triggering the segfault, but I have not been able to identify them. It doesn't segfault with the min/max points.

Do you have an idea how I could easily find the offending point(s)? I could write something that performs binary search among the points until a segfault occurs, but the big tables has 100 million rows.

Just in case, is it possible that some values simply didn't exist in your table? It'd be appreciated to give some hints(e.g. structure of dicts, data that causes segfaults etc) because now it's not clear what can go wrong here.

@cpg314
Copy link
Author

cpg314 commented Jan 11, 2024 •
edited

It'd be appreciated to give some hints(e.g. structure of dicts, data that causes segfaults etc) because now it's not clear what can go wrong here.

The dict looks like this:

CREATE DICTIONARY dict
(
  ident String,
  geom MultiPolygon
)
PRIMARY KEY geom
SOURCE(TABLE 'source')
LIFETIME(MIN 0 MAX 0)
LAYOUT(POLYGON(STORE_POLYGON_KEY_COLUMN 1))

The source table has the same fields.

The big_table like this:

CREATE TABLE IF NOT EXISTS big_table  (
    name String,
    position Point,
)
ENGINE = ReplacingMergeTree()
PRIMARY KEY name

No segfault: SELECT sum(dictHas(dict, position)) FROM source
Segfault: SELECT sum(dictHas(dict, position)) FROM big_table

The segfault seems to be triggered by a combination of certain polygons in the dictionary and certain points in the big_table. Indeed:

  • On many subsets of big_table, the query above does not segfault (SELECT sum(dictHas(dict, position)) FROM (SELECT position from big_table LIMIT N).
  • As mentioned in the first message, if I exclude some polygons (the ones with high longitudes) from the dictionary, then no segfault either. I made another dictionary with only these polygons, which also triggers the segfault.

The dictionary is very small (3MB), so I could share it privately. On the other hand, the other table is very large, so I would need to identify the offending points first.

Just in case, is it possible that some values simply didn't exist in your table?

All of the position in big_table are finite floating point, and summing them works fine.

Also there is no segfault with POLYGON_SIMPLE, so it has to be something with the polygon index.

@cpg314
Copy link
Author

cpg314 commented Jan 11, 2024 •
edited

Ok, with a bit of pain, I managed to find a single point that makes the server segfault with:
SELECT dictHas(dict,(174.84729269276494,-36.99524960275426))
I can share dict privately if that helps debugging.

@yariks5s
Copy link
Member

yariks5s commented Jan 12, 2024 •
edited

Ok, with a bit of pain, I managed to find a single point that makes the server segfault with: SELECT dictHas(dict,(174.84729269276494,-36.99524960275426)) I can share dict privately if that helps debugging.

It will definitely help. I have link to my telegram in my github account info: https://t.me/yarik_kk. We can discuss sharing there

@cpg314
Copy link
Author

cpg314 commented Jan 12, 2024

Thanks, I have messaged you the reproduction data.

@yariks5s yariks5s mentioned this issue Jan 12, 2024
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yariks5s yariks5s

Labels
potential bug To be reviewed by developers and confirmed/rejected.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Segfault in SlabsPolygonIndex::find yariks5s/ClickHouse
3 participants
@KochetovNicolai @cpg314 @yariks5s