Can not read Avro file with zstd codec.

[ucasfl]

Received exception from server (version 52.8.6):
Code: 1001. DB::Exception: Received from. DB::Exception: avro::Exception: Unknown codec in data file: zstandard. (STD_EXCEPTION)

StackTrace:

2024.01.12 11:53:52.629046 [ 83 ] {068906a0-24e4-4b29-b8e5-c5835cfb0901} <Error> TCPHandler: Code: 1001. DB::Exception: avro::Exception: Unknown codec in data file: zstandard. (STD_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. std::runtime_error::runtime_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x1aa99f6d in /usr/bin/clickhouse
1. avro::DataFileReaderBase::readHeader() @ 0x18ac5108 in /usr/bin/clickhouse
2. avro::DataFileReaderBase::DataFileReaderBase(std::__1::unique_ptr<avro::InputStream, std::__1::default_delete<avro::InputStream> >) @ 0x18ac570b in /usr/bin/clickhouse
3. DB::AvroRowInputFormat::readPrefix() @ 0x16449ac2 in /usr/bin/clickhouse
4. DB::IRowInputFormat::generate() @ 0x16463b73 in /usr/bin/clickhouse
5. DB::ISource::tryGenerate() @ 0x16401fb5 in /usr/bin/clickhouse
6. DB::ISource::work() @ 0x16401c46 in /usr/bin/clickhouse
7. DB::ExecutionThreadContext::executeTask() @ 0x1641dee3 in /usr/bin/clickhouse
8. DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) @ 0x1641409a in /usr/bin/clickhouse
9. DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*) @ 0x16413bc9 in /usr/bin/clickhouse
10. DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x16421535 in /usr/bin/clickhouse
11. DB::HDFSSource::generate() @ 0x1486cf6c in /usr/bin/clickhouse
12. DB::ISource::tryGenerate() @ 0x16401fb5 in /usr/bin/clickhouse
13. DB::ISource::work() @ 0x16401c46 in /usr/bin/clickhouse
14. DB::ExecutionThreadContext::executeTask() @ 0x1641dee3 in /usr/bin/clickhouse
15. DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) @ 0x1641409a in /usr/bin/clickhouse
16. DB::PipelineExecutor::executeImpl(unsigned long) @ 0x16413971 in /usr/bin/clickhouse
17. DB::PipelineExecutor::execute(unsigned long) @ 0x16413818 in /usr/bin/clickhouse
18. void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x16420a2a in /usr/bin/clickhouse
19. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xb5db70e in /usr/bin/clickhouse
20. void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()> >(void*) @ 0xb5dda2e in /usr/bin/clickhouse
21. start_thread @ 0x7ea5 in /usr/lib64/libpthread-2.17.so
22. __clone @ 0xfe9fd in /usr/lib64/libc-2.17.so

[ucasfl]

And I checked the code of avro library a little, and looks like that the C++ API can not reading file with zstd codec? @Avogar

[alexey-milovidov]

version 52.8.6

No such version exists.
We don't support any alternative, modified ClickHouse builds.

[ucasfl]

@alexey-milovidov Sorry didn't provide proper information, the issue also exists in master branch, and should be fixed in #58805.