FileHandles aren't closed outside of C++ destructors - leading to unhandled I/O exceptions. #12146

rustyconover · 2024-05-20T19:32:55Z

What happens?

I wrote a DuckDB extension called shellfs, it derives from the FileSystem class and returns file handles that derive from FileHandle.

When there is a problem with a subprocess started by popen() the exit code or signal is only available once the pipe is closed with pclose(). I added the code that does the error checking and it throws an IOException when the subprocess exits with a non-zero exit code.

The problem is that the Close() method is only called as part of the destructor cleanup of the FileHandle. And in C++ throwing exceptions in destructors is a no no.

What is the recommended way to ensure that Close() is called on FileHandle implementations rather than just letting them be closed via destructors when the smart pointers are cleaned up?

Here is the code for the extension:

https://github.com/rustyconover/duckdb-shellfs-extension

Here is the backtrace for the error:

libc++abi: terminating due to uncaught exception of type duckdb::IOException: {"exception_type":"IO","exception_message":"Pipe process exited with non-zero exit code=\"127\": foo |"}
Process 80815 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x00000001993a6a60 libsystem_kernel.dylib`__pthread_kill + 8

 thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x00000001993a6a60 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x00000001993dec20 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x00000001992eba20 libsystem_c.dylib`abort + 180
    frame #3: 0x0000000199395d30 libc++abi.dylib`abort_message + 132
    frame #4: 0x0000000199385fcc libc++abi.dylib`demangling_terminate_handler() + 320
  * frame #5: 0x00000001990241e0 libobjc.A.dylib`_objc_terminate() + 160
    frame #6: 0x00000001993950f4 libc++abi.dylib`std::__terminate(void (*)()) + 16
    frame #7: 0x0000000199395098 libc++abi.dylib`std::terminate() + 108
    frame #8: 0x0000000100009878 duckdb`__clang_call_terminate + 12
    frame #9: 0x000000010a377fac duckdb`duckdb::ShellFileHandle::~ShellFileHandle(this=0x000000011b692010) at shell_file_system.cpp:42:3
    frame #10: 0x000000010a3769b4 duckdb`duckdb::ShellFileHandle::~ShellFileHandle(this=0x000000011b692010) at shell_file_system.cpp:41:30
    frame #11: 0x000000010a376a20 duckdb`duckdb::ShellFileHandle::~ShellFileHandle(this=0x000000011b692010) at shell_file_system.cpp:41:30
    frame #12: 0x0000000104e64ea8 duckdb`std::__1::default_delete<duckdb::FileHandle>::operator()[abi:ue170006](this=0x000000011c500be8, __ptr=0x000000011b692010) const at unique_ptr.h:68:5
    frame #13: 0x0000000104e64ac8 duckdb`std::__1::unique_ptr<duckdb::FileHandle, std::__1::default_delete<duckdb::FileHandle>>::reset[abi:ue170006](this=0x000000011c500be8, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #14: 0x0000000104e64834 duckdb`std::__1::unique_ptr<duckdb::FileHandle, std::__1::default_delete<duckdb::FileHandle>>::~unique_ptr[abi:ue170006](this=0x000000011c500be8) at unique_ptr.h:266:75
    frame #15: 0x0000000104e647c4 duckdb`duckdb::unique_ptr<duckdb::FileHandle, std::__1::default_delete<duckdb::FileHandle>, true>::~unique_ptr(this=0x000000011c500be8) at unique_ptr.hpp:13:7
    frame #16: 0x0000000104e6069c duckdb`duckdb::unique_ptr<duckdb::FileHandle, std::__1::default_delete<duckdb::FileHandle>, true>::~unique_ptr(this=0x000000011c500be8) at unique_ptr.hpp:13:7
    frame #17: 0x0000000105b6b848 duckdb`duckdb::CSVFileHandle::~CSVFileHandle(this=0x000000011c500ba0) at csv_file_handle.hpp:20:8
    frame #18: 0x0000000105b6b7cc duckdb`duckdb::CSVFileHandle::~CSVFileHandle(this=0x000000011c500ba0) at csv_file_handle.hpp:20:8
    frame #19: 0x0000000105b6b648 duckdb`std::__1::default_delete<duckdb::CSVFileHandle>::operator()[abi:ue170006](this=0x000000011bb436d8, __ptr=0x000000011c500ba0) const at unique_ptr.h:68:5
    frame #20: 0x0000000105b6b390 duckdb`std::__1::unique_ptr<duckdb::CSVFileHandle, std::__1::default_delete<duckdb::CSVFileHandle>>::reset[abi:ue170006](this=0x000000011bb436d8, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #21: 0x0000000105b6b0fc duckdb`std::__1::unique_ptr<duckdb::CSVFileHandle, std::__1::default_delete<duckdb::CSVFileHandle>>::~unique_ptr[abi:ue170006](this=0x000000011bb436d8) at unique_ptr.h:266:75
    frame #22: 0x0000000105b6b08c duckdb`duckdb::unique_ptr<duckdb::CSVFileHandle, std::__1::default_delete<duckdb::CSVFileHandle>, true>::~unique_ptr(this=0x000000011bb436d8) at unique_ptr.hpp:13:7
    frame #23: 0x0000000105b695e8 duckdb`duckdb::unique_ptr<duckdb::CSVFileHandle, std::__1::default_delete<duckdb::CSVFileHandle>, true>::~unique_ptr(this=0x000000011bb436d8) at unique_ptr.hpp:13:7
    frame #24: 0x0000000105b694a4 duckdb`duckdb::CSVBufferManager::~CSVBufferManager(this=0x000000011bb436d8) at csv_buffer_manager.hpp:22:7
    frame #25: 0x0000000105b693fc duckdb`duckdb::CSVBufferManager::~CSVBufferManager(this=0x000000011bb436d8) at csv_buffer_manager.hpp:22:7
    frame #26: 0x0000000105b68cf4 duckdb`std::__1::__shared_ptr_emplace<duckdb::CSVBufferManager, std::__1::allocator<duckdb::CSVBufferManager>>::__on_zero_shared(this=0x000000011bb436c0) at shared_ptr.h:324:24
    frame #27: 0x00000001001fe478 duckdb`std::__1::__shared_count::__release_shared[abi:ue170006](this=0x000000011bb436c0) at shared_ptr.h:173:9
    frame #28: 0x00000001001fe1ac duckdb`std::__1::__shared_weak_count::__release_shared[abi:ue170006](this=0x000000011bb436c0) at shared_ptr.h:214:27
    frame #29: 0x0000000105b6c6e4 duckdb`std::__1::shared_ptr<duckdb::CSVBufferManager>::~shared_ptr[abi:ue170006](this=0x000000011ba2d428) at shared_ptr.h:773:23
    frame #30: 0x0000000105b6630c duckdb`std::__1::shared_ptr<duckdb::CSVBufferManager>::~shared_ptr[abi:ue170006](this=0x000000011ba2d428) at shared_ptr.h:771:5
    frame #31: 0x0000000105c540b0 duckdb`duckdb::ReadCSVData::~ReadCSVData(this=0x000000011ba2d080) at read_csv.hpp:75:8
    frame #32: 0x0000000105ade230 duckdb`duckdb::ReadCSVData::~ReadCSVData(this=0x000000011ba2d080) at read_csv.hpp:75:8
    frame #33: 0x0000000105ade29c duckdb`duckdb::ReadCSVData::~ReadCSVData(this=0x000000011ba2d080) at read_csv.hpp:75:8
    frame #34: 0x00000001006db920 duckdb`std::__1::default_delete<duckdb::FunctionData>::operator()[abi:ue170006](this=0x000000011b80f860, __ptr=0x000000011ba2d080) const at unique_ptr.h:68:5
    frame #35: 0x00000001006db540 duckdb`std::__1::unique_ptr<duckdb::FunctionData, std::__1::default_delete<duckdb::FunctionData>>::reset[abi:ue170006](this=0x000000011b80f860, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #36: 0x00000001006db2ac duckdb`std::__1::unique_ptr<duckdb::FunctionData, std::__1::default_delete<duckdb::FunctionData>>::~unique_ptr[abi:ue170006](this=0x000000011b80f860) at unique_ptr.h:266:75
    frame #37: 0x00000001006db23c duckdb`duckdb::unique_ptr<duckdb::FunctionData, std::__1::default_delete<duckdb::FunctionData>, true>::~unique_ptr(this=0x000000011b80f860) at unique_ptr.hpp:13:7
    frame #38: 0x00000001006da1bc duckdb`duckdb::unique_ptr<duckdb::FunctionData, std::__1::default_delete<duckdb::FunctionData>, true>::~unique_ptr(this=0x000000011b80f860) at unique_ptr.hpp:13:7
    frame #39: 0x0000000108548c3c duckdb`duckdb::PhysicalTableScan::~PhysicalTableScan(this=0x000000011b80f680) at physical_table_scan.hpp:20:7
    frame #40: 0x000000010853f50c duckdb`duckdb::PhysicalTableScan::~PhysicalTableScan(this=0x000000011b80f680) at physical_table_scan.hpp:20:7
    frame #41: 0x000000010853f578 duckdb`duckdb::PhysicalTableScan::~PhysicalTableScan(this=0x000000011b80f680) at physical_table_scan.hpp:20:7
    frame #42: 0x000000010756cac8 duckdb`std::__1::default_delete<duckdb::PhysicalOperator>::operator()[abi:ue170006](this=0x000000011b511a10, __ptr=0x000000011b80f680) const at unique_ptr.h:68:5
    frame #43: 0x00000001072413c8 duckdb`std::__1::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>>::reset[abi:ue170006](this=0x000000011b511a10, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #44: 0x0000000107bcd68c duckdb`std::__1::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>>::~unique_ptr[abi:ue170006](this=0x000000011b511a10) at unique_ptr.h:266:75
    frame #45: 0x0000000107bcd61c duckdb`duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>::~unique_ptr(this=0x000000011b511a10) at unique_ptr.hpp:13:7
    frame #46: 0x0000000107bcd5b0 duckdb`duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>::~unique_ptr(this=0x000000011b511a10) at unique_ptr.hpp:13:7
    frame #47: 0x0000000107bcd548 duckdb`std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>::destroy[abi:ue170006](this=0x000000011c101320, __p=0x000000011b511a10) at allocator.h:172:15
    frame #48: 0x0000000107bcd4a0 duckdb`void std::__1::allocator_traits<std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::destroy[abi:ue170006]<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, void>(__a=0x000000011c101320, __p=0x000000011b511a10) at allocator_traits.h:315:13
    frame #49: 0x0000000107bcd414 duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::__base_destruct_at_end[abi:ue170006](this=0x000000011c101310 size=1, __new_last=0x000000011b511a10) at vector:944:9
    frame #50: 0x0000000107bcccdc duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::__clear[abi:ue170006](this=0x000000011c101310 size=1) at vector:938:29
    frame #51: 0x0000000107bcc8d0 duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::__destroy_vector::operator()[abi:ue170006](this=0x000000016fdfb440) at vector:489:20
    frame #52: 0x0000000107bcc5f8 duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::~vector[abi:ue170006](this=0x000000011c101310 size=1) at vector:500:67
    frame #53: 0x0000000107bcc428 duckdb`duckdb::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, true>::~vector(this=0x000000011c101310) at vector.hpp:21:7
    frame #54: 0x0000000107bcbb5c duckdb`duckdb::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, true>::~vector(this=0x000000011c101310) at vector.hpp:21:7
    frame #55: 0x0000000107b50d6c duckdb`duckdb::PhysicalOperator::~PhysicalOperator(this=0x000000011c101300) at physical_operator.hpp:42:2
    frame #56: 0x000000010851bda8 duckdb`duckdb::PhysicalProjection::~PhysicalProjection(this=0x000000011c101300) at physical_projection.hpp:16:7
    frame #57: 0x000000010850c624 duckdb`duckdb::PhysicalProjection::~PhysicalProjection(this=0x000000011c101300) at physical_projection.hpp:16:7
    frame #58: 0x000000010850c690 duckdb`duckdb::PhysicalProjection::~PhysicalProjection(this=0x000000011c101300) at physical_projection.hpp:16:7
    frame #59: 0x000000010756cac8 duckdb`std::__1::default_delete<duckdb::PhysicalOperator>::operator()[abi:ue170006](this=0x000000011b5118f0, __ptr=0x000000011c101300) const at unique_ptr.h:68:5
    frame #60: 0x00000001072413c8 duckdb`std::__1::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>>::reset[abi:ue170006](this=0x000000011b5118f0, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #61: 0x0000000107bcd68c duckdb`std::__1::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>>::~unique_ptr[abi:ue170006](this=0x000000011b5118f0) at unique_ptr.h:266:75
    frame #62: 0x0000000107bcd61c duckdb`duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>::~unique_ptr(this=0x000000011b5118f0) at unique_ptr.hpp:13:7
    frame #63: 0x0000000107bcd5b0 duckdb`duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>::~unique_ptr(this=0x000000011b5118f0) at unique_ptr.hpp:13:7
    frame #64: 0x0000000107bcd548 duckdb`std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>::destroy[abi:ue170006](this=0x000000011c70ae60, __p=0x000000011b5118f0) at allocator.h:172:15
    frame #65: 0x0000000107bcd4a0 duckdb`void std::__1::allocator_traits<std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::destroy[abi:ue170006]<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, void>(__a=0x000000011c70ae60, __p=0x000000011b5118f0) at allocator_traits.h:315:13
    frame #66: 0x0000000107bcd414 duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::__base_destruct_at_end[abi:ue170006](this=0x000000011c70ae50 size=1, __new_last=0x000000011b5118f0) at vector:944:9
    frame #67: 0x0000000107bcccdc duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::__clear[abi:ue170006](this=0x000000011c70ae50 size=1) at vector:938:29
    frame #68: 0x0000000107bcc8d0 duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::__destroy_vector::operator()[abi:ue170006](this=0x000000016fdfba20) at vector:489:20
    frame #69: 0x0000000107bcc5f8 duckdb`std::__1::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, std::__1::allocator<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>>>::~vector[abi:ue170006](this=0x000000011c70ae50 size=1) at vector:500:67
    frame #70: 0x0000000107bcc428 duckdb`duckdb::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, true>::~vector(this=0x000000011c70ae50) at vector.hpp:21:7
    frame #71: 0x0000000107bcbb5c duckdb`duckdb::vector<duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>, true>::~vector(this=0x000000011c70ae50) at vector.hpp:21:7
    frame #72: 0x0000000107b50d6c duckdb`duckdb::PhysicalOperator::~PhysicalOperator(this=0x000000011c70ae40) at physical_operator.hpp:42:2
    frame #73: 0x0000000107c5a364 duckdb`duckdb::PhysicalUngroupedAggregate::~PhysicalUngroupedAggregate(this=0x000000011c70ae40) at physical_ungrouped_aggregate.hpp:23:7
    frame #74: 0x0000000107bb3644 duckdb`duckdb::PhysicalUngroupedAggregate::~PhysicalUngroupedAggregate(this=0x000000011c70ae40) at physical_ungrouped_aggregate.hpp:23:7
    frame #75: 0x0000000107bb36b0 duckdb`duckdb::PhysicalUngroupedAggregate::~PhysicalUngroupedAggregate(this=0x000000011c70ae40) at physical_ungrouped_aggregate.hpp:23:7
    frame #76: 0x000000010756cac8 duckdb`std::__1::default_delete<duckdb::PhysicalOperator>::operator()[abi:ue170006](this=0x000000011c04c6e8, __ptr=0x000000011c70ae40) const at unique_ptr.h:68:5
    frame #77: 0x00000001072413c8 duckdb`std::__1::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>>::reset[abi:ue170006](this=0x000000011c04c6e8, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #78: 0x0000000107bcd68c duckdb`std::__1::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>>::~unique_ptr[abi:ue170006](this=0x000000011c04c6e8) at unique_ptr.h:266:75
    frame #79: 0x0000000107bcd61c duckdb`duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>::~unique_ptr(this=0x000000011c04c6e8) at unique_ptr.hpp:13:7
    frame #80: 0x0000000107bcd5b0 duckdb`duckdb::unique_ptr<duckdb::PhysicalOperator, std::__1::default_delete<duckdb::PhysicalOperator>, true>::~unique_ptr(this=0x000000011c04c6e8) at unique_ptr.hpp:13:7
    frame #81: 0x0000000108ea8e34 duckdb`duckdb::PreparedStatementData::~PreparedStatementData(this=0x000000011c04c6d8) at prepared_statement_data.cpp:14:1
    frame #82: 0x0000000108ea8eac duckdb`duckdb::PreparedStatementData::~PreparedStatementData(this=0x000000011c04c6d8) at prepared_statement_data.cpp:13:49
    frame #83: 0x000000010167d354 duckdb`std::__1::__shared_ptr_emplace<duckdb::PreparedStatementData, std::__1::allocator<duckdb::PreparedStatementData>>::__on_zero_shared(this=0x000000011c04c6c0) at shared_ptr.h:324:24
    frame #84: 0x00000001001fe478 duckdb`std::__1::__shared_count::__release_shared[abi:ue170006](this=0x000000011c04c6c0) at shared_ptr.h:173:9
    frame #85: 0x00000001001fe1ac duckdb`std::__1::__shared_weak_count::__release_shared[abi:ue170006](this=0x000000011c04c6c0) at shared_ptr.h:214:27
    frame #86: 0x00000001010c60e0 duckdb`std::__1::shared_ptr<duckdb::PreparedStatementData>::~shared_ptr[abi:ue170006](this=0x000000011c055190) at shared_ptr.h:773:23
    frame #87: 0x0000000100f1680c duckdb`std::__1::shared_ptr<duckdb::PreparedStatementData>::~shared_ptr[abi:ue170006](this=0x000000011c055190) at shared_ptr.h:771:5
    frame #88: 0x0000000108ea4258 duckdb`duckdb::PreparedStatement::~PreparedStatement(this=0x000000011c055180) at prepared_statement.cpp:19:1
    frame #89: 0x0000000108ea42cc duckdb`duckdb::PreparedStatement::~PreparedStatement(this=0x000000011c055180) at prepared_statement.cpp:18:41
    frame #90: 0x00000001002722e4 duckdb`std::__1::default_delete<duckdb::PreparedStatement>::operator()[abi:ue170006](this=0x000000011bd0b460, __ptr=0x000000011c055180) const at unique_ptr.h:68:5
    frame #91: 0x0000000100271d64 duckdb`std::__1::unique_ptr<duckdb::PreparedStatement, std::__1::default_delete<duckdb::PreparedStatement>>::reset[abi:ue170006](this=0x000000011bd0b460, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #92: 0x0000000100272c18 duckdb`std::__1::unique_ptr<duckdb::PreparedStatement, std::__1::default_delete<duckdb::PreparedStatement>>::~unique_ptr[abi:ue170006](this=0x000000011bd0b460) at unique_ptr.h:266:75
    frame #93: 0x0000000100272ba8 duckdb`duckdb::unique_ptr<duckdb::PreparedStatement, std::__1::default_delete<duckdb::PreparedStatement>, true>::~unique_ptr(this=0x000000011bd0b460) at unique_ptr.hpp:13:7
    frame #94: 0x000000010023f874 duckdb`duckdb::unique_ptr<duckdb::PreparedStatement, std::__1::default_delete<duckdb::PreparedStatement>, true>::~unique_ptr(this=0x000000011bd0b460) at unique_ptr.hpp:13:7
    frame #95: 0x000000010027d9a0 duckdb`sqlite3_stmt::~sqlite3_stmt(this=0x000000011bd0b440) at sqlite3_api_wrapper.cpp:49:8
    frame #96: 0x00000001002507bc duckdb`sqlite3_stmt::~sqlite3_stmt(this=0x000000011bd0b440) at sqlite3_api_wrapper.cpp:49:8
    frame #97: 0x000000010024a890 duckdb`duckdb_shell_sqlite3_finalize(pStmt=0x000000011bd0b440) at sqlite3_api_wrapper.cpp:746:3
    frame #98: 0x0000000100112c5c duckdb`shell_exec(pArg=0x000000016fdfd540, zSql="SELECT count(distinct column0) from read_csv('foo |');", pzErrMsg=0x000000016fdfcba0) at shell.c:13098:13
    frame #99: 0x00000001001d2524 duckdb`runOneSqlLine(p=0x000000016fdfd540, zSql="SELECT count(distinct column0) from read_csv('foo |');", in=0x0000000000000000, startline=1) at shell.c:19499:8
    frame #100: 0x0000000100115efc duckdb`process_input(p=0x000000016fdfd540) at shell.c:19610:17
    frame #101: 0x00000001000ccbb8 duckdb`main(argc=1, argv=0x000000016fdfebb8) at shell.c:20432:12
    frame #102: 0x00000001990560e0 dyld`start + 2360

To Reproduce

With the shellfs extension installed just run:

SELECT count(distinct column0) from read_csv('foo |');

OS:

Mac OS X

DuckDB Version:

v0.10.2

DuckDB Client:

CLI

Full Name:

Rusty Conover

Affiliation:

n/a

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?

Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

Yes, I have

The text was updated successfully, but these errors were encountered:

carlopi · 2024-05-20T19:51:32Z

Super cool extension!

I don't think there is any real way to do this at the moment, but for logging the error to some structure your extension owns, and then say rethrow at a later point that you control.

rustyconover · 2024-05-20T20:36:47Z

Thank you @carlopi, it feels really good that you liked the extension. I appreciate your compliment! DuckDB is awesome.

The problem is that the popen()'ed process's error should be associated with the executing query. So deferring notification to the user is kind of hard to do, but I'm going to think about your suggestion.

carlopi · 2024-05-20T20:41:43Z

I don't see exactly either, it can be possibly hacked on, say by rewriting the query to be the regular query + a shellfs callback/cleanup, or something equivalent, but query execution is NOT my area, someone more at ease will pass with some feedback.

rustyconover · 2024-05-21T15:00:06Z

If the pclose() is called when the read() returns zero, any error reported are properly raised in the context of query execution.

rustyconover added the needs triage label May 20, 2024

rustyconover closed this as completed May 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FileHandles aren't closed outside of C++ destructors - leading to unhandled I/O exceptions. #12146

FileHandles aren't closed outside of C++ destructors - leading to unhandled I/O exceptions. #12146

rustyconover commented May 20, 2024

carlopi commented May 20, 2024

rustyconover commented May 20, 2024

carlopi commented May 20, 2024

rustyconover commented May 21, 2024

FileHandles aren't closed outside of C++ destructors - leading to unhandled I/O exceptions. #12146

FileHandles aren't closed outside of C++ destructors - leading to unhandled I/O exceptions. #12146

Comments

rustyconover commented May 20, 2024

What happens?

To Reproduce

OS:

DuckDB Version:

DuckDB Client:

Full Name:

Affiliation:

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

Did you include all relevant data sets for reproducing the issue?

Did you include all code required to reproduce the issue?

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

carlopi commented May 20, 2024

rustyconover commented May 20, 2024

carlopi commented May 20, 2024

rustyconover commented May 21, 2024