Skip to content

feat(c/driver_manager): add new function to allow loading by manifest #2918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 63 commits into from
Jun 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
aabf3a2
feat(c/driver_manager): add new function to allow loading by manifest
zeroshade Jun 3, 2025
5e3316d
vendor tomlplusplus
zeroshade Jun 3, 2025
1d9c2ba
fix linting
zeroshade Jun 3, 2025
78e7760
fix python build
zeroshade Jun 3, 2025
8ce444e
make windows happier
zeroshade Jun 3, 2025
0742a7b
fix meson build vendored dir
zeroshade Jun 3, 2025
ee87f96
windows is annoying
zeroshade Jun 3, 2025
7622405
use vendor dir for R?
zeroshade Jun 3, 2025
a765978
goddamit windows
zeroshade Jun 3, 2025
4e5a4f8
wchar everywhere makes me sad
zeroshade Jun 3, 2025
1aaaa41
use c++17 and fix python manifest
zeroshade Jun 3, 2025
05ae88c
forgot to update go driver manager with windows fixes
zeroshade Jun 3, 2025
5f62d5e
more windows fixes
zeroshade Jun 3, 2025
4c7f0ef
linting
zeroshade Jun 3, 2025
451b512
Update c/driver_manager/adbc_driver_manager.cc
zeroshade Jun 4, 2025
cce58cd
updates from feedback
zeroshade Jun 4, 2025
65354ed
updates from feedback
zeroshade Jun 5, 2025
ab9ccfb
utf8 encode
zeroshade Jun 5, 2025
70f3dd4
fix str conversion
zeroshade Jun 5, 2025
fa74bea
windows, maybe?
zeroshade Jun 5, 2025
44864bc
use architecture triples
zeroshade Jun 6, 2025
caa440f
protect musl check
zeroshade Jun 6, 2025
764375e
linting
zeroshade Jun 6, 2025
377420d
fix windows
zeroshade Jun 6, 2025
f7e0940
fix finding the symbol for load
zeroshade Jun 6, 2025
46a2f1e
fix windows default driver load
zeroshade Jun 6, 2025
c5ce546
update current_arch based on duckdb learnings
zeroshade Jun 10, 2025
5dac8d1
fix lint
zeroshade Jun 10, 2025
0101fd3
Link to stdc++fs/c++fs with old g++/clang++
kou Jun 11, 2025
8ebadce
initial set of tests for manifests
zeroshade Jun 16, 2025
e53c794
pre-commit
zeroshade Jun 16, 2025
2413c5f
fix python build
zeroshade Jun 16, 2025
85e720c
add windows.h for driver_manager_test
zeroshade Jun 16, 2025
8c517eb
missed the python manifest include
zeroshade Jun 17, 2025
0ac0ba0
updated based on comments
zeroshade Jun 17, 2025
3dad8c3
fix test
zeroshade Jun 17, 2025
f91f8a1
trailing whitespace
zeroshade Jun 17, 2025
eb555f1
use convention for Utf8Encode/Decode
zeroshade Jun 18, 2025
f7416c7
ensure we ignore io.EOF in flightsql driver
zeroshade Jun 18, 2025
c5ab1da
add more tests
zeroshade Jun 18, 2025
03f3f49
fix python 3.13 windows build
zeroshade Jun 18, 2025
5030e44
system-level test with sudo
zeroshade Jun 18, 2025
6a986e9
why is postgres tests being weird
zeroshade Jun 18, 2025
04de8c4
lint
zeroshade Jun 18, 2025
d0e6e07
Update .github/workflows/native-unix.yml
zeroshade Jun 19, 2025
7ace53a
Update c/driver_manager/adbc_driver_manager_test.cc
zeroshade Jun 19, 2025
d1cfb5e
more updates from comments
zeroshade Jun 19, 2025
f54694b
linting
zeroshade Jun 19, 2025
b773074
Update c/cmake_modules/AdbcDefines.cmake
zeroshade Jun 19, 2025
fb7ca7e
fix windows build
zeroshade Jun 19, 2025
c91ef38
use `shlobj.h` instead of `shlobj_core.h` for R build
zeroshade Jun 19, 2025
188fd4b
windows libs
zeroshade Jun 19, 2025
535bae3
possibly fix R build
zeroshade Jun 19, 2025
d00d8c0
Makevars.win R build
zeroshade Jun 19, 2025
2dcc385
needs -luuid
zeroshade Jun 19, 2025
7e89beb
add link args for python
zeroshade Jun 19, 2025
4a60ac4
fix python windows build
zeroshade Jun 19, 2025
aa73891
add test with non-ascii filepath
zeroshade Jun 19, 2025
efa2ee7
Update c/include/arrow-adbc/adbc_driver_manager.h
zeroshade Jun 20, 2025
96504f4
Update c/include/arrow-adbc/adbc_driver_manager.h
zeroshade Jun 20, 2025
12feba7
update options for consistency
zeroshade Jun 20, 2025
60d0d52
propagate updated header
zeroshade Jun 20, 2025
480c249
updates from comments
zeroshade Jun 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/native-unix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,7 @@ jobs:
BUILD_DRIVER_MANAGER: "1"
BUILD_DRIVER_POSTGRESQL: "1"
BUILD_DRIVER_SQLITE: "1"
ADBC_DRIVER_MANAGER_USER_CONFIG_TEST: "1"
run: |
./ci/scripts/cpp_build.sh "$(pwd)" "$(pwd)/build"
- name: Test
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/native-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ jobs:
BUILD_DRIVER_BIGQUERY: "0"
BUILD_DRIVER_FLIGHTSQL: "0"
BUILD_DRIVER_SNOWFLAKE: "0"
BUILD_DRIVER_MANAGER_USER_CONFIG_TEST: "1"
run: |
.\ci\scripts\cpp_build.ps1 $pwd ${{ github.workspace }}\build

Expand Down Expand Up @@ -149,6 +150,7 @@ jobs:
env:
BUILD_ALL: "0"
BUILD_DRIVER_MANAGER: "1"
BUILD_DRIVER_MANAGER_USER_CONFIG_TEST: "1"
run: .\ci\scripts\cpp_build.ps1 $pwd $pwd\build
- name: Build Driver PostgreSQL
env:
Expand All @@ -164,6 +166,7 @@ jobs:
env:
BUILD_ALL: "0"
BUILD_DRIVER_MANAGER: "1"
BUILD_DRIVER_MANAGER_USER_CONFIG_TEST: "1"
run: .\ci\scripts\cpp_test.ps1 $pwd\build
- name: Test Driver SQLite
env:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/packaging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -907,7 +907,7 @@ jobs:
shell: cmd
run: |
where python.exe
CALL "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvars64.bat"
CALL "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
pushd adbc
set SETUPTOOLS_SCM_PRETEND_VERSION=%VERSION%
.\ci\scripts\python_wheel_windows_build.bat %cd% %cd%\build
Expand Down
25 changes: 25 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,31 @@ public domain, released using the CC0 1.0 Universal dedication (*).

--------------------------------------------------------------------------------

The files in c/vendor/toml++ contain code from

https://github.com/marzer/tomlplusplus

and have the following copyright notice:

MIT License

Copyright (c) Mark Gillard <mark.gillard@outlook.com.au>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

--------------------------------------------------------------------------------

The files python/*/*/_version.py and python/*/*/_static_version.py
contain code from

Expand Down
5 changes: 3 additions & 2 deletions c/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,15 @@
cmake_minimum_required(VERSION 3.18)
get_filename_component(REPOSITORY_ROOT ".." ABSOLUTE)
list(APPEND CMAKE_MODULE_PATH "${REPOSITORY_ROOT}/c/cmake_modules/")
include(AdbcDefines)
include(BuildUtils)
include(AdbcVersion)
project(adbc
VERSION "${ADBC_BASE_VERSION}"
LANGUAGES C CXX)
set(CMAKE_C_STANDARD 99)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
include(AdbcDefines)
include(BuildUtils)

include(CTest)

Expand Down
3 changes: 2 additions & 1 deletion c/cmake_modules/AdbcDefines.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ enable_language(C CXX)

set(BUILD_SUPPORT_DIR "${REPOSITORY_ROOT}/ci/build_support")

include(AdbcVersion)
include(CheckLinkerFlag)
include(DefineOptions)
include(GNUInstallDirs) # Populates CMAKE_INSTALL_INCLUDEDIR
Expand Down Expand Up @@ -92,6 +91,8 @@ if(MSVC)
add_compile_options(/wd4711)
# Don't warn about padding added after members
add_compile_options(/wd4820)
# Don't warn about enforcing left-to-right evaluation order for operator[]
add_compile_options(/wd4866)
add_compile_options(/wd5027)
add_compile_options(/wd5039)
add_compile_options(/wd5045)
Expand Down
2 changes: 1 addition & 1 deletion c/cmake_modules/BuildUtils.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ function(ADD_ARROW_LIB LIB_NAME)

if(BUILD_SHARED)
add_library(${LIB_NAME}_shared SHARED ${LIB_DEPS})
target_compile_features(${LIB_NAME}_shared PRIVATE cxx_std_11)
target_compile_features(${LIB_NAME}_shared PRIVATE cxx_std_17)
set_property(TARGET ${LIB_NAME}_shared PROPERTY CXX_STANDARD_REQUIRED ON)
adbc_configure_target(${LIB_NAME}_shared)
if(EXTRA_DEPS)
Expand Down
6 changes: 6 additions & 0 deletions c/cmake_modules/DefineOptions.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,12 @@ if("${CMAKE_SOURCE_DIR}" STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
"shared"
"static")

define_option(ADBC_DRIVER_MANAGER_TEST_MANIFEST_USER_LEVEL
"Build driver manager manifest user-level tests" OFF)

define_option(ADBC_DRIVER_MANAGER_TEST_MANIFEST_SYSTEM_LEVEL
"Build driver manager manifest system-level tests" OFF)

#----------------------------------------------------------------------
set_option_category("Lint")

Expand Down
46 changes: 44 additions & 2 deletions c/driver_manager/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,34 @@
# specific language governing permissions and limitations
# under the License.

set(ADBC_DRIVER_MANAGER_SHARED_LINK_LIBS ${CMAKE_DL_LIBS})
set(ADBC_DRIVER_MANAGER_STATIC_LINK_LIBS ${CMAKE_DL_LIBS})
# std::filesystem with old g++/clang++ require libstdc++fs/libc++fs.
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS "9")
list(APPEND ADBC_DRIVER_MANAGER_SHARED_LINK_LIBS stdc++fs)
list(APPEND ADBC_DRIVER_MANAGER_STATIC_LINK_LIBS stdc++fs)
endif()
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS "8")
list(APPEND ADBC_DRIVER_MANAGER_SHARED_LINK_LIBS c++fs)
list(APPEND ADBC_DRIVER_MANAGER_STATIC_LINK_LIBS c++fs)
endif()
endif()

if(WIN32)
list(APPEND
ADBC_DRIVER_MANAGER_SHARED_LINK_LIBS
advapi32
uuid
shell32)
list(APPEND
ADBC_DRIVER_MANAGER_STATIC_LINK_LIBS
advapi32
uuid
shell32)
endif()

add_arrow_lib(adbc_driver_manager
SOURCES
adbc_driver_manager.cc
Expand All @@ -25,9 +53,9 @@ add_arrow_lib(adbc_driver_manager
PKG_CONFIG_NAME
adbc-driver-manager
SHARED_LINK_LIBS
${CMAKE_DL_LIBS}
${ADBC_DRIVER_MANAGER_SHARED_LINK_LIBS}
STATIC_LINK_LIBS
${CMAKE_DL_LIBS}
${ADBC_DRIVER_MANAGER_STATIC_LINK_LIBS}
SHARED_LINK_FLAGS
${ADBC_LINK_FLAGS})

Expand Down Expand Up @@ -62,6 +90,20 @@ if(ADBC_BUILD_TESTS)
adbc_validation
${TEST_LINK_LIBS})
target_compile_features(adbc-driver-manager-test PRIVATE cxx_std_17)

if(ADBC_DRIVER_SQLITE)
target_compile_definitions(adbc-driver-manager-test
PRIVATE ADBC_DRIVER_MANAGER_TEST_LIB="${CMAKE_BINARY_DIR}/driver/sqlite/libadbc_driver_sqlite${CMAKE_SHARED_LIBRARY_SUFFIX}"
)
endif()
if(ADBC_DRIVER_MANAGER_TEST_MANIFEST_USER_LEVEL)
target_compile_definitions(adbc-driver-manager-test
PRIVATE ADBC_DRIVER_MANAGER_TEST_MANIFEST_USER_LEVEL=1)
endif()
if(ADBC_DRIVER_MANAGER_TEST_MANIFEST_SYSTEM_LEVEL)
target_compile_definitions(adbc-driver-manager-test
PRIVATE ADBC_DRIVER_MANAGER_TEST_MANIFEST_SYSTEM_LEVEL=1)
endif()
target_include_directories(adbc-driver-manager-test SYSTEM
PRIVATE ${REPOSITORY_ROOT}/c/ ${REPOSITORY_ROOT}/c/include/
${LIBPQ_INCLUDE_DIRS} ${REPOSITORY_ROOT}/c/driver)
Expand Down
Loading
Loading