Skip to content

Add Arrow C Data Interface support with defensive resource management #594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
robertbuessow wants to merge 11 commits into
base: main
Choose a base branch
from
+2,130 −1
Open

Add Arrow C Data Interface support with defensive resource management #594

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
34145c9
Add Arrow C Data Interface support (#593)
robertbuessow Apr 8, 2026
7f7b72d
Merge branch 'main' of github.com:apache/arrow-julia
robertbuessow May 11, 2026
781579c
Fix double-free and type narrowing in C Data Interface import
robertbuessow May 11, 2026
feece16
Fix BoundsError when dict-encoding CategoricalArrays with missing values
robertbuessow May 12, 2026
ad641dd
cleanup
robertbuessow May 14, 2026
b2406e3
Add comprehensive C Data Interface tests and fix import bugs
robertbuessow May 15, 2026
5dee1f0
formatting
robertbuessow May 19, 2026
36829d9
Add finalizer to CDataHandle to detect missing release_c_data calls
robertbuessow May 21, 2026
d3b6e73
release handle in test
robertbuessow May 21, 2026
b700090
Verify ArrowSchema/ArrowArray field offsets against C compiler
robertbuessow May 21, 2026
d7ad6b6
Guard CImportedArray getindex against use-after-release
robertbuessow May 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion src/Arrow.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ This implementation supports the 1.0 version of the specification, including sup
It currently doesn't include support for:
* Tensors or sparse tensors
* Flight RPC
* C data interface

Third-party data formats:
* csv and parquet support via the existing [CSV.jl](https://github.com/JuliaData/CSV.jl) and [Parquet.jl](https://github.com/JuliaIO/Parquet.jl) packages
Expand All @@ -56,6 +55,8 @@ using DataAPI,
StringViews

export ArrowTypes
export ArrowSchema,
ArrowArray, CImportedArray, CImportedTable, from_c_data, to_c_data, release_c_data

using Base: @propagate_inbounds
import Base: ==
Expand All @@ -79,6 +80,7 @@ include("table.jl")
include("write.jl")
include("append.jl")
include("show.jl")
include("cdatainterface.jl")

const ZSTD_COMPRESSOR = Lockable{ZstdCompressor}[]
const ZSTD_DECOMPRESSOR = Lockable{ZstdDecompressor}[]
Expand Down Expand Up @@ -138,6 +140,10 @@ function __init__()
resize!(empty!(ZSTD_COMPRESSOR), nt)
resize!(empty!(LZ4_FRAME_DECOMPRESSOR), nt)
resize!(empty!(ZSTD_DECOMPRESSOR), nt)
global _SCHEMA_RELEASE_CFUNC =
@cfunction(_release_exported_schema, Cvoid, (Ptr{ArrowSchema},))
global _ARRAY_RELEASE_CFUNC =
@cfunction(_release_exported_array, Cvoid, (Ptr{ArrowArray},))
return
end

Expand Down
Loading