Materialize into Apache Arrow through ADBC, using psqlodbc 16 (adbcBridge 0.1.0) #38563
singhpratech
started this conversation in
Technical musings
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
adbcBridge is a small Apache-2.0 ADBC driver written in C11 that loads an ODBC driver and exposes it through the Arrow ADBC C ABI: block-cursor reads straight into Arrow record batches, bulk ingest, metadata, partitioned parallel reads. I ran Materialize through it as one of 46 databases in a single compatibility workload and wanted to share the entry here.
What was verified (Materialize 26.38; driver: psqlodbc 16)
What the compatibility entry records: PostgreSQL SQL layer, so no driver quirks — but no
SAVEPOINT, so the entry sets psqlodbc'sProtocol=7.4-0to stop the driver wrapping the second batch of a large ingest in one; its single 39-digitNUMERICis wider than an Arrow decimal128, so decimals read back as exact strings; the entry also ingests into and reads back an incrementally maintainedMATERIALIZED VIEW; ingest 23.6k rows/s, fetch 322k rows/s. Full entry: docs/COMPATIBILITY.md.The
postgresqlADBC driver may also work with Materialize over its wire protocol; this path goes through psqlodbc and records exactly what it needed, so it is a second, documented route rather than a replacement.Trying it (Python; Rust, Go, Java and C# are on the docs site):
Links: repository · docs · upstream notes · PyPI
It is a 0.1.0. If the entry says something wrong about Materialize, or you run a version or driver I didn't, an issue on the repository with the details is the most useful thing you could send.
All reactions