Repository navigation
Releases: underloam/xqdb
Release list
v0.1.8
Fixed — empty dictionaries round-trip
An empty q dictionary decoded to {} on Python and Node, but sending that {}
back failed with Not supported empty dictionary, so a value the library handed
out could not be passed back as a query argument.
{} now serializes as (`symbol$())!() — the symbol-keyed frame this
library already emits for every non-empty dictionary and the frame q itself
produces for an emptied symbol dictionary — and nested empty dictionaries
serialize the same way. The wire form is 99 11 0 <0 keys> 0 0 <0 values>, 13
bytes, matching the length pass exactly.
Decoding also accepts q's canonical empty literal ()!(), whose keys are an
empty general list rather than an empty symbol list. It used to fail with
Only support symbol keys dictionary or keyed table, got k type 0; it now
decodes to {} alongside (`symbol$())!() and typed-value forms such as
0#`a`b!1 2. A general list that holds any keys is still rejected with the
same error, and a dictionary frame truncated before its key type or its values
now reports a deserialization error instead of an internal parser panic.
v0.1.7
Added — opt-in lossy decoding for q text that is not valid UTF-8
q stores symbols and strings as raw bytes and never validates them, while Arrow
string columns and Rust strings must be valid UTF-8. Until now a single stray
Latin-1 or binary byte anywhere in a response failed the whole result:
"q symbol-list value is not valid UTF-8: invalid utf-8 sequence of 1 bytes from index 3"
A connection can now opt in to transcoding instead. The policy is
symbolEncoding on Node (QOptions and readBinary6(path, { symbolEncoding }))
and symbol_encoding on Python (Q(...), the Q.symbol_encoding property, and
read_binary6(path, symbol_encoding=...)), accepting "strict" or "lossy":
"strict"(default) keeps the existing behaviour: the first invalid byte
sequence fails the response with an error naming the offending value."lossy"replaces each maximal invalid subsequence with U+FFFD REPLACEMENT
CHARACTER — the same substitutionString::from_utf8_lossyperforms — so
every other value in the response survives intact. A Latin-1caf\xe9decodes
ascaf\ufffd.
The policy applies to every place wire text becomes a string: symbol atoms,
symbol lists and columns, nested symbol lists, char columns, string columns,
lambda source and context, dictionary keys, and table column names. String
column rows are transcoded individually, so a multi-byte code point split across
two rows is replaced in both rather than silently validated as a whole. Valid
text decodes identically under both policies.
Any other value ("latin1", "Lossy", …) is rejected at construction: a
RangeError on Node, a ValueError on Python, and XQDB_CONVERSION from the
native readBinary6.
Changed
- q error messages are now always decoded lossily. A server error whose text
contained a non-UTF-8 byte used to surface as a deserialization failure that
hid the actual error; it now surfaces as the server error with replacement
characters, under either policy. - Symbol lists are validated once instead of twice: the pass that measures a
list's wire length no longer decodes each symbol, leaving that to the pass that
materializes it. Strict-mode errors for invalid symbols are unchanged in type
and still name the offending value.
Every buffer the lossy path allocates is sized from wire data and reserved
fallibly, in line with 0.1.5 and 0.1.6.
Note on 0.1.6
The 0.1.6 npm release built and validated successfully but was never approved
for publication, so @xbbg/xqdb@0.1.6 does not exist on npm; that run has been
cancelled in favour of this release. 0.1.7 contains everything 0.1.6 did.
Python users upgrading from 0.1.3
This is the first PyPI release since 0.1.3, so it also delivers the 0.1.4
through 0.1.6 changes. One of them is breaking:
q timestamp atoms are naive. 0.1.3 returned UTC-aware datetime values;
0.1.4 and later return naive values with tzinfo is None, matching Arrow
timestamp[ns] columns, so the same q value compares equal whether read as an
atom or out of a column and can be passed back as a query argument. If you need
an aware value, attach the zone yourself — and only when the data really is UTC,
because a process that stores .z.P holds local wall-clock times:
ts = conn.sync("first exec time from trade") # naive datetime
ts = ts.replace(tzinfo=timezone.utc)Query arguments accept both shapes: a naive datetime keeps its wall clock and
an aware one is normalized through its UTC offset.
The rest is additive: no fixed size ceiling on results, every wire-sized
allocation reports an error instead of aborting the interpreter, 40-bit framing
for messages above 4 GiB, and the dependency advisories patched in 0.1.4. See
docs/release-notes/v0.1.4.md through v0.1.6.md for the details.
v0.1.3
v0.1.2
XQDB v0.1.1
XQDB 0.1.0
First public XQDB release.\n\n- High-performance Rust q IPC core\n- Python bindings with Narwhals and Arrow C Stream exchange for PyArrow, pandas, and Polars\n- TypeScript/Node.js bindings with Apache Arrow tables\n- setuptools-rust Python builds with a CPython 3.10+ ABI3 wheel contract\n- Live q validation and cross-language benchmark harnesses\n\nThis source release preserves the upstream BSD-3-Clause attribution. Registry publication remains separately gated through the manual release workflow.