Skip to content

select with take: returns empty strings for pool-backed STR values #404

Description

@alexey-milovidov

Summary

Any select with a take: clause returns empty strings for STR values
that live in the column's byte pool (i.e. longer than the 12-byte inline
limit). The values are correct without take:, and correct again if the query
also sorts (asc: / desc:). Since take: is how LIMIT is expressed, every
limited query over a wide-string column is affected.

Reproduction

(set t (table [s v] (list ["long-string-value-zero" "long-string-value-one" "long-string-value-two"] [0 1 2])))
(println (format "where, no take     -> [%]" (at (at (select {from: t s: s where: (> v 0)}) 's) 0)))
(println (format "where + take       -> [%]" (at (at (select {from: t s: s where: (> v 0) take: 1}) 's) 0)))
(println (format "take, no where     -> [%]" (at (at (select {from: t s: s take: 1}) 's) 0)))
(println (format "where + take + asc -> [%]" (at (at (select {from: t s: s where: (> v 0) asc: s take: 1}) 's) 0)))
(println (format "identity + take    -> [%]" (at (at (select {from: t take: 1}) 's) 0)))
(exit 0)

Output:

where, no take     -> [long-string-value-one]
where + take       -> []
take, no where     -> []
where + take + asc -> [long-string-value-one]
identity + take    -> []

So (select {from: t take: 1}) — no filter, no projection, just a row limit —
already loses the string.

Expected

take: slices rows; it should not change their values. The STR column should
carry the same strings the unlimited query returns.

Notes

  • Only pool-backed values are lost. Strings of 12 bytes or fewer, stored inline
    in ray_str_t, survive take: — which is why this can hide in tables of
    short strings.
  • Reproduces identically on an in-memory table, on a table round-tripped
    through .db.splayed.set / .db.splayed.get, and on a splayed table written
    by .csv.splayed, at every row count tried (3 to 2M).
  • strlen over the same column in the same query returns the correct lengths,
    so the descriptors survive the slice. It looks like the per-vector pool is
    simply not carried over to the sliced result
    (col_propagate_str_pool).
  • The result is silently wrong rather than an error, which is what makes it
    worth prioritising: a LIMIT query over a URL or title column just reports
    empty strings.

Versions

Reproduced on v2.5.14 (release tag, make release) and on dev at
95defb7aef2ff0863e70ab3b79785036cd529b1d, Linux aarch64.

Found while porting Rayforce to
ClickBench.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions