Skip to content

--where: reserved-word and dotted column names are unaddressable, and the backtick rule is documented too narrowly #59

Description

@TMYTiMidlY

Edited. The original version proposed dropping dot-path splitting from --where, on the assumption that nested paths were unreachable in the current single-command surface. That was wrong — string .length resolves through src/lib/expr.ts:166-172 and is pinned by test/expr.test.ts:41-44 ("dot paths and .length"), so removing it would be a regression. That recommendation, and the accompanying speculation about the project's history, have been removed. A reserved-word repro was added and several line citations corrected. The underlying reports are unchanged.

Reproduction

ax 0.1.23, Linux x86_64 (WSL2).

1. A header literally named foo.bar is read as a nested path

printf '%s' '<table><tr><th>foo.bar</th></tr><tr><td>x</td></tr></table>' > dot.html

ax dot.html table --table --where 'foo.bar ~ /x/'
# ax: note: 0 of 1 rows match --where          <- false negative

ax dot.html table --table --where 'foo.bar == null'
# ax: note: 1 rows extracted, no empty fields
# foo.bar
# x                                             <- false positive

ax dot.html table --table --where '`foo.bar` ~ /x/'
# correct: 1 row

The false negative is indistinguishable from a genuine zero-match, and the false positive arrives with a clean N rows extracted, no empty fields note.

Reachable through user-chosen field names too, not just scraped headers:

ax dot.html 'td' --row 'a.b=' --where 'a.b ~ /x/'
# ax: note: 0 of 1 rows match --where

2. Headers named true / false / null are unaddressable

printf '%s' '<table><tr><th>true</th><th>n</th></tr><tr><td>yes</td><td>1</td></tr></table>' > kw.html

ax kw.html table --table --where 'true ~ /yes/'
# ax: note: 0 of 1 rows match --where

ax kw.html table --table --where '`true` ~ /yes/'
# ax: note: 0 of 1 rows match --where     <- backticks don't help

ax kw.html table --table --where 'n == 1'
# works, so the row is otherwise reachable

expr.ts:121-122 converts these to literals, and because quoted and bare names share one token type (:69-73), backticking does not exempt them. There is currently no way to reference such a column.

3. Non-ASCII headers error rather than mismatch

printf '%s' '<table><tr><th>名称</th></tr><tr><td>甲</td></tr></table>' > cjk.html

ax cjk.html table --table --where '名称 ~ /甲/'
# ax: error: cannot parse expression near: 名称 ~ /甲/

ax cjk.html table --table --where '`名称` ~ /甲/'
# correct: 1 row

Docs describe backticks as being for spaces only

The bare-identifier rule (src/lib/expr.ts:82) is /^[A-Za-z_$][\w$]*(\.[A-Za-z_$][\w$]*)*/, so a bare reference must be an ASCII identifier, optionally dotted. But:

  • src/commands/root.ts:56`col name` for headers with spaces
  • src/agent-context.txt:43backtick column names with spaces: ...
  • src/lib/expr.ts:69/* quoted column name — for headers with spaces */
  • skills/ax/SKILL.md:24 — only demonstrates `Col With Spaces`

test/expr.test.ts:58-61 does cover a punctuated header (Change(%)) with backticks, so punctuation is known to need them — but the docs never say so, and there is no coverage for slashes, hyphens, leading digits, non-ASCII names, or dotted names.

Expected

  1. Give backtick-quoted names their own token type so they skip the literal conversion at expr.ts:121-122, making `true` / `false` / `null` addressable.

  2. For dotted names, consider preferring a literal column match over path resolution (look up row[raw] first, fall back to the path) — .length and genuine nested access would keep working, while a header literally named foo.bar would resolve as intended. If that ambiguity is better left to the user, a note when both readings apply would still remove the silent case.

  3. Generalize the documented rule, e.g.:

    Bare column references must be ASCII identifiers ([A-Za-z_$][A-Za-z0-9_$]*), and a dot in a bare reference means path traversal (name.length works). Backtick any other literal column name — names with spaces, punctuation, leading digits, non-ASCII characters, or dots. true, false and null are reserved and cannot currently be used as column names.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions