Skip to content

feat(pxq): Stage A skeleton — PXF → gojq → PXF (loose mode) - #32

Merged
trendvidia merged 1 commit into
mainfrom
feat/pxq-impl-a-skeleton
May 14, 2026
Merged

feat(pxq): Stage A skeleton — PXF → gojq → PXF (loose mode)#32
trendvidia merged 1 commit into
mainfrom
feat/pxq-impl-a-skeleton

Conversation

@trendvidia

Copy link
Copy Markdown
Owner

Summary

Implementation Stage A for pxq, the jq-style query tool whose
design was landed in #30 / #31. End-to-end spine — read a PXF file,
run a jq query, emit PXF — in loose mode with PXF as the only
input adapter
. JSON/YAML/CSV adapters, strict mode, the @pxf.*
extension namespace, the @proto(...) constructor, and the
infer-schema subcommand all land in follow-up stages.

Why bundled this way

The README design covers a broad surface (4 input formats, 2 modes,
schema chain, gojq integration). A single end-to-end PR would be too
large to review carefully. This stage demonstrates the minimum
interesting demo — a jq query against a real PXF document with
@dataset rows working — so the moving parts are exercised together
before any one part grows.

What's here

File Purpose
cmd/pxq/main.go cobra entrypoint; positional <query> <file> args
cmd/pxq/load.go pxf.Parse → gojq's untyped any graph (loose-mode rules per README)
cmd/pxq/run.go gojq embedding; loose-mode errors-as-null with stderr hints
cmd/pxq/emit.go untyped graph → PXF text (schema-less best-effort)
cmd/pxq/main_test.go 11 tests, including the README's quick-start filter example

Loose-mode rules (from the README)

  • INTint64 if it fits, else *big.Int
  • FLOATfloat64
  • BYTES → base64 string with "b" prefix (round-trips through emit as b"...")
  • @type / @dataset / @proto / @<name> → exposed on synthetic
    __pxf_* top-level keys for raw jq access; Stage C swaps these for
    the @pxf.directive(name) extension surface
  • Runtime errors degrade to null (jq-compatible)

Demo

$ cat trades.pxf
@dataset trades.v1.Trade ( symbol, price, qty )
( "AAPL", 188.42, 100 )
( "MSFT", 415.10,  50 )
( "AAPL", 188.55,  75 )

$ pxq '.__pxf_datasets[0].rows | map(select(.symbol == "AAPL")) | length' trades.pxf
value = 2

Once Stage C lands, the __pxf_datasets synthetic key becomes
@pxf.directive("dataset") — the README's documented form.

Dependencies

  • protowire-go 0.75.0 → 1.0.0 (consumes the v1.0 AST surface:
    Document.Datasets / .Protos / .Directives)
  • New: github.com/itchyny/gojq v0.12.19

Test plan

  • go build ./... clean
  • go test ./... — 11 new cmd/pxq tests pass; existing
    internal/pxfschema tests unaffected
  • Smoke test against the README's quick-start example
  • CI passes on the matrix

End-to-end spine for the jq-style query tool documented in
cmd/pxq/README.md. Stage A is loose-mode-only with a single input
adapter (PXF); JSON/YAML/CSV adapters, schema-bound strict mode, the
@pxf.* extension namespace, the @proto(...) constructor, and the
infer-schema subcommand land in follow-up stages.

The spine has three files:

* load.go — pxf.Parse → gojq's untyped map[string]any / []any graph.
  Loose-mode type inference per the README: INT→int64 (or *big.Int on
  overflow), FLOAT→float64, BYTES→base64 string with a "b" prefix so
  the emitter can round-trip it, BOOL/NULL/IDENT mapped to native Go
  types. Document-level @type / @dataset / @proto / @<name> directives
  are flattened onto synthetic "__pxf_*" keys so a raw jq query can
  reach them; Stage C will swap these for the @pxf.directive(name)
  extension surface.
* run.go — gojq embedding. Errors-as-null per the README: jq runtime
  errors degrade to nil with a one-line hint to stderr. Compile-time
  errors in the query string itself propagate to the caller.
* emit.go — untyped graph → PXF text. Schema-less emission walks the
  any graph and picks the natural PXF token; bytes round-trip back as
  b"..."; nested blocks indent two spaces. Top-level results that
  aren't a map are wrapped as `value = <v>` so the result stays a
  valid PXF document (top level requires field_entry, not bare
  scalars).

Dependencies: bumps protowire-go 0.75.0 → 1.0.0 to consume the v1.0
AST surface (Document.Datasets / Protos / Directives), adds gojq
v0.12.19.

Validation:
* 11 Go tests cover field access, list/length, nested blocks,
  @dataset rows (filter + empty/null cells), @proto directive
  exposure, loose-mode runtime-error fallback, bytes round-trip, and
  big.Int round-trip.
* End-to-end smoke against the README's quick-start example:
    pxq '.__pxf_datasets[0].rows | map(select(.symbol == "AAPL")) | length'
  yields 2 against a 3-row @dataset.

This is the smallest interesting demo of the design. Follow-ups:
  Stage B — JSON / YAML / CSV input adapters (loose mode)
  Stage C — schema layer, strict mode, @pxf.* extensions, @proto(...)
  Stage D — `pxq infer-schema` subcommand
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant