feat(pxq): Stage A skeleton — PXF → gojq → PXF (loose mode) - #32
Merged
Conversation
End-to-end spine for the jq-style query tool documented in cmd/pxq/README.md. Stage A is loose-mode-only with a single input adapter (PXF); JSON/YAML/CSV adapters, schema-bound strict mode, the @pxf.* extension namespace, the @proto(...) constructor, and the infer-schema subcommand land in follow-up stages. The spine has three files: * load.go — pxf.Parse → gojq's untyped map[string]any / []any graph. Loose-mode type inference per the README: INT→int64 (or *big.Int on overflow), FLOAT→float64, BYTES→base64 string with a "b" prefix so the emitter can round-trip it, BOOL/NULL/IDENT mapped to native Go types. Document-level @type / @dataset / @proto / @<name> directives are flattened onto synthetic "__pxf_*" keys so a raw jq query can reach them; Stage C will swap these for the @pxf.directive(name) extension surface. * run.go — gojq embedding. Errors-as-null per the README: jq runtime errors degrade to nil with a one-line hint to stderr. Compile-time errors in the query string itself propagate to the caller. * emit.go — untyped graph → PXF text. Schema-less emission walks the any graph and picks the natural PXF token; bytes round-trip back as b"..."; nested blocks indent two spaces. Top-level results that aren't a map are wrapped as `value = <v>` so the result stays a valid PXF document (top level requires field_entry, not bare scalars). Dependencies: bumps protowire-go 0.75.0 → 1.0.0 to consume the v1.0 AST surface (Document.Datasets / Protos / Directives), adds gojq v0.12.19. Validation: * 11 Go tests cover field access, list/length, nested blocks, @dataset rows (filter + empty/null cells), @proto directive exposure, loose-mode runtime-error fallback, bytes round-trip, and big.Int round-trip. * End-to-end smoke against the README's quick-start example: pxq '.__pxf_datasets[0].rows | map(select(.symbol == "AAPL")) | length' yields 2 against a 3-row @dataset. This is the smallest interesting demo of the design. Follow-ups: Stage B — JSON / YAML / CSV input adapters (loose mode) Stage C — schema layer, strict mode, @pxf.* extensions, @proto(...) Stage D — `pxq infer-schema` subcommand
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implementation Stage A for
pxq, the jq-style query tool whosedesign was landed in #30 / #31. End-to-end spine — read a PXF file,
run a jq query, emit PXF — in loose mode with PXF as the only
input adapter. JSON/YAML/CSV adapters, strict mode, the
@pxf.*extension namespace, the
@proto(...)constructor, and theinfer-schemasubcommand all land in follow-up stages.Why bundled this way
The README design covers a broad surface (4 input formats, 2 modes,
schema chain, gojq integration). A single end-to-end PR would be too
large to review carefully. This stage demonstrates the minimum
interesting demo — a jq query against a real PXF document with
@datasetrows working — so the moving parts are exercised togetherbefore any one part grows.
What's here
cmd/pxq/main.go<query> <file>argscmd/pxq/load.gopxf.Parse→ gojq's untypedanygraph (loose-mode rules per README)cmd/pxq/run.gocmd/pxq/emit.gocmd/pxq/main_test.goLoose-mode rules (from the README)
INT→int64if it fits, else*big.IntFLOAT→float64BYTES→ base64 string with"b"prefix (round-trips through emit asb"...")@type/@dataset/@proto/@<name>→ exposed on synthetic__pxf_*top-level keys for raw jq access; Stage C swaps these forthe
@pxf.directive(name)extension surfacenull(jq-compatible)Demo
Once Stage C lands, the
__pxf_datasetssynthetic key becomes@pxf.directive("dataset")— the README's documented form.Dependencies
protowire-go0.75.0 → 1.0.0 (consumes the v1.0 AST surface:Document.Datasets / .Protos / .Directives)github.com/itchyny/gojqv0.12.19Test plan
go build ./...cleango test ./...— 11 newcmd/pxqtests pass; existinginternal/pxfschematests unaffected