docs(pxq): land design doc for the jq-style query tool - #30
Merged
Conversation
`pxq` is a `jq`-style command-line query tool whose core operates on PXF documents, with transparent input adapters for CSV, JSON, and YAML. Output is always PXF, so downstream pipeline stages stay format- agnostic regardless of where the data started. This commit lands the design doc only — implementation lives in a follow-up. Key decisions captured: * gojq embedded as the core engine; `@pxf.*` reserved namespace for query-time operations that have no jq equivalent * `@proto(<dotted>)` constructor for typed object output, with schema resolution sourcing from bundled schemas, `-p`, or protoregistry * Implicit strict/loose mode (schema present → strict, schema absent → loose) with `--strict` / `--loose` overrides * Fail-fast schema inference by default (`pxq infer-schema`); `--full- scan` aggregates every contradiction in a single pass * PXF/JSON/YAML/CSV adapters with explicit disambiguation rules for each known cross-format ambiguity (JSON int-vs-float, YAML implicit coercions, CSV empty-cell) * Self-describing PXF inputs (`@proto` directives) are honoured as if the embedded schema had been passed via `-p` Co-locates a one-paragraph pointer in the top-level README so the tool is discoverable from the project landing page.
3 tasks
trendvidia
added a commit
that referenced
this pull request
May 14, 2026
Three focused refinements based on the v1.0 spec freeze and review of the design doc landed in #30: * Add an up-front terminology note disambiguating the two `@proto` forms — the document-level directive (draft §3.4.5, schema flows in) and the query-level constructor (typed output, schema flows out). They share a name on purpose because they're complementary halves of the same protobuf-typing primitive, but a reader who only knows one spec layer would otherwise hit them as a name collision. * Restructure the dense PXF-adapter paragraph into a table of the four `@proto` body shapes (anonymous, named, source, descriptor) and add an explicit schema-resolution-order rule covering bundled schemas, in-document `@proto` directives, `-p`, and protoregistry. The order matters because user-supplied `-p` can shadow stale embedded schemas while still being overridable by protoregistry. * Add a "Quick start — self-describing PXF" section to Install with an end-to-end example: a `@proto`-headed PXF file that needs no flags, contrasted with the schema-external form. Closes the gap between "v1.0 introduced @proto" (the design says) and "what does the user actually type" (it didn't show). Also extends the `@pxf.directive` table row with the schema-bound row semantics introduced by the quick-start example, and clarifies the `@proto(...)` constructor's resolution chain to include in-document `@proto` directives as an additional source.
This was referenced May 14, 2026
This was referenced Jun 5, 2026
trendvidia
added a commit
that referenced
this pull request
Jun 5, 2026
Adds docs/M1-kickoff-protocompile.md — the concrete first-PR plan for protocompile's M1 work (extended grammar parser) tracked by RFC-001 issue #30 in the umbrella tracker (#55). Scope: - Splits issue #30 into 5 sequential PRs (lexer keywords, type decls, function decls, annotation decls, @annotation use sites) - PR 1 is a minimal-footprint change: 5 keywords reserved at the lexer level, no grammar productions yet — ~5 lines plus tests - Acceptance criteria, soft-break audit grep commands, branch strategy, and follow-ups for the maintainer - PRs 2-5 sketched with file pointers and estimates Doc is implementation-specific but lands here for consistency with the rest of the RFC-001 planning corpus.
This was referenced Jul 16, 2026
trendvidia
added a commit
that referenced
this pull request
Jul 17, 2026
…README layout (#115) - Umbrella #30–#35 ticked: M1 shipped June + the spec-#91/#35 catch-up landed 2026-07-17 (protocompile#66/#67/#69 closed, PRs #70–#72 + follow-ups #77–#80). - #4 round-trip box ticked: annotations.proto verified against protocompile @ 03ef9d8 (GH #58 closed) — parses clean, all 9 declarations lower into FileAnnotationDecls; fixtures 01–06, 09, 10, 12 all compile with zero diagnostics. - #5 boxes ticked; README repository-layout now lists proto/schema/v1/{annotations,descriptor,report}.proto, proto/schema/config/v1/config.proto, and the draft -01 artifacts — the missing README listing was the one genuinely unmet box.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lands the design doc for
pxq, ajq-style command-line query toolwhose core operates on PXF documents. Implementation will follow in a
separate PR — this change is documentation-only.
What's here
cmd/pxq/README.md(178 lines) — full design covering the querylanguage, strict/loose modes, schema inference workflow, per-format
input adapters (PXF, JSON, YAML, CSV),
@pxf.*extension namespace,@prototyped-object constructor, and engine internals.list, between the
protowireCLI andprotoregistrysections, plusa
cmd/pxq/entry in the project structure tree.Key design decisions captured
@pxf.*reserved namespace forquery-time operations that have no jq equivalent.
validation against the schema, runtime mismatches abort); schema
absent → loose (jq-compatible, runtime mismatches degrade to
null).@proto(<dotted>)constructor for typed object output, sourcingschemas from bundled canonical schemas,
-p, or protoregistry.pxq infer-schema) with--full-scanto aggregate every contradiction in one pass.@protodirectives (draft §3.4.5), the embedded schema is honoured without
needing
-p. Aligns the tool with the v1.0 freeze.Why design-first
The design surface is broad (four input formats, two modes, schema
resolution chain, gojq integration) and the renames during the v1.0
spec freeze materially affected the document. Landing the design as
a referenceable artifact in the repo prevents future implementation
work from drifting from the original intent.
Test plan
cmd/pxq/README.mdrenders cleanly in GitHub's markdown@dataset/@proto/ draft section references align withv1.0 spec freeze (§3.4.2-§3.4.6)