Skip to content

feat(pxq): Stage C — schema loading + pxf_* extensions + pxf_proto - #34

Merged
trendvidia merged 1 commit into
mainfrom
feat/pxq-impl-c-schema
May 14, 2026
Merged

feat(pxq): Stage C — schema loading + pxf_* extensions + pxf_proto#34
trendvidia merged 1 commit into
mainfrom
feat/pxq-impl-c-schema

Conversation

@trendvidia

Copy link
Copy Markdown
Owner

Summary

Schema-aware query layer for the jq-style tool. Closes the README's
extension surface: all five pxf_* functions are live, and
-p schema.proto is wired to a descriptor registry that
pxf_directive("dataset") (for row binding) and pxf_proto(...) (for
typed output construction) consume.

Strict-mode compile-time AST validation against the schema stays
deferred to a follow-up — Stage C is "schema available, errors at
runtime" rather than "schema enforced at compile time."

Function surface

Function Purpose
pxf_directive(name) Directive list by name, source order. Recognised names: "type", "dataset", "proto", or any generic @<name>. Dataset rows are schema-bound field-name maps when a descriptor is in scope; otherwise schemaless. Proto entries surface shape / typeName / body.
pxf_fieldnames Declared field names for the bound message; falls back to the document's @type when input is the top-level body map. Errors in loose mode.
pxf_type Proto type of the input value, or jq-level type as fallback.
pxf_has(field) Schema-aware has() that distinguishes absent from zero-value.
pxf_proto(name; obj) Typed object construction. Validates the descriptor exists, attaches an @type sentinel the emitter lifts into a document-level @type Name directive.

Naming convention

gojq reserves @<name> for built-in string formatters (@uri, @base64, …) and doesn't allow third-party @-extensions. The README's @pxf.* surface lands as pxf_* plain gojq functions — same namespacing intent, valid grammar. README updated in this PR.

Architectural changes

  • loadedDoc shape: adapters previously injected __pxf_datasets / __pxf_protos / __pxf_directives synthetic keys at the top level of the gojq graph (Stage A scaffolding). Stage C separates body from directives; pxf_directive(name) is now the documented access path. CSV's synthetic dataset wrapper becomes a real pxf.DatasetDirective so it round-trips identically with native PXF datasets.
  • Emitter: an @type map key on the result is lifted into the document-level @type Name directive form on output, so pxf_proto(...) produces valid PXF on the wire rather than the invalid @type = "Name" field-assignment form.

Demo

The README's quick-start example now works as written:

$ pxq 'pxf_directive("dataset")[0].rows | map(select(.symbol == "AAPL")) | length' trades.pxf
value = 2

Test plan

  • go build ./... clean
  • go test ./cmd/pxq — 45 tests pass (28 from A/B + 17 new)
    • Schema loader: nested messages registered, compile errors
      propagate, empty-args returns nil
    • pxf_directive: type/dataset/proto/generic/missing cases
    • pxf_fieldnames: no-schema error, schema-resolved field list
    • pxf_type: jq-level fallbacks for all six kinds
    • pxf_has: absent vs present cell semantic
    • pxf_proto: no-schema error, successful bind with @type
      emission, unregistered-type error, pipe-form invocation
  • README quick-start smoke test verified end-to-end
  • CI passes on the matrix

Follow-ups (queued)

  • In-document @proto directive as a schema source (Stage C2)
  • protoregistry resolution (-s/-n/--schema)
  • Strict-mode compile-time AST validation
  • pxq infer-schema subcommand

Schema-aware query layer for the jq-style tool. Closes the README's
extension surface — pxf_directive, pxf_fieldnames, pxf_type, pxf_has,
and pxf_proto are all live, with -p schema.proto wired to a
descriptor registry. Strict-mode AST validation (compile-time field-
name checking against the schema) stays deferred to a follow-up.

Five new functions, registered via gojq.WithFunction:

* pxf_directive(name)  — directive list by name, in source order.
  Recognised names: "type", "dataset", "proto", and any generic
  @<name>. For "dataset", rows are exposed as schema-bound field-name
  maps when a descriptor is in scope; otherwise as the schemaless
  graph. For "proto", entries surface shape/typeName/body.
* pxf_fieldnames       — declared field names for the bound message.
  Resolves the type from the input's @type sentinel, falling back to
  the document's @type when the input is the top-level body map.
  Errors with a help string in loose mode.
* pxf_type             — proto type of the input value, or jq-level
  type when none is bound.
* pxf_has(field)       — schema-aware has() that distinguishes absent
  from zero-value.
* pxf_proto(name; obj) — typed object construction. Validates that
  the descriptor exists in the registry, then attaches an @type
  sentinel the emitter lifts into a top-of-document @type directive.

Naming convention: gojq reserves @<name> for built-in string formatters
and doesn't allow third-party @-extensions. The README's `@pxf.*`
surface is implemented as `pxf_*` plain functions instead — same
namespacing intent, valid gojq grammar. README updated to reflect.

Architectural change in this PR:

* The loadXXX adapters used to inject `__pxf_datasets` / `__pxf_protos`
  / `__pxf_directives` synthetic keys at the top level of the gojq
  graph (Stage A scaffolding so raw jq could reach directives without
  an extension surface). Stage C drops those — loadedDoc now carries
  body + directive lists separately, and pxf_directive(name) is the
  documented access path. CSV's synthetic dataset wrapper becomes a
  real pxf.DatasetDirective so it round-trips identically with PXF.

Emitter learns to lift an `@type` map key into the document-level
`@type Name` directive form, so `pxf_proto(...)` output is a valid
PXF document on the wire (not the invalid `@type = "Name"` field-
assignment form).

Validation:
* 45 Go tests (28 from A/B + 17 new) — schema loader (nested
  messages, error propagation, empty-args nil), pxf_directive
  (type/dataset/proto/generic/missing), pxf_fieldnames (no-schema
  error, schema-resolved), pxf_type (jq-level fallbacks), pxf_has
  (schema-aware absent vs zero-value), pxf_proto (no-schema error,
  successful bind with @type emission, unregistered-type error,
  pipe-form invocation).
* README quick-start example now works as written:
    pxq 'pxf_directive("dataset")[0].rows | map(select(.symbol == "AAPL")) | length' trades.pxf
  yields `value = 2`.

Follow-ups: in-document @proto schema source (currently only -p);
protoregistry resolution; strict-mode compile-time AST validation;
pxq infer-schema subcommand.
@trendvidia
trendvidia merged commit 97c067f into main May 14, 2026
4 checks passed
@trendvidia
trendvidia deleted the feat/pxq-impl-c-schema branch May 14, 2026 04:00
trendvidia added a commit that referenced this pull request Jul 16, 2026
trendvidia added a commit that referenced this pull request Jul 16, 2026
…FC-001 §8.5) (#110)

* spec: no pxf.* dual-emission — @required/@default are carrier-only (RFC-001 §8.5)

Resolves the #92 decision as carrier-only:

- §5.2/§8.5: @required/@default lower exclusively to the 50400
  carrier; the compiler MUST NOT synthesize (pxf.required)/
  (pxf.default) from them. Brackets and annotations are disjoint
  surfaces, neither back-filled from the other; coexistence allowed,
  MAY warn on conflicting values, MUST NOT reconcile.
- Appendix A migration notes: legacy-option consumers see only
  bracket-written options — upgrade consumers before migrating
  schemas.
- Draft -01 Coexistence section: drops the 'lowers to set both'
  claim, states the disjoint-surface rule and migration ordering.
- STABILITY.md: corrects the 'same descriptor shape' claim; new
  'No legacy dual-emission' bullet declaring v1.1 legacy-option
  readers unsupported against annotation-form schemas.
- Fixture 04 header: dual-emission expectation replaced with the
  disjoint-lowering expectation.

Ratifies shipped protocompile M1 carrier-only emission as-is; the
#33/#34 dual-emission acceptance boxes are struck in the
resolution record.

* docs: record #23 carrier-only resolution (GH #92, PR #110); strike #33/#34 dual-emission boxes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant