Flat: a resolved model with direct access, not a stream of elements - #232
Open
milyin wants to merge 7 commits into
Open
Flat: a resolved model with direct access, not a stream of elements#232milyin wants to merge 7 commits into
milyin wants to merge 7 commits into
Conversation
`core::language` modelled one thing and was named for another. What it parses is the **flat API** — the single flat namespace a `#[prebindgen]` crate exports — so `Language` becomes `Flat` and `api/core/language/` becomes `api/core/flat/`. Mechanical, and separated from the model changes that follow so those arrive as a readable diff. The boundary ledger's skipped-path constant and header move with the directory; the count does not change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Element` mixed two levels: `Function | Struct | Variant | Enum | Const` set type declarations beside functions and constants, when the kinds a binding distinguishes are a function, a type, and a constant. Types now group under `Element::Type`, and the type *reference* — which held the name `Type` — becomes `TypeRef`, so a declaration and a use site stop sharing a word. `Opaque` becomes the entity for a type whose contents do not cross, and it arrives two ways: * `#[prebindgen] pub type X = path;` — this **reverses** #227, where a marked alias was `Unsupported`. It is now how a handle enters the flat API deliberately: a foreign or crate-private type gets a name here without any claim about its contents. That is what makes the API closable, and it is the prerequisite for requiring references to resolve. * a marked tuple struct, whose fields no adapter has ever crossed — unchanged acceptance, now named for what it always meant. So `Struct::fields` drops its `Option`. `None` was the opaque case; an empty list now means the source wrote a struct with no fields, which is a different thing. `MaybeUninit<T>` joins the grammar as `TypeKind::Uninit`. It is a boundary concept — an out-parameter whose slot the caller supplies and the callee fills — and cbindgen already models it as exactly that, so this moves a classification out of the adapter and into the frontend, per #211. It is also the one foreign generic that no alias could name, a generic alias being a generic binder. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two changes that belong together, because the first is what makes the second decidable. **The model is addressed by name, not iterated.** `FlatBuilder` collects and `build` hands over a `Flat` — `function(name)`, `declared_type(name)`, `constant(name)`, `element(name)`, plus iterators over each kind. Names are unique across the whole model, so a name is a complete address, and that is what every later stage wants: an adapter asks what a declared name *is* rather than scanning a list. L1 carried this as a checklist bullet; it is really a property of the model. Two types rather than one, because a half-built model should not be the same type as a resolved one — `Source::builder()` sets the precedent. **References resolve at parse time.** A third pass walks every `TypeRef` — through `Option`, `Vec`, `&`, `Result`, arrays, callback arguments and generic arguments alike — and an item naming a type the flat API does not declare becomes `Element::Unsupported` with `ItemError::UnresolvedType`. Deferred, not fatal, like every other refusal: an item no binding declares stays harmless. This is what a marked type alias bought. A dangling name previously surfaced far downstream as an unresolved *converter*, from whichever adapter happened to look first — the "one fact, several authorities" #211 exists to end. Note the two remain distinct: resolution here says a name denotes something, while an adapter's resolver still decides whether it supplied a converter for it. A path-qualified name gets its own diagnosis, since `#[prebindgen] pub type foreign::Option = ..` is not a spelling that exists — marked items live in one flat namespace of bare names. Also: `Item::Type` no longer reaches the registry's passthrough. An opaque declaration states something about the API's surface and is not code to copy into the binding; its target is routinely crate-private, so re-emitting it would not compile. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every type a marked signature named had to become a declaration for resolution to mean anything. Two idioms, chosen by what the type actually is rather than by its Rust shape: **A handle gets a marked alias.** `Storage`, the three callback handlers, `Token`, `TokenGc`, `Summary`, `Archive`, `Report`, `EscapeProbe`, `StorageError`, and example-flat's `Calculator` move into a private `handles` module, with `#[prebindgen] pub type X = handles::X;` at the top level. The alias is transparent, so every signature still says `Storage`. `Error` in both crates was already an alias and only needed the attribute — which is exactly the shape zenoh-flat's 26 zenoh re-exports will take. **A public newtype stays a marked struct.** `Millis`, `Celsius`, `Percent` and `Label` are not handles: they cross by `convert!`, and covertest-helpers both constructs them and reads `.0`. Hiding them behind an alias broke that downstream, which is the useful signal — a type alias names the type, not the tuple-struct constructor, and the constructor lives in the value namespace where the struct is defined. In-crate construction of the relocated handlers is qualified `handles::PayloadHandler(..)` for the same reason. Marking these as structs rather than aliases matters for a second reason: a marked struct enters `registry.structs`, and `write.rs` emits `on_struct` for any declared type there — so marking the *handles* as structs would have changed generated output. The alias route is invisible to the registry, which is why the goldens hold. **And the closure is asserted, not assumed.** covertest-kotlin's build script now runs `Flat` over both sources and fails if anything is unsupported. It is the right place: only there do the helper crate's references to perftest-flat's types resolve, since it cannot mark them itself. Verified by deliberately unmarking `Storage` — the build fails naming all twelve referencing functions and the fix. Generation is byte-identical (`examples/regen-check.sh`) and the JVM covertest passes all 47 sections. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The model is now indexed and resolved, which takes two bullets off L1 — elements indexed by name, and the entry point that shares one parser — and adds a prerequisite L0 did not have: the flat API has to be closed for resolution to mean anything. Also records what is left open: zenoh-flat and its two consumers are separate repos whose 28 unmarked types need the same treatment, and `Cow<'_, [u8]>` has no alias spelling. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`clippy::ptr_arg` under CI's no-default-features run: the pass only mutates elements in place, so a slice is the honest signature. My local checks used --all-features only; CI runs three clippy configurations. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`TypeKind::Uninit` wrapped a type, but uninitialized-ness is a property of the
**borrow**: my own doc said `MaybeUninit` is "only meaningful behind a `&mut`",
which is the argument against modelling it as a type at all.
So `Ref` carries the mode, and the `MaybeUninit` is absorbed into it:
Ref { mode: RefMode, inner: Box<TypeRef> }
enum RefMode { Shared, Exclusive, Out }
`&T`, `&mut T`, `&mut MaybeUninit<T>` — one axis, three values, and `inner` is
always the borrowed *value's* type. One variant fewer than the `mutable` flag plus
a wrapper, and the combinations that mean nothing at a boundary can no longer be
written down: uninitialized storage owned, returned or in a field promises nothing
a destination language can use, and `&MaybeUninit<T>` promises a readable `T` that
may not be one. Both are refused, each naming why.
`Out` rather than `Uninit` because it names the boundary role every destination
language has — C's `T *out` — which is the fact an adapter acts on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage L0.5 of #229. #227 produced a
Vec<Element>; this makes it a model thatcan be asked questions, and closes the flat API so its references mean something.
Targets
language-integration, notmain.Why
Three things about L0's output were wrong for what consumes it next.
It was a stream, so nobody could ask it anything. Every later stage needs to
look an item up by name. L1's checklist carried "elements are indexed by name" as
an afterthought; it is a property of the model.
References were unresolved. A parameter naming
StorageyieldedTypeKind::Named { id }and nothing more. Whether that name denoted anythingsurfaced much later as
ResolveError::Unresolvedfrom the fixed-point resolver —the "one source-language fact, several authorities" #211 exists to end. Names live
in one flat namespace, so resolution is decidable at parse time.
Elementmixed two levels.Function | Struct | Variant | Enum | Constputtype declarations beside functions and constants.
And the module was called
languagewhile the thing it models is the flat API.What
The type reference becomes
TypeRef, freeingTypefor the declaration.Struct::fieldsdrops itsOption—Nonewas the opaque case, which is now itsown entity.
Opacity is declared
#[prebindgen] pub type X = path;yieldsType::Opaque. This reverses #227,where a marked alias became
Unsupported. It is how a foreign or crate-privatetype gets a name in the flat API without any claim about its contents — and it
is the prerequisite for requiring references to resolve. A marked tuple struct
declares the same thing, unchanged from #227.
References resolve at parse time
A third pass walks every
TypeRef— throughOption,Vec,&,Result,arrays, callback arguments and generic arguments alike. An item naming a type the
flat API does not declare becomes
Element::UnsupportedwithItemError::UnresolvedType: deferred, not fatal, like every other refusal, so anitem no binding declares stays harmless. A path-qualified name gets its own
diagnosis, since
#[prebindgen] pub type foreign::Option = ..is not a spellingthat exists.
Note the two resolutions stay distinct: this says a name denotes something,
while an adapter's resolver still decides whether it supplied a converter for it.
An out-parameter is a mode of borrowing
MaybeUninitis only meaningful behind a&mut, so uninitialized-ness is aproperty of the borrow, not of a type.
Refcarries the mode and absorbs it:One axis, three values, and
inneris always the borrowed value's type — so thecombinations that mean nothing at a boundary cannot be written down. Owned,
returned or in a field, uninitialized storage promises nothing;
&MaybeUninit<T>promises a readable
Tthat may not be one. Both are refused, each naming why.Outrather thanUninitbecause it names the boundary role every destinationlanguage has (C's
T *out) — the fact an adapter acts on. cbindgen was alreadyclassifying
MaybeUninititself, so this moves that decision into the frontend per#211; it is also the one foreign generic no alias can name, a generic alias being a
generic binder.
Closing the example flat APIs
Every referenced type had to become a declaration. Two idioms, chosen by what the
type is rather than by its Rust shape:
Storage, the three callback handlers,Token,TokenGc,Summary,Archive,Report,EscapeProbe,StorageError,Calculatormove into a privatehandlesmodule with#[prebindgen] pub type X = handles::X;. The alias is transparent, so everysignature still reads the same.
Errorin both crates was already an alias andneeded only the attribute — the shape zenoh-flat's 26 re-exports will take.
Millis,Celsius,Percent,Labelare not handles: they cross byconvert!, and covertest-helpers bothconstructs them and reads
.0. An alias names the type but not thetuple-struct constructor, which lives in the value namespace — hiding them broke
that downstream, which was the useful signal.
That second distinction also protects the goldens: a marked struct enters
registry.structs, andwrite.rsemitson_structfor any declared type there —so marking the handles as structs would have changed generated output. A marked
alias is invisible to the registry (
Item::Typeno longer reaches passthrougheither, since an opaque declaration is not code to copy into a binding and its
target is routinely crate-private).
The closure is asserted, not assumed. covertest-kotlin's build script runs
Flatover both its sources and fails if anything is unsupported — the only placethe helper crate's references to perftest-flat's types resolve, since it cannot
mark them itself. Verified by deliberately unmarking
Storage: the build fails,naming all twelve referencing functions and the fix.
Verification
502 unit tests and 17 doctests pass on
--all --all-featuresand on defaultfeatures; clippy clean; the boundary ledger unmoved.
examples/regen-check.shreports byte-identical — the source crates were restructured and the model
rebuilt, and generation did not move.
covertest-kotlin's JVM run passes all 47sections.
Out of scope, and will break
zenoh-flat,zenoh-flat-candzenoh-flat-jniare separate repos. Their 28unmarked types (26 zenoh aliases plus
DurationandCow<'_, [u8]>) need the sametreatment before they parse.
Cow<'_, [u8]>has no alias spelling, sozbytes_to_bytesneeds either theMaybeUninittreatment or a signature change.Noticed while surveying:
zenoh-flat-c/build.rs:539declareszbytes_as_bytes,which no longer exists in zenoh-flat (renamed
zbytes_to_bytes) — that repo isalready stale against zenoh-flat HEAD.
Refs #211. Umbrella: #229.
🤖 Generated with Claude Code