Dynamic Module Metadata #47
lollobene
started this conversation in
Featured Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Abstract
This proposal refines the
PackageMetadatamodel defined in IIP-0010 by moving per-module metadata out of the immutablePackageMetadataV1object and into dynamic fields. Instead of embedding an inlineVecMap<String, ModuleMetadataV1>, thePackageMetadataV1object now records a metadata-layout version and references a map ofModuleMetadataobjects, each derived deterministically from the package id and module name. EachModuleMetadatais itself an extensible key-value store backed by dynamic fields, where every key holds a distinct kind of module metadata.This makes the set of recorded module metadata kinds open-ended: new kinds can be added by writing a new dynamic field, without changing the
PackageMetadataobject type or introducing a new top-level struct version. The first new kind enabled by this layout is view function metadata — the list of#[view]functions (IIP-0011) per module — recorded alongside the existing authenticator metadata.The change is gated behind a protocol feature flag and is backwards compatible: packages published under the previous layout keep working, and the legacy accessors continue to function.
Motivation
IIP-0010 stores all module metadata inline in the frozen
PackageMetadataV1object:This shape couples the content of module metadata to the type of the metadata object. IIP-0010 already anticipated that new metadata kinds (e.g. view functions) would be added, and proposed handling growth by bumping the struct version:
PackageMetadataV2,V3, and so on (see IIP-0010 — Backwards Compatibility).Versioning the struct type for every new metadata kind has two drawbacks:
ModuleMetadata/PackageMetadatastruct version, and every reader must learn the new type. A struct field is also strongly typed and ordered, which is a poor fit for an open, growing set of optional metadata kinds.VecMapis read as a whole. As more kinds are added, readers and tooling that only care about one kind (say, view functions) still materialize the entire module metadata.IIP-0011 introduces
#[view]functions and states that view functions are exposed throughPackageMetadata, "similarly to how authenticator functions are currently exposed", so that clients and tooling can discover them without static analysis. Adding view function metadata is therefore the first concrete case that requires growing the per-module metadata, and it is the trigger for this refinement.This proposal addresses both drawbacks by representing module metadata as dynamic fields keyed by metadata kind. New kinds are added by writing a new key; readers fetch only the keys they need; and the
PackageMetadataobject type stays stable.Specification
This proposal modifies only the publish/upgrade object-creation phase and the runtime accessors of the Package Metadata model. The compilation phase, the trust model, and the package-metadata object id derivation are unchanged from IIP-0010: metadata is still extracted from verified bytecode by the protocol, the object is still frozen, and its id is still derived from the package storage id.
Layout
When the relevant feature flag is enabled (see Activation), the protocol builds
PackageMetadataV1with the following dynamic-field layout instead of the inlinemodules_metadatamap.The
PackageMetadataV1object carries two dynamic fields:PackageMetadataVersionFieldNamekey, set to2. Its presence and value distinguish the dynamic-field layout from the legacy inline layout.ModulesMetadataFieldNamekey, of typeVecMap<ModuleName, ModuleMetadata>, mapping each module name to itsModuleMetadataobject.The inline
modules_metadata: VecMap<ascii::String, ModuleMetadataV1>field ofPackageMetadataV1remains in the struct definition for backwards compatibility but is left empty under this layout; it is deprecated and MUST NOT be read by new code.Each
ModuleMetadatais a standalone object that behaves as a key-value store backed by dynamic fields:Each entry in a
ModuleMetadatais a dynamic field whose key type identifies a kind of metadata and whose value holds that kind's payload. The protocol defines the following keys for this version:ModuleMetadataV1FieldName→ModuleMetadataV1— the authenticator metadata from IIP-0010, re-homed under this key. Present only for modules that have at least one authenticator.ViewFunctionMetadataV1FieldName→vector<ascii::String>— the list of view function names declared in the module (new; see View function metadata). Present for every module recorded under this layout.Additional kinds can be introduced later as new key types without changing
ModuleMetadata,PackageMetadataV1, or any existing key's payload.Module metadata object id derivation
Each
ModuleMetadataobject id is derived deterministically from the owning package storage id and the module name, reusing the same derived-object mechanism IIP-0010 uses for the package metadata object id:Where
ModuleMetadataKeywraps the module name (ModuleMetadataKey(ascii::String)). Deriving from the package id together with the module name guarantees that:ModuleMetadataobject (and therefore a distinct dynamic-field namespace), andAs with the package metadata id, this id can be computed on-chain and by off-chain tooling without a lookup.
View function metadata
A function annotated with
#[view](IIP-0011) is recorded in its module'sModuleMetadataunder theViewFunctionMetadataV1FieldNamekey as avector<ascii::String>of function names.The flow mirrors the authenticator flow of IIP-0010:
#[view]attribute is recorded in the function'sRuntimeModuleMetadataand embedded in the module bytecode, exactly as other recognized attributes are.#[view]fails to publish — this is what lets readers trust the recorded list without re-checking signatures.ModuleMetadata.Because the recorded list is protocol-attested, a reader can treat membership in the list as a trusted "this function satisfies the view constraints" claim.
Conditional creation
The conditional-creation rule from IIP-0010 (a
PackageMetadataobject is created only when meaningful metadata exists) is preserved and broadened to the new kinds: under this layout, aPackageMetadataV1is created when any module of the package contributes at least one kind of metadata (an authenticator or a view function). Modules with no recognized metadata are not recorded.Activation
The new layout is gated behind the
package_metadata_with_dynamic_module_metadataprotocol feature flag. The flag depends on the existing package-metadata publishing flag: enabling it requires package-metadata publishing to be enabled.This lets the network adopt the new layout at a protocol upgrade boundary while existing metadata objects remain valid.
Move types and methods
New accessors
On the
ModuleMetadataobject:Deprecated accessors
The IIP-0010 accessors that return
ModuleMetadataV1by value are retained but deprecated. They are made layout-aware: when called on an object built with the dynamic-field layout (version marker== 2), they read through the dynamic fields; otherwise they fall back to the inline map. This keeps existing callers working across both layouts.Rationale
Why dynamic fields instead of struct versioning. IIP-0010 proposed accommodating new metadata kinds by bumping the
PackageMetadata/ModuleMetadatastruct version. Dynamic fields keyed by metadata kind achieve the same extensibility without type churn: a new kind is a new key, not a new struct type, and thePackageMetadataobject type stays stable across additions. Readers fetch only the kinds they care about, and unknown kinds are simply absent keys rather than version mismatches. This matches how the metadata is actually consumed — an open, optional, per-module set of facts — better than a fixed, ordered struct.Why a version marker. The
PackageMetadataVersionFieldNamefield (value2) lets a reader, and the deprecated accessors, distinguish the dynamic-field layout from the legacy inline layout on an existing object, so both can coexist on-chain after the protocol upgrade.Why per-module derived objects. Giving each module its own
ModuleMetadataobject, with an id derived from the package id and module name, keeps each module's dynamic fields in an isolated namespace, makes the module metadata object addressable and computable off-chain, and avoids collisions between identically named modules in different packages.Trust model unchanged. As in IIP-0010, all content is written by the protocol from verified bytecode and frozen. For view functions specifically, the verifier rejects any package that marks a non-conforming function
#[view], so the recorded view function list is trustworthy without re-validation by readers.Backwards Compatibility
This proposal is backwards compatible.
PackageMetadataV1objects unchanged. They have no version marker and no dynamic fields, and the deprecated accessors continue to read them via the inline map.package_metadata_with_dynamic_module_metadataflag is enabled at a protocol upgrade. Before activation, behavior is identical to IIP-0010.modules_metadata_v1,try_get_modules_metadata_v1) remain available and are made layout-aware, so callers compiled against IIP-0010 continue to work against objects in either layout. They are deprecated in favor of the newmodule_metadata/*_function_metadata_v1accessors.The
modules_metadatainline field ofPackageMetadataV1is retained for compatibility but is empty under the new layout and deprecated for reads.Test Cases
#[authenticator]functions; read back via both the new and the deprecated accessors).#[view]functions and queryable viais_view_function_v1/borrow_view_functions_metadata_v1.PackageMetadataobject; a package with only view functions does.#[view]at publish time.Reference Implementation
Implemented in the IOTA core monorepo. Key components:
iota::package_metadataandiota::module_metadata(crates/iota-framework/packages/iota-framework/sources/package_metadata/).iota-execution/latest/iota-adapter/.../package_metadata.rs).iota-execution/latest/iota-verifier/.../view_function_verifier.rs).package_metadata_with_dynamic_module_metadataincrates/iota-protocol-config.See also IIP-0010 (Package Metadata) and IIP-0011 (Core Move View Functions).
Copyright
Copyright and related rights waived via CC0.
Discussion
The discussion should continue to look for improvements for this proposal.
CC @lzpap @iotaledger/vm-language
Beta Was this translation helpful? Give feedback.
All reactions