[spec] data schema #126
[spec] data schema #126
Conversation
|
|
||
| At high level, all the data in the [accounts' dynamic sub-spaces](accounts.md#dynamic-storage-sub-space) is just keys associated with arbitrary bytes. To help the processes that read and write this data (transactions, validity predicates, intents) interpret it and implement interesting functionality on top it, the ledger could provide a way to describe the schema of the stored data. | ||
|
|
||
| For storage data encoding, we're currently using the borsh library, which provides a way to derive schema for data that can describe its structure in a very generic way that can easily be consumed in different data-exchange formats such as JSON. In Rust code, the data can be composed with Rust native ADTs (`struct` and `enum`) and basic collection structures (fixed and dynamic sized array, hash map, hash set). Borsh already has a decent coverage of different implementations in e.g. JS and TypeScript, JVM based languages and Go, which we'll hopefully be able to support in wasm in near future too. |
There was a problem hiding this comment.
Should we implement borsh in Juvix? cc @mariari
| } | ||
| ``` | ||
|
|
||
| When the transaction is applied, the data is stored together with a reference to the derived data schema, e.g.: |
There was a problem hiding this comment.
Where is the schema stored?
There was a problem hiding this comment.
In particular, is this efficient? (we don't want to store the schema along with each copy of the data)
There was a problem hiding this comment.
I'm not sure that a transaction can recognize its type as MultiSig when the transaction reads an account.
I think it can be stored with a key like <height>/schema/<address>/.../<var_name> to the DB.
When a transaction reads the var of the account (the key would be <address>/.../<var_name>), it can read the schema entry with the same key.
In this case, I suppose it has to be stored along with each copy because, for example, another MultiSig which has different members for other accounts exists.
There was a problem hiding this comment.
Yeah, I think it should be possible to reuse schema definitions by e.g. storing the schemas in storage outside of accounts sub-spaces with some unique identifiers (hash of the schema). Then accounts sub-spaces data could store the identifier to the schema (if specified).
With the MultiSig example, we would store the schema at <height>/schema/<schema_id> and there could be many accounts using this schema with e.g.:
MultiSigencoded data at<height>/subspace/<address>/<sub_key...><height>/subspace/<address>/<sub_key...>/schema = <height>/schema/<schema_id>
A possible slight variation would be to have a dedicated special account for schemas, so we could add a validity predicate to it instead of having its logic in the ledger.
I think it would be nice if we could somehow split the storage fee for schemas between its users, so that most commonly used schemas would be very cheap.
the rendered file: https://github.com/heliaxdev/anoma-prototype/blob/63bae79238ec98b91937833395d5447c3760efcb/tech-specs/src/explore/design/data-schema.md