Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Metadata Key/Value Association & Store #8
Metadata Key/Value Association & Store
-WORK IN PROGRESS-
Currently in the version 2 protocol and storage format there is no formalized way to associate related metadata to accounts, assets, etc. This proposal outlines the aproach to introducing a key/value storage implementations
Storage of assocaited key/value pairs, target supported objects:
Key and Value Size Limits
to be finalized
It is desired to have expiration functionality in order to have key/value pairs expire and be null'd out and/or pruned
Current design thinking is that full history will not be stored by default, only the latest value for any asserted key. To accomidate build in audit history it will be designed in such a manner it can be a configurable setting which could take a couple approaches:
One could default to entire metadata history to be stored:
Which would perist and make retrievable the entire history, until forceably pruned/truncated out.
Another approach that could be taken is to have a configuration setting that would take an integer value for a maximum number of historical key/value pairs to be kept around:
Which would store the current asserted value, plus the last two asserted value(s).
TODO: decide if any configuration support for expiration, is it baked in by default, can it be turned off, etc
Various wants and needs when using the existing protocol is to associate one or more pieces of metadata to various objects to enable a fuller solution.
Typically to date most have leveraged the description field on assets and the message field in transactions
Some decisions and tradeoffs will need to be made balancing capabilities, developer/user experience, and overall management of a chains size and networks performance over its lifecycle. Some topics that stand out that will need more detailed design and discussion:
This functionality is currently targeted for initial support in the up and coming "E" release. TODO: add link ref here
Increased storage pressure, possible reduction in TPS support depending on deployment and configuration
TODO: add any refs here
This is going to add a lot of utility to Catapult, especially for "Blockchain of Information" projects. It would be nice to think of a good name for it.
For the TODOs. Here are my first thoughts.
I'm not completely sure the relationship between pruned and expiration is, but assuming it is expired, then it is good to prune it. But given that it is static and perpetual, then I'm assuming it can't and shouldn't be pruned?
Also, about the value being pruned. I'm assuming that if Alice writes on my account with the Key of "email" and Bob writes on my account the Key of "address", and Charlie writes on my account the Key of "phone number" that even though these three were done at different times that each of these keys/values remain? Lastly, then I could go over Alice, Bob, and Charlie's Key/values and make the values all null by leaving the values blank in an edit?
Really, for account permissions, if you just had "only this white list can assign key/values" that will cover most use cases IF we can combine these with aggregates, e.g., all permission are turned off, I make an aggregate with three parts. Part 1: turn on Alice being able to edit key/value data on my account, Part 2: Alice edits say key/value on my account. Part 3: I turn off Alice from being able to edit the key/value. Then basically I am approving Alice each time she edits my account.
I think so too.
It would be nice to know if inner transactions each can have their own metadata. I'm not sure the implications of this but it makes my mind think of things.
Also, wrt Apostille like transactions, it would be nice if we could have a non-mutable option for metadata on an account. So that it was written and fixed forever, not even nullable.
I think who sets the metadata is owner of the metadata. So when A set a metadata with key "X" to B, only A should be able to change the value of "X".
History could be kept by only including the hash of the key/value in the "metadata" transaction and key/value data itself in the state of the object (account, mosaic....) (just an example of storing the history). This allow removing of sensitive data by the owner of the metadata. It can still be validated that it was there at a certain point. I don`t think this is gdpr complient but thats up to users itself.
A) I would set default to "Everybody". This because it gives the capability to use accounts as assets withoud knowing the privatekey.
An idea for metadata
An assumption of metadata, there are three keys used to set the metadata (value). These are: (1) sender account, (2) receiver object type, (3) metadata key.
objects that can receive metadata are:
* pruning means that the data object is removed from the state of the object. The metadata (hash of the key/value pair) is still available in the blockchain history (pruning deserves its own discussion).
The key for me is to have metadata as universal to all the objects as possible.
Examples for metadata
Below I am giving examples for the use of metadata. Before doing so, I need to address the concept of the immutability of metadata:
Non-fungible token (NFT).
Description of the namespace
Description of the mosaic
Please feel free to improve!
It should also be possible to set max
Regarding @jorisadri comments for access control. It is possible to have a simple access control list using account properties to whitelist accounts that can send transactions to it, hence also limiting which accounts can edit or add metadata.
I have a feeling discussion above should probably be restarted, because it seems everyone has different understanding of what metadata IS and how it will behave.
Important thing to note here is: server will not parse OR recognize keys in ANY WAY
If you need any kind of rules, what you're asking about is NOT metadata and should not be discussed within this NIP.
I'd like to weigh my cents here. We've been playing around this metadata idea at ProximaX for a while and I have a very simple approach to it. The entire reason why we want a metadata key-pair is to have an additional data store on all the possible plugins both existing and custom ones. We want to have a generic metadata so that one can attach any association without any rules to it. If one wants to apply some rules, then a custom plugin should be built in accordance to the framework or architecture of catapult.
My suggestion is to just create a key pair object on an abstract object used by any plugins. The size of this object should be limited. In very simple terms, it's like a message field but applied to all plugins :) This is currently what I'm trying to do on our side and we want to it as soon as possible because we see a great deal of use for this.
I'll be able to show some code once it's done it but I'd like to get some thoughts/feedback if this is the intention of this NIP. If not then it's all ok.