-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
set of observables = meta data, worth storing in a separate feed (log) #1
Comments
Thanks for the feedback @tschudin.
This would assume that we could rely on feeds adhering to this standard always, it can never be later. So it wouldn't work for existing SSB classic feeds. One of the goals of observables is that it works on existing feeds and that it even works if the author of the feed is gone, as others can also make statements about a feed. One could have two feeds, but then the problem becomes linking them (aka same-as).
As for bundling things up into 1 data structure I guess is an implementation choice. One could just have 1 main observable and store a k/v store inside this thing. Or one observable per feed. Maybe that part was overspecified.
I see these observables as snapshots and their size are quite small and I don't think their history is interesting. If somebodyis cheating you store those in some kind of proof system, but otherwise only the latest version is needed. It keeps the protocol simpler that you can just rely on the latest version and have that propagate fast through the network using something like EBT. |
thanks @arj03 for the additional color. In defense of the history (and of using a log for propagating observable state):
Regarding backwards compatibility where to announce a meta feed I agree that using the first entry in a log is not working for classic feeds. But the "every 100th entry" would be a pragmatic fallback solution if you don't find the meta pointer in a log's first entry? |
True, the difference between history and no history basically boils down to: is this a feed or a single value. It is designed to have version numbers, so it doesn't have the problem with out-of-order. I guess conceptually it would be simpler with a feed where one by default is assumed to only store the latest version. I was explicitly trying to make this different as I don't think it should be stored in the main log. Now the where do you find the entry points to observables is perhaps a more interesting problem. At first I was looking at it from the angle that first one finds the observables, then the data that the observables describe. Instead as you can see in the partial replication proposal, the state of the observables are included in the results as proofs. Of course one could use the partial replication to find all the observables as they have a specific type, this would most relevant for observables that are not proofs of something. |
@tschudin I'm closing this issue as I think you got me convinced :-) I wrote a first draft of a https://github.com/arj03/ssb-meta-feed spec. |
Why not have a meta feed as a companion to the main feed (this meta feed could be announced in the very first entry of the main log) for hosting a key/value store inside it, like Hypertrie for Hypercore?
This would allow for efficient lookup of the latest "release=?" or "profile=?" binding etc, and is easy to locate because only one redirection away: fetch the first log entry in the main feed once, then work with the k/v store in the meta feed.
If you are afraid that the meta feed grows too large and you want to drop the history, chain several of them by adding an entry "this meta feed now replaced by feed @_X" before closing the feed, or have a really slow growing meta meta feed containing only a series of "meta=@_X" bindings, the latest being the valid one, or have every 100th entry in the main feed be a "meta=@_X" entry. In the latter two cases, old meta feeds can be discarded.
The text was updated successfully, but these errors were encountered: