Skip to content
This repository has been archived by the owner on Jun 29, 2022. It is now read-only.

Add Vector spec #138

Merged
merged 1 commit into from
Aug 14, 2019
Merged

Add Vector spec #138

merged 1 commit into from
Aug 14, 2019

Conversation

rvagg
Copy link
Member

@rvagg rvagg commented Jul 3, 2019

I've not yet included algorithm details for operations I don't yet support in https://github.com/rvagg/iavector, but I don't want those to be a blocker for making progress on getting this merged, they could come later. The current operations are the most useful for putting this thing to work at the moment.

@rvagg rvagg mentioned this pull request Jul 3, 2019
@rvagg
Copy link
Member Author

rvagg commented Jul 3, 2019

One thing I wanted to draw attention to is the section at the bottom "Flexible head for efficient operations at the head and for sub-trees".
I haven't fully implemented that idea yet, made a start but it got complicated really quickly. But if we successfully managed to spec that out then we'd have a pretty performant data structure that can be mutated and sliced in all sorts of ways without the need to do bulk copies (deletes in the middle still require bulk copies).
Losing canonical form guarantees is a bit of a cost but that could be mitigated by maintaining strict rules where these things are used if there is a need for such guarantees. Or we could have a mode that turns off head-modification operations (and opts for bulk copying).
The tricky bit is that it needs time to investigate and I don't think I can justify that right now as I have some other things on my plate.

Copy link
Member

@vmx vmx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, great write-up!

schema-layer/data-structures/vector.md Outdated Show resolved Hide resolved
schema-layer/data-structures/vector.md Outdated Show resolved Hide resolved
schema-layer/data-structures/vector.md Outdated Show resolved Hide resolved
schema-layer/data-structures/vector.md Outdated Show resolved Hide resolved
schema-layer/data-structures/vector.md Show resolved Hide resolved
schema-layer/data-structures/vector.md Outdated Show resolved Hide resolved
schema-layer/data-structures/vector.md Show resolved Hide resolved
schema-layer/data-structures/vector.md Outdated Show resolved Hide resolved
@rvagg
Copy link
Member Author

rvagg commented Jul 26, 2019

On the topic of iterative vs recursive processes: I've started off implementing operations both here and in HashMap as recursive because it feels more appropriate. However I'm migrating toward iterative approaches in order to make these algorithms more atomic, where each step is a discrete thing that operates on a block that has only just the right amount of state.

In js-composites it's only possible to do it this way, the logic gets invoked per block, so https://github.com/ipld/js-composites/pull/5/files#diff-1a52ff098f2bcf415524a4220e8d61feR20 is a single block/node traversal of a Vector. That one's nice because we can traverse a single block as if it's the only traversal, just adjusting the index for each step. For the HashMap we need more state so we have to use a continuation: https://github.com/ipld/js-composites/pull/1/files#diff-7a3b725525de535be6fe08715127453eR31-R46

Forcing this design style has a benefit of getting us theoretically closer to WASM implementations of these algorithms. Execute individual steps, maintain a certain amount of state. In iavector, I've tried to do most of it this way, even calculating size() is a step-by-step traversal https://github.com/rvagg/iavector/blob/01d58752a1f0fc29555680db4d19c917730340e3/iavector.js#L635

Mutations are more awkward and I've started down the road of doing them this way but the state management quickly gets out of hand and I haven't come up with a nice API for it yet. I'm sure there is one but I haven't had time to iterate on the design enough.

@rvagg
Copy link
Member Author

rvagg commented Aug 14, 2019

This is now merged, but it's: Status: Prescriptive - Draft
So amendments are anticipated. PRs invited if you can see ways to improve it.

@rvagg rvagg deleted the rvagg/vector branch August 14, 2019 08:15
Stebalien pushed a commit to Stebalien/specs that referenced this pull request Sep 18, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants