Ctrie Iterator #89

Merged
merged 7 commits into from May 18, 2015

Projects

None yet

5 participants

@tylertreat-wf
Member

Adds Iterator, Size, and Clear operations to the Ctrie. Also adds a Map function to the persistent list.

@dustinhiatt-wf @alexandercampbell-wf @stevenosborne-wf @beaulyddon-wf @rosshendrickson-wf @tannermiller-wf

@alexandercampbell-wf alexandercampbell-wf and 1 other commented on an outdated diff May 15, 2015
trie/ctrie/ctrie.go
+ go func() {
+ traverse(snapshot.root, ch)
+ close(ch)
+ }()
+ return ch
+}
+
+// Size returns the number of keys in the Ctrie.
+func (c *Ctrie) Size() uint {
+ // TODO: The size operation can be optimized further by caching the size
+ // information in main nodes of a read-only Ctrie – this reduces the
+ // amortized complexity of the size operation to O(1) because the size
+ // computation is amortized across the update operations that occurred
+ // since the last snapshot.
+ size := uint(0)
+ for _ = range c.Iterator() {
@alexandercampbell-wf
alexandercampbell-wf May 15, 2015 Contributor

This can just be for range c.Iterator()

@tylertreat-wf
tylertreat-wf May 15, 2015 Member

Is that compatible with go 1.3? Are we still trying to ensure compatibility with 1.3?

@alexandercampbell-wf
alexandercampbell-wf May 15, 2015 Contributor

Good point-- the new syntax was not adopted until 1.4

@alexandercampbell-wf alexandercampbell-wf and 2 others commented on an outdated diff May 15, 2015
trie/ctrie/ctrie.go
+ if c.rdcssRoot(root, gcasRead(root, c), newRoot) {
+ return
+ }
+ }
+}
+
+// Iterator returns a channel which yields the Entries of the Ctrie.
+func (c *Ctrie) Iterator() <-chan *Entry {
+ ch := make(chan *Entry)
+ snapshot := c.ReadOnlySnapshot()
+ go func() {
+ traverse(snapshot.root, ch)
+ close(ch)
+ }()
+ return ch
+}
@alexandercampbell-wf
alexandercampbell-wf May 15, 2015 Contributor

What happens if you only iterate over the first, say, 5 elements of the trie? Would dead channel remain open in the background?

@dustinhiatt-wf
dustinhiatt-wf May 15, 2015 Contributor

Which would probably also keep the goroutine live. This might be leaky in the case where the iterator wasn't exhausted during iteration.

@tylertreat-wf
tylertreat-wf May 15, 2015 Member

Damn, you're right, this is problematic because if the user doesn't read from the channel the goroutine will block. Any ideas on making a nicer iterator which doesn't have this problem?

@tylertreat-wf
tylertreat-wf May 15, 2015 Member

Will probably just go with a stateful iterator with HasNext() and Next(), which means you can't range. Go is bullshit.

@dustinhiatt-wf
dustinhiatt-wf May 15, 2015 Contributor

Switch to C# or Rust :).

You can also just return an object that supports Next and Value and do for iter := c.Iter(); iter.Next(); { val := iter.Value() }

@dustinhiatt-wf
dustinhiatt-wf May 15, 2015 Contributor

Damn, beat me, and yes this is another case where's Go's demos about channels look better than they are.

@tylertreat tylertreat Allow way to cancel Ctrie iterator
This allows a cancel channel to be passed in to the Ctrie Iterator. When
the channel is closed, the iterator channel will close, freeing up the
goroutine.
0f0497f
@tylertreat
Contributor

Think I found a decent compromise on the iterator nonsense @dustinhiatt-wf @alexandercampbell-wf. See last commit.

@beaulyddon-wf

lol. The rule of Go.... when all else fails just add another channel.

+1

@alexandercampbell-wf
Contributor

+1, but I can't say I like introducing another channel.

@dustinhiatt-wf
Contributor

I can merge this, but I have to agree with @alexandercampbell-wf, I'd much prefer to use a stateful iterator instead of leaving it up to the consumer to ensure memory isn't leaked. Especially for people coming from other languages, the duty of having to create a channel and then cancel is a burden people probably aren't used to for a simple iteration.

@tylertreat, I'll merge as is if you want, but I'm just worried people will just forget to cancel or pass in nil and this thing will leak memory. Up to you.

@tylertreat-wf
Member

The stateful iterator will be surprisingly nasty with this type of trie, at least if you want to do it in a "generator" style.

@dustinhiatt-wf dustinhiatt-wf merged commit 9257431 into Workiva:master May 18, 2015
@tylertreat-wf tylertreat-wf deleted the tylertreat-wf:iterator branch May 18, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment