Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow multiple iterators in a single Read / Write transaction #705

Closed
timshannon opened this issue Jan 30, 2019 · 8 comments
Closed

Allow multiple iterators in a single Read / Write transaction #705

timshannon opened this issue Jan 30, 2019 · 8 comments
Labels
area/api Issues related to current API limitations. kind/enhancement Something could be better. priority/P2 Somehow important but would not block a release. status/more-info-needed The issue has been sent back to the reporter asking for clarifications
Milestone

Comments

@timshannon
Copy link
Contributor

Currently you can only have one iterator on a read / write transaction:
https://github.com/dgraph-io/badger/blob/master/iterator.go#L396

I have a scenario where I would want to read the consistent state of the current transaction before it's committed. I'm currently getting around this by re-seeking a single iterator, but the ideal situation would allow for using multiple iterators.

According to this comment: #704 (comment) it may be possible.

@manishrjain manishrjain added the priority/P1 Serious issue that requires eventual attention (can wait a bit) label Feb 1, 2019
@dotnwat
Copy link

dotnwat commented Apr 16, 2019

This would be useful to have. For example, to do basic merge-join in a transaction.

@campoy campoy added area/api Issues related to current API limitations. kind/enhancement Something could be better. priority/P2 Somehow important but would not block a release. status/accepted We accept to investigate or work on it. and removed priority/P1 Serious issue that requires eventual attention (can wait a bit) labels May 28, 2019
@campoy campoy added this to the Unplanned milestone May 28, 2019
@campoy
Copy link
Contributor

campoy commented May 28, 2019

Adding this to the list of API changes we're considering for v2.

@campoy campoy modified the milestones: Unplanned, v2.0 May 28, 2019
@campoy
Copy link
Contributor

campoy commented Jun 13, 2019

It seems we could use multiple iterators as long as it's not concurrently, would that be enough or do we need to change anything in our code?

@campoy campoy added status/more-info-needed The issue has been sent back to the reporter asking for clarifications and removed status/accepted We accept to investigate or work on it. labels Jun 13, 2019
@timshannon
Copy link
Contributor Author

I don't see why that would be an issue.

@campoy
Copy link
Contributor

campoy commented Jun 21, 2019

Why would using multiple iterators concurrently would be an issue? Data races, unfortunately.

Why do you need multiple non-read-only iterators concurrently?
Is there a specific algorithm you're trying to implement?

@stale
Copy link

stale bot commented Jul 21, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the status/stale The issue hasn't had activity for a while and it's marked for closing. label Jul 21, 2019
@timshannon
Copy link
Contributor Author

timshannon commented Jul 23, 2019

Sorry, I've been busy, and I just revisited this and realized that in my glibness my response may have been interpreted differently than I intended. I meant I didn't see any reasons why changing to multiple non-concurrent iterators would cause me any problems. I meant that I'm fine with that change.

Sorry for the confusion.

@stale stale bot removed the status/stale The issue hasn't had activity for a while and it's marked for closing. label Jul 23, 2019
@campoy
Copy link
Contributor

campoy commented Aug 5, 2019

In that case, I think this issue can be closed?
Let me know if there's any other questions you might have and thanks for contributing :)

@campoy campoy closed this as completed Aug 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api Issues related to current API limitations. kind/enhancement Something could be better. priority/P2 Somehow important but would not block a release. status/more-info-needed The issue has been sent back to the reporter asking for clarifications
Development

No branches or pull requests

4 participants