New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Atomic update operation? #6

Closed
phreeza opened this Issue Jun 18, 2014 · 3 comments

Comments

2 participants
@phreeza

phreeza commented Jun 18, 2014

Not sure how hard this would be to implement, but it would be neat to have a safe way of updating an entry. Right now I am deleting and then inserting the new entry, which seems the only way to do this, but it is quite unsafe, for example if the program is terminated right between the write and delete steps the database may become corrupted.

If there is interest in this, I would be willing to put in some work, if someone can give me guidance. Does it fundamentally go against the philosophy, or would it make sense?

@msiemens

This comment has been minimized.

Show comment
Hide comment
@msiemens

msiemens Jun 18, 2014

Owner

Hey phreeza!

First about the update operation: It would defenitely be useful to have it. I thought about something like this:

db.update({'int': 2}, where('char') == 'a')

That would update all elements, where the char field equals 'a' to have a field int equal to 2.
Running this command would result in only one write operation, so it's definitely better that a remove + insert.

I've implemented this update function and would update TinyDB soon.

Concerning the atomicity: At the moment TinyDB does not have real, fault-tolerant atomicity. I've found a interesting article about that topic: Reliable file updates with Python. If you need something closer to real atomacity, feel free to take a method from the article and implement a new storage class (see TinyDB docs) (and submit a PR).

Owner

msiemens commented Jun 18, 2014

Hey phreeza!

First about the update operation: It would defenitely be useful to have it. I thought about something like this:

db.update({'int': 2}, where('char') == 'a')

That would update all elements, where the char field equals 'a' to have a field int equal to 2.
Running this command would result in only one write operation, so it's definitely better that a remove + insert.

I've implemented this update function and would update TinyDB soon.

Concerning the atomicity: At the moment TinyDB does not have real, fault-tolerant atomicity. I've found a interesting article about that topic: Reliable file updates with Python. If you need something closer to real atomacity, feel free to take a method from the article and implement a new storage class (see TinyDB docs) (and submit a PR).

@phreeza

This comment has been minimized.

Show comment
Hide comment
@phreeza

phreeza Jun 19, 2014

that interface is exactly what I had in mind, great. I assume this would update all matches to the query correspondingly?

I hadn't even gone as far as thinking about 'true' atomicity on the filesystem level. I think this will do just fine for my purposes.

phreeza commented Jun 19, 2014

that interface is exactly what I had in mind, great. I assume this would update all matches to the query correspondingly?

I hadn't even gone as far as thinking about 'true' atomicity on the filesystem level. I think this will do just fine for my purposes.

@msiemens msiemens closed this in 30d36f2 Jun 19, 2014

@msiemens

This comment has been minimized.

Show comment
Hide comment
@msiemens

msiemens Jun 19, 2014

Owner

That's exactly what it does :) I'll publish the new version of TinyDB right now.

Owner

msiemens commented Jun 19, 2014

That's exactly what it does :) I'll publish the new version of TinyDB right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment