Add Table.write_back(), replacing documents by ids

[davidlatwe]

Compare to update(), replace() write_back() provide a more customizable workflow
to modify documents.
One can fistly search the documents and modify them by thier conditions,
then input new douments with thier doc_ids to replace() write_back().

Example Usage

from tinydb import TinyDB, Query
from tinydb.storages import MemoryStorage

db = TinyDB(storage=MemoryStorage)

db.purge()

dataset = [
    {"stock": ["cookies", "apple"]},
    {"stock": ["brownies"]},
    {"stock": ["cake"]},
    {"stock": ["cookies", "milk"]},
    {"stock": ["cake"]},
    {"stock": ["pie", "cake"]},
    {"stock": ["apple"]},
]

table = db.table('yumyum')
table.insert_multiple(dataset)
# search
yum = Query()
docs = table.search(yum.stock.any(["cake", "apple"]))
# modify docs
for doc in docs:
    for i, stock in enumerate(doc["stock"]):
        if stock == "cake":
            doc["stock"][i] = "water"
        if stock == "apple":
            doc["stock"][i] = "honey"
# write back
table.write_back(docs)

Or replace write_back with new documents

db.purge()

table = db.table('yumyum')
table.insert_multiple(dataset)

docs = table.all()
# save doc_ids
doc_ids = [doc.doc_id for doc in docs]
# modify docs
for i, doc in enumerate(docs):
    if len(doc["stock"]) == 1:
        docs[i] = {"best": doc["stock"][0]}
    elif len(doc["stock"]) > 1:
        docs[i] = {"good": doc["stock"]}

table.write_back(docs, doc_ids)

Could be a convenient feature :)

[davidlatwe]

Seems like pytest-runner become an issue that can't be solved, it starts failing, even current master branch.
Due to pytest-runner not able to find the package setuptools_scm in Python 2.6.

reference 1
reference 2

In the last commit a2992e6 is the best workaround I can get for now.

Back to the topic, please let me know what you think about this replace feature !
Thank you.

[hugovk]

Here's the pip installs for tinydb from PyPI for last month:

python_version	percent	download_count
2.7	44.5%	4,663
3.6	17.4%	1,826
3.5	15.2%	1,587
3.4	12.4%	1,299
3.2	3.2%	339
2.6	2.6%	269
3.3	2.4%	255
3.7	2.2%	235

Source: pypinfo --start-date -51 --end-date -21 --percent --pip --markdown tinydb pyversion

2.6 has been EOL for over 4 years. Can it be dropped?

[davidlatwe]

Just moved this to issue #185 :)

[msiemens]

Looks good to me 🙂 If @eugene-eeo doesn't have any objections, I'll be glad to merge this!

[eugene-eeo]

Yeah, looks great! My only nitpick would be the name of the method... replace is nice but writeback would be more precise on what is being done.

[davidlatwe]

replace is nice but writeback would be more precise on what is being done.

sure ! @eugene-eeo, I changed to write_back now, with an underscore.

---

Just found out that may cause doc_id duplicated error after this write_back (replace) action if one feeds with doc_id that is greater then _last_id, because this function write doc just like insert :

data[doc_id] = documents.pop()

So added a check in commit d0d0c8f

[davidlatwe]

OK, I also modified the example usage in top comment with the function name write_back, I am good to go :)

[davidlatwe]

Merged with latest master branch, setup.py cleaned.

[davidlatwe]

Hi @msiemens and @eugene-eeo , can we merge this ?
Or Docs should be update as well ?
Thanks :D

[eugene-eeo]

LGTM but it's up to @msiemens to merge. Thanks for contributing!

[msiemens]

Sorry for the silence, everyone. I'm having two exams this week and am a little short on time. All in all, this looks good to me too. But I'd bring up the question of naming again. From an API usability perspective, I'd really prefer replace() instead of write_back(). Most people immediately can recognize what replace() does but will struggle with write_back().

@eugene-eeo What do you think?

[eugene-eeo]

@msiemens yeah I see your point which is something I considered at first, but the way I saw this was that it is a "lower level" version of db.update because you're in charge of maintaining the IDs. Also the write_back name was inspired by the writeback option in shelve.open. However I don't mind if we use replace instead as it seems to be more intuitive.

[msiemens]

the way I saw this was that it is a "lower level" version of db.update because you're in charge of maintaining the IDs

That's a good point however. The question to me is whether this operation is something we would encourage everyday users or not. If we do encourage everyday use, replace is the better name. If this is meant to be a rather low level operation, we should rather go with write_back. I'm not sure what I prefer as one could argue for both cases (and we cannot change the behaviour significantly later on)…

[eugene-eeo]

I think it could be regarded as a lower level operation now, we can always change the method name with a deprecation warning in the future.

[msiemens]

That's a good idea! The last remaining question is where to document this. In Advanced Usage? Only in the API docs?

[eugene-eeo]

Both in the API docs and advanced usage, IMO, and in the API docs we'll put a warning saying that it can be dangerous to use the method if you don't know what you're doing.

[davidlatwe]

If in Advanced Usage, maybe could place it right after the operations example in Updating data section, since this is like a "lower level" version of update that giving user the ability to perform custom operation wildly.

P.S. @msiemens sorry, no mean to rush 😄 , hope for the best !

[msiemens]

Just a quick note: I haven't forgotten about this PR and 'll probably revisit it at the end of next week 🙂

[msiemens]

I've finally merged this and added some documentation. I'll release a new version of TinyDB in the next week or two 🙂

[davidlatwe]

Thank you @msiemens !

[msiemens]

This is now released in v3.8.0 🙂