New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to insert multiple entries in one method call #8

Closed
jamiebullock opened this Issue Jul 21, 2014 · 5 comments

Comments

2 participants
@jamiebullock

jamiebullock commented Jul 21, 2014

It would be very useful if multiple entries could be inserted into a tinydb in one method call, by passing an array of dicts.

My preference would be to overload insert() so you could write:

db.insert([{'x':1}, {'x': 2}])

One common use case would be serialising multiple JSON records consumed via RESTful APIs

@msiemens

This comment has been minimized.

Show comment
Hide comment
@msiemens

msiemens Jul 21, 2014

Owner

Sounds reasonable! Maybe, instead of overloading insert(), we could add a insert_multiple()? That would save us a lot of headaches with checking if the argument is a list or list-like.

Owner

msiemens commented Jul 21, 2014

Sounds reasonable! Maybe, instead of overloading insert(), we could add a insert_multiple()? That would save us a lot of headaches with checking if the argument is a list or list-like.

@jamiebullock

This comment has been minimized.

Show comment
Hide comment
@jamiebullock

jamiebullock Jul 22, 2014

Part of me thinks that insert() should be smart enough to figure out what to do with its argument, mirroring pymongo's bulk insert for example.

The secret sauce in the pymongo insert is:

        docs = doc_or_docs
        return_one = False
        if isinstance(docs, dict):
            return_one = True
            docs = [docs]

So we could check if the argument is a dict, and if it is we wrap it in a list, then internally call insert_multiple() (or equivalent) on docs

Even if you exposed insert_multiple() in the API I reckon there's no way to ensure the user passed in a valid list.

OTOH, if you think overloading insert() causes too man problems, insert_multiple() would still be a welcome improvement!

jamiebullock commented Jul 22, 2014

Part of me thinks that insert() should be smart enough to figure out what to do with its argument, mirroring pymongo's bulk insert for example.

The secret sauce in the pymongo insert is:

        docs = doc_or_docs
        return_one = False
        if isinstance(docs, dict):
            return_one = True
            docs = [docs]

So we could check if the argument is a dict, and if it is we wrap it in a list, then internally call insert_multiple() (or equivalent) on docs

Even if you exposed insert_multiple() in the API I reckon there's no way to ensure the user passed in a valid list.

OTOH, if you think overloading insert() causes too man problems, insert_multiple() would still be a welcome improvement!

@msiemens

This comment has been minimized.

Show comment
Hide comment
@msiemens

msiemens Jul 22, 2014

Owner

I think in the end that's a philosophical question. I'm not comfortable with insert() being smart. I would prefer to minimize such magic, if possible. That way insert() may be dump, but it's also very predictable.

Even if you exposed insert_multiple() in the API I reckon there's no way to ensure the user passed in a valid list.

I wouldn't even try ensure that. The user may pass a generator or even some custom class implementing the contianer protocol. If the user passes something else, ìnsert_multiple() will simply break.

Owner

msiemens commented Jul 22, 2014

I think in the end that's a philosophical question. I'm not comfortable with insert() being smart. I would prefer to minimize such magic, if possible. That way insert() may be dump, but it's also very predictable.

Even if you exposed insert_multiple() in the API I reckon there's no way to ensure the user passed in a valid list.

I wouldn't even try ensure that. The user may pass a generator or even some custom class implementing the contianer protocol. If the user passes something else, ìnsert_multiple() will simply break.

@jamiebullock

This comment has been minimized.

Show comment
Hide comment
@jamiebullock

jamiebullock Jul 22, 2014

Fair enough. I agree that neither approach is "right" and I don't feel that strongly about it either way. TinyDB is a really nice package and the most important thing is that it's consistent with its own philosophy :)

jamiebullock commented Jul 22, 2014

Fair enough. I agree that neither approach is "right" and I don't feel that strongly about it either way. TinyDB is a really nice package and the most important thing is that it's consistent with its own philosophy :)

@msiemens

This comment has been minimized.

Show comment
Hide comment
@msiemens

msiemens Jul 22, 2014

Owner

BTW: I've just released v1.4.0 (GitHub and PyPI) containing the new insert_multiple() method :) Thank you for your suggestions and thoughts!

Owner

msiemens commented Jul 22, 2014

BTW: I've just released v1.4.0 (GitHub and PyPI) containing the new insert_multiple() method :) Thank you for your suggestions and thoughts!

@msiemens msiemens closed this Jul 22, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment