New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some python syntax error in tinydb's document #199

licovery opened this Issue Apr 2, 2018 · 5 comments


3 participants

licovery commented Apr 2, 2018

>>> docs = == 'John')
[{name: 'John', age: 12}, {name: 'John', age: 44}]
>>> for doc in docs:
... = 'Jane'
>>> db.write_back(docs)  # Will update the documents we retrieved
>>> docs = == 'John')
>>> docs = == 'Jane')
[{name: 'Jane', age: 12}, {name: 'Jane', age: 44}]

I am puzzled by this:

>>> for doc in docs:
... = 'Jane'

It seems that the content of docs does not change at all.
I am not sure I'm right, I'm not an expert in python.


This comment has been minimized.

licovery commented Apr 2, 2018

It works when I use the operator[ ]

>>> for doc in docs:
...     doc['name'] = 'Jane'

This comment has been minimized.

licovery commented Apr 2, 2018

>>> for doc in docs:
...     print(

Python come up with an error
AttributeError: 'Document' object has no attribute 'name'


This comment has been minimized.

Phuket2 commented Apr 4, 2018

@licovery , Its nice you brought this up. I have been accessing the magic doc.doc_id without any idea where it comes from (even though it had cross my mind, it seems a bit magical). Just my lack of experience. But in the source in the, you can see the Document Class at the top of the file. The Document Class is subclassed from dict. I am not going to try and use correct terminology about instance variables etc because I am not confident enough to be sure.
But I am pretty sure the is a typo or error in the documentation as each doc you get back iterating over docs is bacially a dict. So you have to access it using doc['key'] notation. The doc_id is a member/instance (or whatever) variable of the Document Class. That's why you can access it with the doc.doc_id notation.

But it got me thinking maybe making my own Document style classes (Just a Class inheriting from dict) might be a nicer way to represent my data in code. The class can be passed directly to tinydb methods as its a dict, to recreate the class from the docs returned from tinydb should be fairly straight fwd (I think!)
But you can add extra functionality to your Class which still appears as a dict for all intent and purposes.
Below is my first attempt to try get the dot notation extended into the dict data. Eg so in My Document style Class I could do something like print( Problem is, for me its tricky. Not enough experience. I am pretty sure what I have done below is not correct. It works, well in my simple test it does. I could only get the dot notation working 1 level deep into the data. I tried added a dict into the data and did something like doc.other.age, that fails. Can do doc.other['age'] (were other is The key to a dict.

I have some other UN-filtered thoughts. Like it could be nice if you wanted to make your own document class/s, you create a class and inherit from Document, then inform tinydb of your Document class. I see after the Document class is defined in the source, below is a statement -
Element = Document
This could be a hack way. :) Not sure, but will play around more

Anyway, food for thought.

d0 = dict(age=36, dob='17/12/1990')
d1 = dict(first='Ian', last='Smith', other=d0)

class MyClass(dict):
    def __init__(self, id=0, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.my_id = id    # not sure if this could be a race condition
    def __getattr__(self, name):
        #print('in __getattr__')
            print('In getattr NO execption')
            return self[name]
            print('In getattr WITH execption')
            #raise AttributeError
    def __setattr__(self, name, value):
        #print('In setattr')
            print('In setattr NO execption')
            #self.setattr(name, value)
            print('In setattr WITH execption')
            super().__setattr__(name, value) # Not sure I need to do this
            # maybe its ok to raise an AttributeError
            #raise AttributeError

mc = MyClass(90, d1)
mc.my_id= 9
    mc.other.age = '90'
    print('Can not do nested dict dot notation')

mc.last = 'smith'        # modify a key 
#print(mc.error_attr)    # will and shoud attribute error

This comment has been minimized.

Phuket2 commented Apr 6, 2018

Its a bit of a digression, but below I did a test inheriting from Tinydb's Document class. Just interesting for me at least. It does go to idea of being able to specify your own Document class to tinydb though. If you inherit from the Document class, call the super and was able too assign your subclassed class to tinydb as the Document, would be no need to do what I did below after reading the data back in from the db (create new objects). I am not sure if you could do the same with custom storage/middleware. Anyway, Its just an idea. I haven't tried to use it for anything practical yet, just simple tests.

from tinydb import TinyDB
from tinydb.database import Document

from datetime import datetime as datetime

from faker import Faker
fake = Faker()

class PersonClass(Document):
    value is a dict
    doc_id is used internally by tinydb.  But we need to pass an Integer
    to be able to call the super.__init__ on the Document class.
    When creating these objects(from sratch), I am passing any int, could be 0.
    When creating these objects from data read back from the db, the doc_id,
    has some meaning as long as you pass the doc_id from the recovered database
    def __init__(self, value, doc_id, **kwargs):
        super().__init__(value, doc_id, **kwargs)
    def full_name(self):
        returns the full name of the person
        return "{} {}".format(self['first'], self['last'])
    def some_date_dt(self):
        The date stored in the db is a string as you have to.
        Just getting a datetime object here. Could help with
        serialisation if you dont need generic code. 
        the_date = datetime.strptime(self['some_date'], "%Y-%m-%d")
        return the_date
    def from_tinydb_doc(cls, obj):
        Create a PersonClass from a tinydb doc obj
        Not required for this simple test, but I am just practicing
        doing it.
        return cls(obj, obj.doc_id)         

def make_dummy_people(num_items=100):
    Create some dummy people to use...
    returns a list of PersonClass objects 
    The data built up using Faker
    return [
                {'idx': i,
                'first': fake.first_name(),
                'last': fake.last_name(),
                'age': fake.random_int(min=18, max=106),
            for i in range(0, num_items)

db_fn = 'doc_test.json'                  # database filename
dummy_data = make_dummy_people(10)

db = TinyDB(db_fn)                       # create TDB instance w/filename
db.purge_tables()                        # clear all the tables, just for testing
db.insert_multiple(dummy_data)           # add the dummy data to the new table

for doc in db.all():
    # creating PersonClass objects from the doc obj's
    # returned from tinydb.  Its double work, but could be worth it
    # for flexibilty.  With this simple example, very easy to re-create
    # your own document class from the db.
    # below line is commented, can use a normal creation or the
    # from_tinydb_doc class method below.  Just trying different things
    #person_doc = PersonClass(doc, doc.doc_id)
    # try a classmethod, to create the PersonClass
    person_doc = PersonClass.from_tinydb_doc(doc)
    # just print some stuff to show it works
    print(person_doc.doc_id, person_doc.full_name, person_doc.some_date_dt)

This comment has been minimized.


msiemens commented Apr 7, 2018

Seems like I got a little to much used to writing JavaScript/TypeScript lately where dot access works with objects 🙈 Anyway, the docs have been fixed in #205 🙂

@msiemens msiemens closed this Apr 7, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment