Skip to content
Persistent dict, backed by sqlite3 and pickle, multithread-safe.
Python Makefile
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
tests Add `get_tablenames` method. Fix #71 (#72) Jan 26, 2018
.travis.yml Update sqlite version to 3.8.2 (#63) Jan 8, 2017
CHANGELOG.txt bump CHANGELOG to 1.6.0 Sep 18, 2018
LICENSE.apache Create LICENSE.apache May 24, 2018
LICENSE.rst Create LICENSE.rst May 24, 2018 Include license files in sdists May 24, 2018
Makefile Ensure debug log entries are captured by nose Apr 21, 2015
README.rst Replace easy_install installation with pip (#77) Jul 30, 2018 bump version to 1.6.0 Sep 18, 2018 Add `get_tablenames` method. Fix #71 (#72) Jan 26, 2018


sqlitedict -- persistent dict, backed-up by SQLite and pickle

Travis License

A lightweight wrapper around Python's sqlite3 database with a simple, Pythonic dict-like interface and support for multi-thread access:

>>> from sqlitedict import SqliteDict
>>> mydict = SqliteDict('./my_db.sqlite', autocommit=True)
>>> mydict['some_key'] = any_picklable_object
>>> print mydict['some_key']  # prints the new value
>>> for key, value in mydict.iteritems():
>>>     print key, value
>>> print len(mydict) # etc... all dict functions work
>>> mydict.close()

Pickle is used internally to (de)serialize the values. Keys are arbitrary strings, values arbitrary pickle-able objects.

If you don't use autocommit (default is no autocommit for performance), then don't forget to call mydict.commit() when done with a transaction:

>>> # using SqliteDict as context manager works too (RECOMMENDED)
>>> with SqliteDict('./my_db.sqlite') as mydict:  # note no autocommit=True
...     mydict['some_key'] = u"first value"
...     mydict['another_key'] = range(10)
...     mydict.commit()
...     mydict['some_key'] = u"new value"
...     # no explicit commit here
>>> with SqliteDict('./my_db.sqlite') as mydict:  # re-open the same DB
...     print mydict['some_key']  # outputs 'first value', not 'new value'


  • Values can be any picklable objects (uses cPickle with the highest protocol).

  • Support for multiple tables (=dicts) living in the same database file.

  • Support for access from multiple threads to the same connection (needed by e.g. Pyro). Vanilla sqlite3 gives you ProgrammingError: SQLite objects created in a thread can only be used in that same thread.

    Concurrent requests are still serialized internally, so this "multithreaded support" doesn't give you any performance benefits. It is a work-around for sqlite limitations in Python.

  • Support for custom serialization or compression:

    # use JSON instead of pickle
    >>> import json
    >>> mydict = SqliteDict('./my_db.sqlite', encode=json.dumps, decode=json.loads)
    # apply zlib compression after pickling
    >>> import zlib, pickle, sqlite3
    >>> def my_encode(obj):
    ...     return sqlite3.Binary(zlib.compress(pickle.dumps(obj, pickle.HIGHEST_PROTOCOL)))
    >>> def my_decode(obj):
    ...     return pickle.loads(zlib.decompress(bytes(obj)))
    >>> mydict = SqliteDict('./my_db.sqlite', encode=my_encode, decode=my_decode)


The module has no dependencies beyond Python itself. The minimum Python version is 2.5, continuously tested on Python 2.6, 2.7, 3.3 and 3.4 on Travis.

Install or upgrade with:

pip install -U sqlitedict

or from the source tar.gz:

python install


Standard Python document strings are inside the module:

>>> import sqlitedict
>>> help(sqlitedict)

(but it's just dict with a commit, really).

Beware: because of Python semantics, sqlitedict cannot know when a mutable SqliteDict-backed entry was modified in RAM. For example, mydict.setdefault('new_key', []).append(1) will leave mydict['new_key'] equal to empty list, not [1]. You'll need to explicitly assign the mutated object back to SqliteDict to achieve the same effect:

>>> val = mydict.get('new_key', [])
>>> val.append(1)  # sqlite DB not updated here!
>>> mydict['new_key'] = val  # now updated

For developers


# pip install nose
# pip install coverage

To perform all tests:

# make test-all

To perform all tests with coverage:

# make test-all-with-coverage

Comments, bug reports

sqlitedict resides on github. You can file issues or pull requests there.

sqlitedict is open source software released under the Apache 2.0 license. Copyright (c) 2011-now Radim Řehůřek and contributors.

You can’t perform that action at this time.