TinyMongo provides a familiar PyMongo-style document API backed by embedded storage instead of a MongoDB server. The default backend uses TinyDB, with optional memory, SQLite, DuckDB, Parquet, PostgreSQL, and MariaDB/MySQL backends.
TinyMongo supports Python 3.9 and newer and is tested in GitHub Actions on Python 3.9, 3.11, 3.13, and 3.14. CPython 3.14's free-threaded build is supported at the beta level for the backends described below.
The latest stable release is 1.3.0 and can be installed from PyPI:
pip install "tinymongo==1.3.0"For development, clone this repository and run pip install -e . from its
root. Use the v1.3.0 tag when you need source and documentation that match
the current stable package exactly; master may contain later changes. See
CHANGELOG.md
for the complete release history and any unreleased work.
The default JSON backend has a small dependency set. Optional database backends may install native binary wheels supplied by DuckDB, PyArrow, or SQL drivers.
- Roadmap: See the TinyMongo roadmap for planned compatibility, GridFS, Compass, wire-server, and browser work.
- Default storage: TinyMongo uses TinyDB-compatible JSON storage unless another backend is selected.
- Table-native backends: SQLite, DuckDB, and Parquet backends store one real table/file per collection instead of one serialized database blob.
- Concurrency: local JSON writes use atomic replace,
fsync, and advisory locks. Table-native and remote backends use their own transactional or file-replacement mechanisms; see the backend-specific documentation. - Async API: async clients keep storage and lock waits off the event loop while sharing the synchronous implementation's behavior.
- Tests & CI: a GitHub Actions workflow is included at
.github/workflows/ci.ymlto run unit tests and linters across Python versions. Seerequirements-dev.txtfor dev dependencies.
TinyMongo exposes MongoClient, ASCENDING, and DESCENDING aliases so small
PyMongo-style scripts can be tried against local file-backed storage by changing
the import:
import tinymongo as pymongo
client = pymongo.MongoClient(
"mongodb://localhost:27017",
serverSelectionTimeoutMS=2000,
tinymongo_folder="/path/to/folder",
)
users = client.app.users
users.insert_one({"email": "ada@example.com", "score": 7})
users.update_one({"email": "ada@example.com"}, {"$inc": {"score": 1}})
rows = list(users.find({}).sort("score", pymongo.DESCENDING))This is intended for the supported TinyMongo subset of PyMongo operations, not
for server features such as authentication, replica sets, sessions, or network
connections. MongoDB URIs, host names, ports, and recognized network connection
kwargs are accepted and ignored so existing code can be tried locally. When
PyMongo is installed, TinyMongo derives those accepted option names from that
installed version; dependency-free installs use a bundled fallback list.
Unknown or misspelled kwargs fail eagerly with ConfigurationError, matching
PyMongo's configuration behavior instead of silently selecting the wrong local
storage. Set TINYMONGO_HOME or pass tinymongo_folder= to choose where
TinyMongo stores files. See examples/pymongo_dropin.py for a runnable example.
The behavior-bearing read options are not ignored. document_class constructs
top-level and nested result documents—including documents inside arrays—with a
mutable mapping class such as OrderedDict or bson.SON. Datetimes are stored
as signed UTC milliseconds, just like BSON. Results are naive UTC by default;
set tz_aware=True for aware UTC values, and optionally pass tzinfo= to
convert the same instant to another timezone. These rules apply equally to the
synchronous and asynchronous PyMongo-shaped clients. Raw BSON views via
RawBSONDocument are not part of the embedded-storage document-class subset.
AsyncMongoClient and AsyncTinyMongoClient expose non-blocking client,
database, collection, and cursor APIs. find() is immediate and lazy; storage
work begins when the cursor is awaited or iterated:
from tinymongo import AsyncMongoClient
async def load_users():
async with AsyncMongoClient(
tinymongo_folder="./tinydb",
backend="sqlite",
) as client:
users = client.app.users
await users.insert_one({"_id": 1, "name": "Ada", "score": 9})
return await users.find({}).sort("score", -1).to_list(length=None)Async cursors support async for, to_list(), sort(), skip(),
limit(), clone(), rewind(), and close(). Potentially blocking
storage, serialization, locking, query, and cleanup work runs outside the event
loop.
tinymongo.patch() temporarily routes pymongo.MongoClient and
pymongo.AsyncMongoClient calls to TinyMongo
without making PyMongo a required TinyMongo dependency. Install PyMongo only for
tests that use the helper:
pip install "tinymongo[pymongo]"The default backend is an isolated in-memory database. Clients created inside one patch scope share data, and the original PyMongo client is restored even if the test raises an exception:
import pymongo
import tinymongo
with tinymongo.patch():
writer = pymongo.MongoClient("mongodb://ignored")
reader = pymongo.MongoClient()
writer.app.users.insert_one({"name": "Ada"})
assert reader.app.users.count_documents({}) == 1The same helper can decorate a test and can select a folder and backend:
import pymongo
import tinymongo
@tinymongo.patch(folder="./test-data", backend="sqlite")
def test_application_code():
client = pymongo.MongoClient()
assert client.server_info()["storage"] == "sqlite"It can also decorate a unittest method:
import unittest
import pymongo
import tinymongo
class ApplicationTest(unittest.TestCase):
@tinymongo.patch()
def test_application_code(self):
client = pymongo.MongoClient()
client.app.users.insert_one({"name": "Ada"})
self.assertEqual(client.app.users.count_documents({}), 1)Patch scopes may be nested. Code must look up pymongo.MongoClient while the
scope is active; a MongoClient name imported directly before the patch cannot
be replaced. Because PyMongo's module attribute is process-global, patch scopes
cannot overlap across threads. Async tests should put a with tinymongo.patch()
block inside the async function instead of decorating the coroutine. Prefer
async with tinymongo.patch() when creating async clients so cleanup can be
awaited.
TinyMongo defaults to TinyDB's JSON storage:
from tinymongo import TinyMongoClient
connection = TinyMongoClient("/path/to/folder")You can select another backend with the backend argument:
memory_connection = TinyMongoClient(backend="memory")
parquet_connection = TinyMongoClient("/path/to/folder", backend="parquet")
sqlite_connection = TinyMongoClient("/path/to/folder", backend="sqlite")
duckdb_connection = TinyMongoClient("/path/to/folder", backend="duckdb")
postgres_connection = TinyMongoClient(
backend="postgres",
dsn="postgresql://user:password@localhost:5432/tinymongo",
)TinyMongoClient and AsyncTinyMongoClient also accept
tinymongo_folder= as an alias for foldername. If a non-default
foldername is also supplied, the two values must agree. Unknown constructor
keywords raise TypeError instead of being silently ignored. The other
supported backend configuration keywords are threads, storage_uri,
duckdb_config, and dsn.
The PyMongo-shaped MongoClient and AsyncMongoClient accept the installed
PyMongo version's recognized connection kwargs for drop-in use and ignore their
network effects. Unknown or misspelled kwargs raise ConfigurationError before
storage opens. document_class, tz_aware, and tzinfo are validated before
storage opens and control returned values recursively.
Parquet can also store collection files in object storage by passing
storage_uri or setting TINYMONGO_STORAGE_URI. Object-storage Parquet is
experimental and currently uses one Parquet file per collection, so
updates/deletes rewrite that file:
s3_connection = TinyMongoClient(
"/unused-local-path",
backend="parquet",
storage_uri="s3://my-bucket/tinymongo",
)When storage_uri is set, it fully determines the Parquet data location. The
API folder argument is optional and ignored; the CLI path argument remains a
required placeholder. Neither creates a local cache or fallback directory.
Remote SQL treats its path argument the same way. Environment-only
TINYMONGO_STORAGE_URI and remote DSN settings also work for CLI database
discovery and migration summaries.
Available backends:
memory: Process-local storage that creates no database or lock files. Each unnamed client is isolated; amemory://NAMEURI explicitly shares a named namespace within one process.tinydborjson: TinyDB-compatible JSON storage. This is the default and writes.jsonfiles.sqlite: Table-native SQLite storage using one SQL table per collection. This writes.sqlitefiles.duckdb: Table-native DuckDB storage using one DuckDB table per collection. This writes.duckdbfiles.parquetorparquetv2: DuckDB-managed Parquet dataset storage using one Parquet file per collection inside a.parquetdirectory.postgresorpostgresql: Remote PostgreSQL storage using one SQL table per database collection.mysqlormariadb: Remote MariaDB/MySQL storage using one SQL table per database collection.
Install only the drivers you need:
pip install "tinymongo[duckdb]"
pip install "tinymongo[parquet]"
pip install "tinymongo[postgres]"
pip install "tinymongo[mysql]"
pip install "tinymongo[bson]"
pip install "tinymongo[pymongo]"
pip install "tinymongo[serialization]"If an optional driver is missing, selecting that backend raises an ImportError
with the corresponding installation command. PyMongo is not a core runtime
dependency. It is selected at runtime for ObjectId, Decimal128, non-generic
Binary subtypes, patching, and conditional PyMongo exception inheritance,
and is also used by development compatibility tests. Install
tinymongo[bson] for BSON values or tinymongo[pymongo] for patching and
installed-version compatibility.
For CPython 3.14t, install the dependency profile that has been verified while the GIL is disabled:
python3.14t -m pip install "tinymongo[free-threaded]"This profile covers the core API, memory, TinyDB/JSON, SQLite, BSON/PyMongo,
MySQL/MariaDB, and PostgreSQL through pure Psycopg. Pure Psycopg requires the
system libpq library. DuckDB and Parquet are not yet available in this
profile because DuckDB does not publish a cp314t wheel; psycopg-binary
likewise has no cp314t distribution. The required 3.14t CI lanes verify that
the GIL remains disabled and exercise in-process concurrency, MongoDB
contracts, remote SQL, and built-package installation. Contributors using
3.14t should install requirements-free-threaded.txt instead of
requirements-dev.txt.
Free-threaded support does not make every object safe for simultaneous
mutation. Shared client, database, and collection handles support independent
operations, but applications should coordinate lifecycle calls such as
close() and drop_database() and should not consume one cursor or iterator
from multiple threads.
| Backend | Dependency | Best fit | Notes |
|---|---|---|---|
memory |
None | Isolated tests and temporary data | Creates no files. Named memory://NAME namespaces can be shared only within one process. |
tinydb / json |
TinyDB | Default local JSON files | Human-readable and simplest to inspect. |
sqlite |
Python standard library | Embedded transactional storage | Uses _id primary keys and JSON document payloads in collection tables. |
duckdb |
duckdb |
SQL-backed local analytics workflows | Uses real DuckDB collection tables and SQL JSON predicates where supported. |
parquet / parquetv2 |
duckdb, pyarrow |
Columnar local or object-storage workflows | Stores collection Parquet files that DuckDB reads and writes. |
postgres / postgresql |
tinymongo[postgres] |
Remote transactional storage | Stores documents in PostgreSQL tables with JSONB payloads. |
mysql / mariadb |
tinymongo[mysql] or tinymongo[mariadb] |
Remote transactional storage | Stores documents in MariaDB/MySQL tables with JSON payloads. |
For test isolation or scratch data, select the memory backend without a named address. Every client receives a separate in-memory database and creates no files:
from tinymongo import TinyMongoClient
client = TinyMongoClient(backend="memory")
client.app.users.insert_one({"name": "Ada"})
assert client.app.users.count_documents({}) == 1
client.close()Use a named URI only when clients in the same process need to share data. A named namespace remains available after a client closes and can be reopened until the process exits:
writer = TinyMongoClient("memory://shared-test", backend="memory")
writer.app.users.insert_one({"name": "Grace"})
writer.close()
reader = TinyMongoClient("memory://shared-test", backend="memory")
assert reader.app.users.find_one({"name": "Grace"}) is not NoneMemory data never persists across process restarts, and named namespaces are not safe for sharing between processes. Prefer unnamed clients, or unique names, in independent tests. Use a durable backend instead when data must survive a test run or application restart. The command-line tool intentionally omits the memory backend because every CLI invocation exits immediately; use it through the Python API instead.
SQLite, DuckDB, and Parquet compile supported Mongo-style filters into SQL over
the _id column and JSON document payload. Unsupported filter shapes fall back
to Python document matching so existing TinyMongo behavior remains available.
Older blob-format SQLite and DuckDB files are migrated to collection tables when
opened.
Local load-test results for these backends are documented in backend benchmarks.
Object-storage setup examples for S3, S3-compatible providers, Backblaze B2,
Cloudflare R2, Google Cloud Storage, Azure Blob Storage, MinIO, Wasabi, and
DigitalOcean Spaces are documented in
the object-storage guide.
PostgreSQL and MariaDB/MySQL setup is documented in
the remote SQL guide.
Remote SQL drivers are optional; if one is missing, TinyMongo raises an
ImportError with the exact pip install ... command to run.
The package installs a tinymongo command for inspecting and moving data:
tinymongo inspect ./tinydb
tinymongo list-dbs ./tinydb
tinymongo list-collections ./tinydb my_tiny_database
tinymongo export ./tinydb my_tiny_database users -o users.json
tinymongo import ./tinydb my_tiny_database users users.json --mode replace
tinymongo migrate ./tinydb ./sqlite-db --to-backend sqliteUse --backend with inspect, list-dbs, list-collections, export, and
import when reading or writing a non-default backend:
tinymongo inspect ./sqlite-db --backend sqlite
tinymongo export ./parquet-db app users --backend parquet -o users.json
tinymongo inspect ./unused --backend parquet --storage-uri s3://my-bucket/tinymongo
tinymongo migrate ./tinydb ./unused --to-backend parquet --target-uri s3://my-bucket/tinymongo
tinymongo migrate ./tinydb ./unused --to-backend postgres --target-dsn "$TINYMONGO_POSTGRES_DSN"Export and import use the same tagged JSON codec as storage, so supported
datetime, ObjectId, bytes, and Binary values remain portable. Datetimes use
BSON's canonical naive-UTC millisecond representation. bytearray is accepted
and imports back as bytes. Inspection, collection listing, and migration omit
TinyDB's internal _default table. Export preserves embedded-document field
order, including document-valued _id values. Replace-mode imports and
migrations preflight the complete write with the destination's indexes before
deleting existing documents, and restore those documents if deletion or the
final insertion fails.
Unit tests exclude integration stress tests by default. Run the normal suite with:
pytestRun local integration stress tests explicitly with:
pytest -m integrationThe concurrent write stress tests are configurable with environment variables:
TINYMONGO_INTEGRATION_PROCS=32 \
TINYMONGO_INTEGRATION_WRITES_PER_PROC=100 \
pytest -m integration tests/integration/test_concurrent_writes.pyThat default bulk-write run produces 3,200 concurrent writes. For a larger local run:
TINYMONGO_INTEGRATION_PROCS=64 \
TINYMONGO_INTEGRATION_WRITES_PER_PROC=250 \
pytest -m integration tests/integration/test_concurrent_writes.pyThe single-insert smoke test can be tuned separately:
TINYMONGO_INTEGRATION_SINGLE_PROCS=16 \
TINYMONGO_INTEGRATION_SINGLE_WRITES_PER_PROC=100 \
pytest -m integration tests/integration/test_concurrent_writes.pyUse TINYMONGO_INTEGRATION_BACKEND=sqlite or another supported backend to run
the same integration tests against a non-default backend.
TinyMongo intentionally implements a practical subset of PyMongo's collection
API. It supports common inserts, finds, updates, deletes, sorting, pagination,
distinct values, collection counting, find_one_and_delete(), database
listing/removal, index inspection, and batched index creation. Cursors support
single- and multi-key sorting, skip(), limit(), clone(), close(),
and to_list(); limit(0) means no limit.
Query support includes equality (including scalar matches against array
members), nested document paths, $gt, $gte, $lt, $lte, $ne,
$nin, $in, $all, $and, $or, $nor, $not, $regex,
case-insensitive $options, $exists, $size, $elemMatch, $type, and
$mod. $comment metadata is accepted and ignored at the top level and inside
document-form $elemMatch filters; using $comment as a field operator raises
OperationFailure with MongoDB error code 2. $size matches an exact array
length, while $elemMatch requires one array member to satisfy the complete
nested condition. $type accepts MongoDB type names, numeric codes, or a list
of either; $mod follows MongoDB's integer-truncation and array-member matching
rules.
Every CRUD method that accepts a filter validates it before reading or changing
storage. Unsupported or misspelled $-prefixed query operators raise
TinyMongoNotSupportedError instead of silently returning no matches, and
malformed operands for recognized operators raise OperationFailure.
$not accepts a non-empty query document or a compiled native/BSON regex;
bare strings, numbers, lists, and empty documents raise OperationFailure
with MongoDB error code 2.
MongoDB treats missing fields differently depending on the negated value.
{"field": {"$ne": None}} and $nin lists containing None exclude
documents where field is missing, while $ne and $nin with only non-null
values continue to include missing fields. The same rule applies to dotted
paths and array members across TinyMongo backends.
Update support includes $set, $unset, $inc, $min, $max, $rename,
$push, $pop, $pull, and $addToSet, including upsert=True. As in
PyMongo, update_one() and update_many() require update operators; use
replace_one() for full-document replacement. $push accepts $each,
$position, $sort, and $slice; $addToSet accepts $each; and $pull
accepts literal, query, and document operands.
$min and $max set a missing field or replace an existing value only when
the candidate is lower or higher in TinyMongo's MongoDB-compatible BSON order.
$rename moves an existing field without creating a value for a missing
source. $pop removes the first array item for -1 or the last for 1, and is
a no-op for a missing or empty array. Dotted paths are supported; the new
operators also follow numeric array positions where MongoDB permits them, while
$rename rejects array-element source and destination paths.
Update documents are validated before any change is stored. Non-document
operator operands, invalid or conflicting paths, traversal through a scalar,
non-array $pop targets, and attempts to change _id raise
PyMongo-compatible WriteError exceptions with the applicable MongoDB code,
and the complete update remains atomic. Applying $addToSet to an existing
null or non-array field similarly raises WriteError code 2.
find() and find_one() accept Mongo-style inclusion and exclusion
projections. Dotted paths project nested fields, _id follows MongoDB's special
include/exclude rules, and projection happens after filtering and sorting:
users.find({"active": True}, {"profile.email": 1, "_id": 0})
users.find_one({"email": "ada@example.com"}, {"password_hash": 0})Inclusion and exclusion cannot be mixed except for _id. Computed expressions,
$meta, positional projection, and numeric array positions are rejected with a
clear error. The second positional argument to find() is now the PyMongo-style
projection argument; pass sorting as sort= or call .sort() on the cursor.
For unsorted SQLite scans, TinyMongo applies a normalized projection as each row
is read and releases the complete source document before reading the next row.
The common {"_id": 1} sweep goes further and reads only each JSON _id value,
so large unrequested payloads never enter Python memory. Filters still evaluate
against complete source values when Python-side matching is required.
SQLite find() cursors defer their scan until first consumption. For
unindexed filters handled by SQLite's existing SQL compiler, a final skip()
and limit() can therefore be applied before later payloads are read.
Python-only filters stream until that result window is filled; paths that must
merge scalar and array index candidates retain their complete-candidate
fallback. count_documents() uses a native SQL count for the same SQL-routed
filters and otherwise counts a row-at-a-time scan without retaining documents.
The same paths back synchronous and asynchronous collections.
This memory bound has explicit limits. Sorting must see unprojected sort keys, so a sorted SQLite cursor falls back to complete source documents before it projects the results. Other TinyMongo backends also continue to materialize complete matched documents before projection. Finally, exclusion projections that return most of each document naturally retain the size of those results.
Collections expose durable indexes and unique constraints:
collection.create_index("email", unique=True, name="login_email")
collection.find({"email": "person@example.com"})
collection.list_indexes()
collection.drop_index("login_email")Index definitions and unique constraints survive client restarts on persistent
backends. SQLite, PostgreSQL, and MariaDB/MySQL create native database indexes;
JSON, DuckDB, and Parquet persist metadata and enforce unique constraints through
TinyMongo. Named memory databases retain metadata while their process-local
namespace exists. Duplicate _id and unique-index violations raise
DuplicateKeyError with MongoDB-compatible code 11000.
Plural create_indexes() accepts duck-typed PyMongo IndexModel batches
without importing PyMongo in TinyMongo's core. Unique indexes are enforced.
Performance-only declarations degrade safely and emit
TinyMongoUnsupportedWarning: descending and hashed declarations use
ascending equality indexes, and compound declarations use their ascending
leading-field prefix. Sparse membership is not honored and TTL expiration is
not performed; both are reported explicitly. A semantically unsafe unique
declaration is rejected before any batch entry is created.
Unique indexes support JSON scalar values, Decimal128, UUID/Binary, regex, and
flat arrays on embedded backends; missing and null share one unique key.
Object values, ObjectId, datetime, nested arrays, non-finite numbers, and array
traversal inside a dotted index path are not supported for unique indexes yet.
Remote SQL stores a versioned canonical token digest beside each unique-indexed
value and protects it with a native constraint. This preserves exact int/float
numeric identity across processes, including very large integers and doubles,
while keeping booleans distinct from numbers. Remote SQL still rejects array,
Decimal128, UUID/Binary, and regex values under unique indexes because those
tokens cannot yet guarantee MongoDB multikey or BSON identity.
SQL, DuckDB, and Parquet storage uses typed physical _id keys for new rows.
Existing databases with older stringified keys remain readable and mutable.
BSON-equivalent IDs share one key (1 and 1.0, or native bytes and Binary
subtype 0), while BSON-distinct values such as True and 1 or identical
binary data with different subtypes remain separate.
Cursor sorting follows MongoDB's recursive comparison order for the supported
BSON scalar, document, and array families, including BinData, ObjectId,
booleans, and datetimes. The same order is used by aggregation $min and
$max and by the $gt, $gte, $lt, and $lte query operators, including
range predicates inside $pull. Range queries apply MongoDB's BSON type
bracketing: numeric representations share one family, while values from other
type families do not compare with one another. Array fields expose both the
whole array and its direct members where MongoDB does, and documents and arrays
are compared recursively. Values with equal comparison keys retain their input
order for ascending and descending sorts. TinyMongo uses its shared Python
matcher whenever a backend-native or indexed predicate cannot guarantee these
rules. Extending unique-index identity to the remaining supported BSON values
is still tracked separately in the roadmap and continues to fail closed where
exact enforcement is unavailable. Update and aggregation $min and $max
share this BSON comparison order, including recursive document and array
comparisons and numeric equivalence across supported numeric representations.
Collection.aggregate() supports the production-driven core of $match,
$sort, $skip, $limit, $count, $project, $set, $addFields, $unset,
and $group. $match uses the same query operators as find(). $project
supports inclusion, exclusion, renamed or computed fields, nested
specifications, dotted output paths through arrays, and MongoDB's special
_id rules. The available projection expressions are $ifNull, $literal,
and $size; $$REMOVE conditionally omits or removes a field. $group accepts
a field-path or None _id and the $addToSet, $avg, $first, $last,
$max, $min, $push, and $sum accumulators:
activity = events.aggregate(
[
{"$match": {"course_id": {"$in": course_ids}}},
{
"$project": {
"course_id": 1,
"count": {"$size": {"$ifNull": ["$lectures", []]}},
}
},
{
"$group": {
"_id": "$course_id",
"total": {"$sum": "$count"},
}
},
]
)
rows = activity.to_list()In the async API, await aggregate() to obtain an async cursor, then use
async for or await cursor.to_list(). $sort accepts up to 32 ascending or
descending keys and shares find().sort() field-path, array, and BSON ordering
rules. $skip accepts a nonnegative 64-bit integer, $limit requires a positive
64-bit integer, and $count emits one named count document—or no document for
empty input. Integral floating-point values are accepted for sort directions,
skip, and limit like MongoDB. Compound sorts preserve values from the same
array element and reject independent parallel arrays; canonical numeric path
parts select array indexes.
$set and $addFields are aliases that retain the input document, support
nested and dotted array-aware writes, and evaluate every right-hand expression
against the original stage input. $unset accepts one field name or a nonempty
list of field names and follows exclusion projection behavior. Numeric
array-index paths remain unsupported for projection output.
$min and $max ignore null and missing inputs unless every input is null or
missing, in which case they return null. $sum and $avg ignore missing and
nonnumeric values; $avg returns null when it sees no numbers and returns a
Decimal128 result when any input is Decimal128. $first and $last follow
the incoming document order and turn a selected missing value into null.
$push retains input order, skips missing values, and keeps explicit nulls;
$addToSet applies recursive BSON equality while making no output-order
promise. Accumulator operands may use TinyMongo's supported expressions; wrap
a literal array in $literal because a direct array is parsed as an invalid
argument list, matching MongoDB. An empty input produces no groups, including
for _id: None.
TinyMongo keeps first-seen group order for repeatable local results, but—as with
MongoDB—$group output order is not a public guarantee.
Other stages, accumulators, expressions, and aggregation options raise
TinyMongoNotSupportedError with the unsupported feature named. The structured
client.capabilities()["aggregation"] value lists the exact supported stages,
accumulators, and expressions; client.supports("aggregation") reports whether
any aggregation subset is available. In particular, $replaceRoot,
$replaceWith, $meta sort expressions, variables other than $$REMOVE, and
MongoDB's broader expression language are not part of this slice yet.
datetime, bytes, native uuid.UUID, and compiled re.Pattern values are
supported by every backend. Datetimes are converted to BSON's signed UTC
millisecond representation; naive inputs are treated as UTC and aware inputs
are converted to UTC. bytearray is accepted and reads back as bytes. Install
the optional BSON extra to use bson.ObjectId,
bson.Decimal128, bson.Regex, or non-generic bson.Binary subtypes without
making PyMongo a core dependency:
pip install "tinymongo[bson]"When BSON support is installed, TinyMongo gives writes that omit _id a native
ObjectId, including insert_many() and upsert paths. The returned value can
be reconstructed with ObjectId(str(result.inserted_id)). A dependency-free
installation falls back to a 32-character UUID string. The public
tinymongo.generate_id() helper always returns that portable UUID string for
callers, such as MongoEngine models, that explicitly want string IDs. Existing
string and integer IDs remain readable and are never rewritten.
from datetime import datetime
import re
from uuid import UUID
from bson import Binary, Decimal128, ObjectId, Regex
episode_id = ObjectId()
episodes.insert_one(
{
"_id": episode_id,
"created_date": datetime.now(),
"price": Decimal128("19.95"),
"image": b"\x89PNG\r\n\x1a\n",
"asset_id": Binary(bytes(range(16)), subtype=4),
"session_id": UUID("00112233-4455-6677-8899-aabbccddeeff"),
"topic_pattern": Regex("python|mongodb", "i"),
"local_pattern": re.compile("async", re.IGNORECASE),
"title": "Async Python",
}
)
episode = episodes.find_one({"_id": episode_id})JSON-backed storage uses an explicit tagged representation and restores native
values on read. Existing plain JSON files and string or integer IDs remain
compatible. Binary payloads are base64-encoded, which increases their stored
size by roughly one third. Generic subtype 0 reads back as bytes; other
subtypes preserve bson.Binary.subtype. The exact two-key mapping shape using
__tinymongo_type_v1__ and value is reserved for TinyMongo's persistence
codec; new user mappings with that shape are escaped automatically. If an
older database already contains that valid tag shape as ordinary data, whether
written through an earlier API or edited manually, rename one of those keys
before upgrading so it is not interpreted as the tagged value. UUID values use
the RFC-4122 byte order and share BSON identity with subtype-4 Binary values.
TinyMongo consistently uses this standard UUID representation; unlike a
default PyMongo client, it does not require a per-client UUID representation
setting for native UUID values.
Native compiled patterns retain their Python representation; bson.Regex
values retain their BSON representation, pattern, and flags. Regex identity is
the pattern plus MongoDB's canonical option string. Preserving a native
re.Pattern on read is a TinyMongo convenience extension; PyMongo decodes the
same wire value as bson.Regex. An implicit regex filter,
$regex, $in, $nin, $all, or $not pattern-matches strings and exact
stored regex values, while explicit $eq compares only stored regex identity.
Regex values used directly as operands of $gt, $gte, $lt, or $lte are
rejected with MongoDB error code 2. A regex nested inside a document or array
range operand is comparison data instead, so TinyMongo validates its BSON
transport shape without compiling it as an executable pattern.
TinyMongo evaluates patterns with Python's re engine rather than MongoDB's
PCRE2 engine, so a pattern must be valid Python syntax and some advanced syntax
or matching details can differ between the two engines. A stored BSON regex
using the locale flag can still be retrieved by exact regex identity even when
Python cannot apply that flag to a Unicode string predicate.
Non-finite floats (NaN, positive infinity, and negative infinity) also use
strict JSON-safe tags. Remote SQL keeps the encoded document as a normal,
indexable object in its existing JSON/JSONB data column and stores a second
copy in the nullable text data_ordered column to preserve embedded-document
field order. Older rows without that copy remain readable through data;
rewriting one populates its ordered copy.
Sorts normalize naive datetimes as UTC and convert aware datetimes to UTC.
BinData—including UUID—sorts by length, subtype, and then unsigned bytes,
matching MongoDB; legacy subtype 2 includes its four-byte inner-length prefix
when its comparison length is calculated. Regex values sort by pattern and
canonical options after the other supported scalar families. Numeric NaN
sorts below every other numeric value, matching MongoDB. Persistence, returned
values, equality, range comparisons, and sorting all use MongoDB's signed UTC
millisecond precision.
Decimal128 values retain their exact BSON representation and participate in
numeric equality and range queries, sorting, embedded-backend unique indexes,
$inc, and the supported $group numeric accumulators. Remote SQL rejects
Decimal128 values under unique indexes until its native constraints can enforce
MongoDB-equivalent numeric identity across concurrent writers. Reading
Decimal128 requires the same optional BSON extra used to write it.
When a sort encounters a genuinely unsupported value type, TinyMongo emits one
TinyMongoUnsupportedWarning per field and type instead of silently returning
insertion order.
insert_many() accepts any iterable of document dictionaries and honors
PyMongo's ordered option, which defaults to True. An ordered batch keeps the
successful prefix and stops at the first duplicate-key error; an unordered
batch inserts every valid document and reports all duplicate failures in a
PyMongo-shaped BulkWriteError. Values are encoded before storage changes, so
a client-side serialization failure leaves the entire TinyMongo input
unwritten. This is a stronger whole-list guarantee than PyMongo provides for
very large inputs, which it may split across multiple wire batches.
An unsupported value raises tinymongo.InvalidDocument before storage is
changed. The exception retains the rejected document in its document
attribute, and its message identifies the collection and nested value path
(plus the batch index for insert_many()). With PyMongo installed, it can be
caught as either bson.errors.InvalidDocument or pymongo.errors.PyMongoError;
dependency-free callers can catch tinymongo.errors.TinyMongoError.
TinyMongo includes PyMongo-shaped contract tests that run application code with
import pymongo redirected to TinyMongo:
pytest tests/test_pymongo_contract.py tests/test_pymongo_dropin.pyThe shared compatibility contracts run the same application-facing behaviors against every embedded backend:
pytest tests/contractsTo include a real MongoDB server explicitly:
TINYMONGO_MONGODB_URI=mongodb://127.0.0.1:27017/?directConnection=true \
pytest -o addopts='' -q -m 'contract and mongodb' tests/contractsUse -m contract instead of -m 'contract and mongodb' to run the complete
embedded-plus-MongoDB matrix in one session.
CI publishes the JUnit results together with deterministic JSON and Markdown reports containing per-backend outcomes and a documented compatibility score. See Compatibility reports to generate them locally and understand how passes, expected gaps, skips, and the MongoDB reference affect the score.
The shared Talk-Python-derived contracts run through both TinyMongo API modes. To exercise the actual application without rewriting its PyMongo call sites, follow the Talk Python acceptance run.
PyMongo remains optional. It is needed for these comparisons,
tinymongo.patch(), ObjectId and Decimal128 support, and non-generic
Binary subtypes, but it is not required for normal TinyMongo clients,
datetime storage, or native subtype-0 byte values. When it is installed,
TinyMongo error classes also inherit the matching pymongo.errors classes, so
existing PyMongoError handlers continue to work. InvalidDocument
additionally preserves BSON's standard error identity while remaining inside
TinyMongo's portable error hierarchy.
PyMongo's full upstream driver test suite targets a real MongoDB server and driver internals, so it is not expected to pass against TinyMongo. The contract tests are the supported compatibility boundary for local file-backed usage.
TinyMongo reports behavior that each configured backend can honor:
client = TinyMongoClient("./data", backend="sqlite")
capabilities = client.capabilities()
print(capabilities)
print(capabilities["query_operators"]["field"])
print(capabilities["query_operators"]["ignored"])
print(capabilities["update_operators"]["operators"])
print(capabilities["update_operators"]["modifiers"])
print(capabilities["bson_types"]["pymongo"])
print(client.supports("multiprocess_writes"))The capability map covers persistence, remote and object storage, table-native
storage, multiprocess writes, native indexes, projections, bulk writes,
aggregation, query and update operators, BSON types, sessions, transactions,
and change streams. query_operators separates top-level logical operators
from field operators and accepted-but-ignored metadata operators such as
$comment. update_operators lists every supported update operator and maps
$push and $addToSet to their accepted modifiers.
bson_types separates dependency-free native families from the richer
pymongo types available when the optional BSON extra is installed. These
values are structured mappings; use client.supports() for a Boolean feature
check or inspect their tuples when selecting a particular operator or type.
Unknown capability names raise ValueError so configuration mistakes are
visible.
For local persistent backends, multiprocess_writes=True promises safe writes,
not parallel write throughput. TinyDB JSON, SQLite, and DuckDB clients sharing
one storage folder serialize writes across databases and collections through a
store-wide advisory lock. Parquet uses one lock per logical database directory.
Lock acquisition waits for up to 30 seconds before timing out. SQLite uses WAL
mode, so reads can continue while another process holds the write lock, but its
writes remain serialized.
Operations whose semantics TinyMongo cannot honor raise
TinyMongoNotSupportedError. This includes sessions, transactions, change
streams, aggregation features outside the documented core subset, bulk writes,
database commands, non-default read/write concerns, and unsupported index
specifications. Connection options that only describe an ignored network target
remain harmless for drop-in use.
bypass_document_validation is accepted as a compatibility no-op and never
disables _id or unique-index enforcement. Where a compatibility method
accepts a session keyword, session=None is allowed; non-None sessions and
start_session() remain unsupported.
Basic MongoEngine CRUD is supported by passing TinyMongo as the client class.
The example keeps a string primary key for maximum portability. Native
ObjectId values also round-trip when tinymongo[bson] is installed:
import mongoengine as me
import tinymongo
me.connect(
"app",
host="mongodb://localhost",
mongo_client_class=tinymongo.MongoClient,
tinymongo_folder="./tinydb",
uuidRepresentation="standard",
)
class Person(me.Document):
id = me.StringField(primary_key=True, default=tinymongo.generate_id)
name = me.StringField(required=True)The tested subset covers document creation, repeated saves, queries, updates, deletes, counts, and collection drops. Aggregation beyond the documented core subset, sessions, and MongoDB server features remain outside TinyMongo's compatibility scope.
The quick start is shown below. For a more detailed look at tinymongo, take a look at demo.py within the repository.
from tinymongo import TinyMongoClient
# you can include a folder name or absolute path
# as a parameter if not it will default to "tinydb"
connection = TinyMongoClient()
# either creates a new database file or accesses an existing one named `my_tiny_database`
db = connection.my_tiny_database
# either creates a new collection or accesses an existing one named `users`
collection = db.users
# insert data adds a new record returns _id
record_id = collection.insert_one({"username": "admin", "password": "admin", "module":"somemodule"}).inserted_id
user_info = collection.find_one({"_id": record_id}) # returns the record inserted
# you can also use it directly
db.users.insert_one({"username": "admin"})
# returns a list of all users of 'module'
users = db.users.find({'module': 'module'})
# update data and inspect matched_count or modified_count on the result
upd = db.users.update_one({"username": "admin"}, {"$set": {"module":"someothermodule"}})
# Sorting users by its username DESC
# omitting `filter` returns all records
db.users.find(sort=[('username', -1)])
# Pagination of the results
# Getting the first 20 records
db.users.find(sort=[('username', -1)], skip=0, limit=20)
# Getting next 20 records
db.users.find(sort=[('username', -1)], skip=20, limit=20)
# Getting the total of records
db.users.count()HINT: Learn more about TinyDB storages and Serializers in documentation
You have to subclass TinyMongoClient and provide custom storages like
CachingMiddleware or other available TinyDB Extension.
from tinymongo import TinyMongoClient
from tinydb.storages import JSONStorage
from tinydb.middlewares import CachingMiddleware
class CachedClient(TinyMongoClient):
"""This client has cache"""
@property
def _storage(self):
return CachingMiddleware(JSONStorage)
connection = CachedClient('/path/to/folder')HINT: You can nest middlewares:
FirstMiddleware(SecondMiddleware(JSONStorage))
To convert your data to a format that is writable to disk TinyDB uses the Python JSON module by default. It's great when only simple data types are involved but it cannot handle more complex data types like custom classes.
To support serialization of complex types you can write
your own serializers using the tinydb-serialization extension.
First install it with pip install "tinymongo[serialization]".
datetime, binary values, and optional ObjectId values are handled directly
by TinyMongo; they do not need a custom serializer. For application-specific
classes, subclass TinyMongoClient and provide a tinydb-serialization
middleware as described in the TinyDB extension documentation. The legacy
tinymongo.serializers.DateTimeSerializer remains available for existing
custom-storage integrations, but new TinyMongo storage does not require it.
This extension can work with Flask-Admin which gives a web based administrative panel to your TinyDB. Flask-Admin has features like filtering, search, web forms to perform CRUD (Create, Read, Update, Delete) of the TinyDB records.
You can find the example of Flask-Admin with TinyMongo in Flask-Admin Examples Repository
Datetime fields work with TinyMongo's built-in storage codec.
Contributions are welcome! Currently, the most valuable contributions would be:
- adding test cases
- adding functionality consistent with PyMongo
- improving documentation
- identifying bugs and issues
Planned compatibility, aggregation, GridFS, wire-server, Compass, and browser work is tracked in the TinyMongo roadmap.
MIT License
