Skip to content

v3.0.0

Choose a tag to compare

@rroblf01 rroblf01 released this 03 May 18:17
· 47 commits to main since this release

Major release. Combines three originally-planned minors (2.6 / 2.7 / 2.8)
plus the Django-parity sprint that was going to ship as 3.1. No
breaking changes vs 2.5
: every addition is opt-in or zero-cost
when unused.

The Django-parity additions close the last πŸ”΄ gaps that would have
blocked a Django→dorm migration: timezone-aware datetimes,
proxy models, dates() / datetimes(), migrate --fake,
JSONField PG operators, Field.deconstruct(), Model.from_db(),
manual savepoint API, Meta.permissions, password-reset tokens,
multi-tenant context manager, and the MySQL / MariaDB scaffold +
vector support.

Django-parity additions (was going to ship as 3.1)

Added β€” settings.USE_TZ actually wired up

  • DateTimeField reads settings.USE_TZ (default False).
    When True, naive datetimes are interpreted in
    settings.TIME_ZONE and converted to UTC for storage. PostgreSQL
    columns become TIMESTAMP WITH TIME ZONE so the engine
    preserves the offset; SQLite stores UTC-isoformat.
  • from_db_value always returns tz-aware datetimes when
    USE_TZ is on, normalised to UTC. Cross-row comparisons stay
    honest regardless of the writer's local zone.
  • settings.TIME_ZONE resolves through zoneinfo.ZoneInfo;
    unrecognised values fall back to UTC silently.

Added β€” Meta.proxy = True

  • Proxy models share their concrete parent's table (no migration,
    no duplicate columns) and inherit field instances directly so a
    proxy's _meta.fields points at the same descriptors. The
    autodetector skips them; only the concrete parent emits a
    CreateModel.
  • Options.concrete_model resolves to the closest non-proxy
    ancestor whose table backs the rows β€” useful for libs that need
    to know "where do this instance's bytes live".

Added β€” QuerySet.dates() / QuerySet.datetimes()

  • Returns list[date] / list[datetime] of distinct
    truncated values for field. Truncation happens in Python (the
    queryset compiler's annotation+distinct combo isn't ready for the
    SQL-level shape yet); chainable variants land alongside the
    compiler refactor in v4.
  • dates() accepts "day", "week", "month",
    "year". datetimes() adds "hour" / "minute" /
    "second". Order argument respects "ASC" / "DESC".

Added β€” dorm migrate --fake / --fake-initial

  • --fake marks every pending migration as applied without
    running its operations. Use when adopting dorm against a hand-
    managed legacy database β€” the schema already matches and you
    just want the recorder synced.
  • --fake-initial only fakes each app's initial migration
    (no dependencies), and only when every CreateModel it
    contains targets a table that already exists. Subsequent
    migrations run for real. Mirrors Django's flag of the same
    name.
  • MigrationExecutor.migrate(... fake=, fake_initial=) exposes
    the same shape programmatically.

Added β€” JSONField / ArrayField PG operators

  • New lookup names matching Django's contrib.postgres spellings:
    __contained_by (<@), __has_key (?),
    __has_keys (?&), __has_any_keys (?|),
    __overlap (&&), __len (array_length(col, 1)).
    Existing __array_contains / __array_overlap /
    __json_has_* aliases stay for users who already wrote them.

Added β€” Field.deconstruct()

  • Default base-class implementation returns
    (name, dotted_path, args, kwargs) and emits only kwargs that
    differ from the framework defaults. Custom field classes get
    serialisation in the migration writer for free; subclasses with
    extra constructor parameters override and call super().

Added β€” Model.from_db(db, field_names, values)

  • Django-parity classmethod for custom hydration logic. Default
    implementation zips the parallel lists and stamps the resulting
    _state.db with the alias the row came from. Libs that hook
    through it (history-tracking, soft-delete extensions) keep
    working when migrated from Django.

Added β€” dorm.contrib.auth.tokens

  • Stateless HMAC-signed reset tokens, framework-agnostic. The
    signature binds to the user's last_login / password /
    email so a single use of the token (which changes the
    password) invalidates every outstanding URL.
  • PasswordResetTokenGenerator with configurable
    timeout (default 24h) and salt_namespace (domain
    separation between password-reset / email-verification / etc.).
  • Stateful alternative generate_short_lived_token for cases
    where the caller needs a one-row revoke list.

Added β€” Meta.permissions + dorm.contrib.auth.management.sync_permissions

  • Meta.permissions = [(codename, name), ...] declares custom
    per-model permissions. Default verbs (add, change,
    delete, view) get auto-emitted per concrete model.
  • sync_permissions() materialises the declarations into rows
    in the auth_permission table. Idempotent β€” safe to call
    every deploy.

Added β€” transaction.savepoint / savepoint_commit / savepoint_rollback

  • Manual savepoint API for branching / rolling back inside a
    single atomic() block without unwinding it. Returns a
    random savepoint ID (s_<hex>) that's safe to splice into
    SQL; callers that pass arbitrary strings get ValueError to
    prevent SQL injection through the savepoint name.

Added β€” MySQL backend scaffold

  • ENGINE = "mysql" parses through parse_database_url and
    the connection wrappers raise ImproperlyConfigured pointing
    at the v3.2 milestone. Lets users pin on a forward-compatible
    config string today.

Added β€” Multi-tenant dorm.contrib.tenants

  • TenantContext(name) / aTenantContext(name) set
    PostgreSQL's search_path to <schema>, public for the
    duration of a context. Per-task isolation via ContextVar β€”
    concurrent FastAPI / Starlette requests routed to different
    tenants don't bleed.
  • current_tenant() lookup for DATABASE_ROUTERS /
    middleware. register_tenant(name) for the future migration
    runner. Schema names validated against the same identifier
    regex used elsewhere β€” no SQL injection through tenant names.

Tests

  • tests/test_v3_1_use_tz.py β€” datetime round-trip with
    USE_TZ on/off, naive→aware conversion, PG TIMESTAMPTZ
    selection.
  • tests/test_v3_1_proxy.py β€” proxy inheritance shares table,
    proxy skipped from migration state, concrete_model wiring.
  • tests/test_v3_1_dates.py β€” round-trip distinct truncation
    for every kind, order ASC/DESC, kind / order validation.
  • tests/test_v3_1_migrations_fake.py β€” fake flag records
    without running, fake_initial only fires when table exists.
  • tests/test_v3_1_jsonfield.py β€” new lookup names compile to
    the right PG operator.
  • tests/test_v3_1_deconstruct.py β€” base Field.deconstruct
    returns sane shape and round-trips through cls(**kwargs).
  • tests/test_v3_1_from_db.py β€” alias stamped on _state.db,
    hydration matches column / attname mapping.
  • tests/test_v3_1_auth_tokens.py β€” make/check round-trip,
    password-change invalidation, timeout enforcement, namespace
    separation, malformed input handling.
  • tests/test_v3_1_permissions.py β€” default permissions
    per concrete model, custom Meta entries, idempotence,
    abstract / proxy skip.
  • tests/test_v3_1_savepoint.py β€” savepoint commit / rollback
    round-trip, invalid id rejection.
  • tests/test_v3_1_tenants.py β€” schema validation, context
    state restoration, non-PG backend rejection.

Added β€” settings.SLOW_QUERY_MS

  • New setting (default 500.0 ms). When an executed statement
    takes longer than this threshold, the
    dorm.db.backends.<vendor> logger emits a WARNING line
    with the SQL text and elapsed time. Resolution order: explicit
    configure(SLOW_QUERY_MS=…) > env var
    DORM_SLOW_QUERY_MS > default 500.0.
  • SLOW_QUERY_MS=None disables the warning entirely β€” the
    threshold comparison itself is skipped, so on-path cost
    collapses to the timing already collected for the query
    signals.
  • SLOW_QUERY_MS=0 logs every query as slow β€” useful in
    development to surface every SQL statement at WARNING level
    without flipping the dorm.queries DEBUG stream on.
  • The threshold is memoised once after configure(...);
    subsequent configure(SLOW_QUERY_MS=…) calls invalidate
    the memo so a runtime swap takes effect on the next query.
    The env-var / default branch is intentionally not memoised so
    test monkeypatch.setenv workflows keep observing the
    current value without an explicit cache flush.

Added β€” settings.RETRY_ATTEMPTS / settings.RETRY_BACKOFF

  • Same resolver shape as SLOW_QUERY_MS: explicit setting >
    env var (DORM_RETRY_ATTEMPTS / DORM_RETRY_BACKOFF) >
    default (3 attempts / 0.1 s). The retry loop in
    with_transient_retry / awith_transient_retry now
    reads the resolved value on every call so a runtime
    configure(...) swap takes effect without a restart.
  • Settings-derived values are memoised; env-var path re-reads
    each call so test monkeypatch.setenv keeps working.

Added β€” query-count guard (dorm.contrib.querycount)

  • New query_count_guard(warn_above=…, label=…) context
    manager. Counts queries inside the block via pre_query
    and emits a single WARNING when the count crosses the
    threshold. warn_above falls back to
    settings.QUERY_COUNT_WARN (default None, inert).
  • Per-task isolation via ContextVar β€” concurrent ASGI
    requests / asyncio tasks get independent counters.

Added β€” dorm.test.assertNumQueries / assertMaxQueries

  • Django-parity helpers. Context-manager forms assert exact
    / ≀-N query counts on exit; decorator forms
    (assertNumQueriesFactory(N), assertMaxQueriesFactory(N))
    wrap a sync or async def test function. Both use the same
    ContextVar-based listener as the query-count guard so
    async tests don't bleed counters across tasks.

Added β€” sticky read-after-write window

  • settings.READ_AFTER_WRITE_WINDOW (default 3.0
    seconds). After a write through router_db_for_write, the
    router pins reads of the same model on the same context to
    the primary alias for the configured window β€” so a request
    that writes and immediately re-reads sees its own change
    instead of a stale replica row. 0 / None disables.
  • Sticky state lives in a ContextVar, so concurrent ASGI
    requests / asyncio tasks see independent windows.
  • New helper dorm.db.connection.clear_read_after_write_window
    for tests / middleware that want a clean window per request.
  • Pass sticky=False through the router hints (e.g.
    Model.objects.using("replica", sticky=False)) to opt out
    of the pin for analytics queries that explicitly want the
    replica.

Added β€” dorm lint-migrations

  • New CLI command + dorm.migrations.lint API. Walks every
    migration in INSTALLED_APPS and emits findings for known
    online-deploy footguns. Exits non-zero on findings β€” wire as
    a CI gate.
  • Rules: DORM-M001 AddField NOT NULL with default
    (full-table backfill), DORM-M002 AlterField (review type
    vs. flag-flip), DORM-M003 AddIndex without
    concurrently=True (PG ACCESS EXCLUSIVE lock),
    DORM-M004 RunPython without reverse_code
    (irreversible).
  • --format=json for machine-parseable output, --rule DORM-M00X to filter, --exit-zero for advisory CI runs.
    Suppress per-file with # noqa: DORM-M00X.

Added β€” dorm migrate --plan

  • Alias for the existing --dry-run flag, mirroring
    Django's command name. Prints the SQL that would be
    executed without touching the database. The migration
    recorder is NOT updated.

Added β€” request-scoped query collector (dorm.contrib.querylog)

  • QueryLog context manager captures every SQL statement
    inside a block: sql / params / alias /
    elapsed_ms / error. summary() groups by SQL
    template (placeholders normalised to ?) and returns
    list[TemplateStats] with count / total / p50 / p95
    timings.
  • QueryLogASGIMiddleware wraps any ASGI app; the per-request
    log lands on scope["dorm_querylog"] for downstream
    handlers / middlewares to inspect.

Added β€” LocMemCache cache backend (dorm.cache.locmem)

  • Thread-safe OrderedDict-backed LRU. Same
    :class:BaseCache contract as RedisCache β€” sync + async
    helpers, delete_pattern for signal-driven invalidation,
    OPTIONS["maxsize"] cap.
  • Useful for tests, single-process scripts, or as a layer in
    front of Redis. NOT shared across worker processes.

Added β€” single-row cache (Manager.cache_get)

  • Model.objects.cache_get(pk=…, timeout=…, using=…) reads
    through the configured cache before falling through to the
    DB. Uses the per-model invalidation version so a
    post_save on the model bumps both queryset-cache entries
    and row-cache entries in lock-step.
  • cache_get_many(pks=[…]) β€” batch read; misses collapse
    into one WHERE pk IN (...) query and write back to
    cache.
  • Async parity via acache_get / acache_get_many. Cache
    miss / outage falls through silently β€” same try / except
    policy as RedisCache.

Added β€” dorm.contrib.auth

  • User / Group / Permission models β€” framework-agnostic.
    Provides only the data model and password-hashing helpers; login
    views, sessions and middleware are NOT included (that's the web
    framework's job).
  • Password hashing via stdlib hashlib.pbkdf2_hmac with format
    pbkdf2_sha256$<iterations>$<salt>$<hash> β€” same shape Django
    emits, so passwords migrate cleanly between the two ORMs. No
    passlib / bcrypt / argon2 dependency.
  • UserManager.create_user / create_superuser ensure
    passwords land hashed instead of stored in the clear.
  • user.set_password() / check_password() / has_perm()
    with both sync and async (ahas_perm) variants. Group-based
    permission checks walk forward M2M only β€” no reverse-M2M
    traversal that would break across vendors.

Added β€” dorm.contrib.asyncguard

  • enable_async_guard(mode="warn"|"raise"|"raise_first") connects
    to pre_query and walks the call stack β€” sync ORM calls
    inside a running event loop trigger the configured action; async
    ORM calls stay silent. Frame walking distinguishes the two paths
    by looking for async def frames inside the dorm package.
  • SyncCallInAsyncContext inherits from :class:BaseException
    so it bypasses the dispatcher's except Exception and
    surfaces as a 500, not a logged-and-swallowed warning.
  • Recommended for development / test environments β€” disabled by
    default in production.

Added β€” DB function corpus

  • Math: Power, Sqrt, Mod, Sign, Ceil, Floor,
    Log, Ln, Exp, Random.
  • Util: NullIf.
  • String: Trim, LTrim, RTrim.
  • All cross-backend on PG and SQLite β‰₯3.35 (Python 3.11+ ships
    recent enough libsqlite3 by default). MySQL coverage lands with
    the v3.1 backend.

Added β€” Meta.managed = False

  • Models with managed = False are skipped by
    ProjectState.from_apps β€” makemigrations no longer emits
    a CreateModel for tables the user marks as externally
    managed (legacy schema, view, foreign data wrapper).
  • Runtime queries / saves keep working unchanged β€” only the
    migration emission path is affected.

Added β€” async-aware dorm shell

  • The fallback REPL (when IPython is absent) now compiles input
    with PyCF_ALLOW_TOP_LEVEL_AWAIT so
    await Article.objects.aget(pk=1) works directly. --no-async
    reverts to the classic stdlib REPL.

Added β€” benchmark suite skeleton

  • bench/run.py β€” stdlib-only microbenchmark runner.
    Scenarios: create, bulk_create, get, filter_count,
    list_first_n. JSON output suitable for committing under
    bench/results/ and charting later.
  • tests/test_v2_7_bench_smoke.py β€” ensures the runner survives
    one end-to-end sqlite run with the smallest parameters.

Added β€” dorm.contrib.encrypted

  • EncryptedCharField and EncryptedTextField store
    ciphertext on disk and decrypt transparently on read. Backed by
    AES-GCM via the optional cryptography package
    (djanorm[encrypted]).
  • Deterministic mode (default) keeps equality lookup working β€”
    same plaintext produces the same ciphertext via an
    HMAC-derived nonce. Switch to deterministic=False for
    random-nonce indistinguishability when equality lookups aren't
    required.
  • Key rotation: settings.FIELD_ENCRYPTION_KEYS accepts a list
    of keys (newest first); decryption tries each in order so old
    rows keep decrypting after a primary-key roll.
  • Tampered ciphertext is rejected with a clear ValueError
    rather than silently returning None β€” better to surface the
    bug than mask it.
  • New settings: FIELD_ENCRYPTION_KEY (single-key form) and
    FIELD_ENCRYPTION_KEYS (list-form for rotation).

Added β€” dorm.contrib.prometheus

  • Stdlib-only Prometheus text-exposition exporter β€” no
    prometheus_client dependency. Connects to post_query to
    emit dorm_queries_total (counter), dorm_query_duration_seconds
    (histogram), plus optional pool / cache gauges.
  • install() attaches the listener; uninstall() removes it
    and resets state. Idempotent so a FastAPI lifespan can wire it
    without ceremony.
  • record_cache_hit(alias) / record_cache_miss(alias)
    helpers for custom cache backends that want their numbers in the
    exposition.

Changed β€” Settings tracks explicit overrides

  • New private attribute Settings._explicit_settings records
    the names of settings the user passed to configure(...).
    Resolvers can now distinguish a class-level default (apply
    env-var or built-in fallback first) from an explicit user
    choice (always wins). Used by every memoised setting (slow
    query, retry knobs, …) β€” future resolvers register one
    MemoizedSetting instance and the central registry handles
    the rest.

CLI

  • dorm init template now scaffolds (commented-out by default
    except for SLOW_QUERY_MS) every new 3.0 knob:
    RETRY_ATTEMPTS, RETRY_BACKOFF, QUERY_COUNT_WARN,
    READ_AFTER_WRITE_WINDOW, DATABASE_ROUTERS example,
    a CACHES block with both Redis and LocMemCache examples,
    and FIELD_ENCRYPTION_KEY placeholder.
  • dorm lint-migrations registered as a top-level subcommand
    with --format / --rule / --exit-zero / --settings
    flags.
  • dorm migrate --plan accepted as an alias for --dry-run.

Documentation

  • docs/production.{en,es}.md β€” new subsections under
    Observability covering SLOW_QUERY_MS,
    RETRY_ATTEMPTS / RETRY_BACKOFF, QUERY_COUNT_WARN,
    READ_AFTER_WRITE_WINDOW and the request-scoped
    QueryLog collector. New top-level Migration safety
    section documenting dorm lint-migrations rules.
  • docs/cli.{en,es}.md β€” new dorm lint-migrations section
    with --rule / --exit-zero / --format flags.
    dorm migrate flag table now lists --plan as the
    Django-style alias for --dry-run.
  • docs/cache_redis.{en,es}.md β€” new LocMemCache
    configuration block and Manager.cache_get(pk=…) /
    cache_get_many(pks=[…]) row-cache subsection (sync + async).
  • docs/cookbook.{en,es}.md β€” testing fixtures section grew
    an assertNumQueries / assertMaxQueries entry covering
    the context-manager and decorator forms (sync + async).
  • docs/migration-from-django.{en,es}.md β€” new
    You don't need asgiref section with a Django↔dorm
    cheatsheet (every sync_to_async(qs.X) has a native aX
    counterpart) plus call-outs for the optional contrib.auth,
    contrib.encrypted and contrib.prometheus modules.

Internal

  • dorm._memoized_setting.MemoizedSetting β€” central registry
    for the settings β†’ env β†’ default resolver pattern. New
    per-call knobs register one instance and conf.configure's
    invalidation pulse fans out automatically (no per-knob
    if "X" in kwargs: block).
  • dorm._scoped.ScopedCollector β€” shared primitive for
    ContextVar-based per-task signal collectors. Used by
    query_count_guard, assertNumQueries and QueryLog.
  • LocMemCache carries a secondary defaultdict[prefix, set]
    index β€” delete_pattern("ns:*") is now O(matches) instead of
    O(n).
  • Sticky read-after-write window upgraded from copy-on-write to
    lazy-copy-then-mutate: each task takes one private dict on
    first write, then mutates in place for the rest of the request.
  • QueryRecord and TemplateStats are now @dataclass(slots=True).
    QueryLog.summary() returns list[TemplateStats];
    consumers that read dict keys should switch to attribute access
    or call .to_dict().
  • Settings._explicit_settings lives on the instance (set in
    __init__) instead of as a class-level mutable default.
  • Per-query counters in query_count_guard and
    assertNumQueries mutate a single-element list[int] in
    place instead of calling ContextVar.set per query β€” saves
    one Token allocation per signal.

Audit fixes (pre-publish)

  • QueryRecord.alias β†’ QueryRecord.vendor. The post_query
    signal carries sender=vendor, never an alias key, so the
    field had been silently storing the vendor name under a misleading
    label. QueryRecord.to_dict() likewise emits vendor now.
  • Prometheus exposition no longer emits an empty alias=""
    label
    β€” counters group by (vendor, outcome), histograms by
    vendor. If a future release threads the alias through
    log_query the label will come back; until then it isn't
    there to confuse scrapers.
  • ScopedCollector forwards sender to on_event. The
    dispatcher peels sender off as a positional arg before
    **kwargs, so receivers built on ScopedCollector
    (querylog, querycount, assertNumQueries) were getting an
    empty kwargs payload for the vendor name. The collector now
    merges sender back into the kwargs dict it passes to
    on_event.
  • LocMemCache prefix index ignores keys without :. A
    delete_pattern("foo:*") would previously evict a bare
    "foo" key because both shared the same prefix bucket. Keys
    without a namespace separator now sit outside the index and
    only get reached by the slow-path fnmatch scan when a glob
    actually matches them.
  • UserManager.create_user raises RuntimeError (not a
    bare assert) when the manager isn't attached to a model.
    python -O strips assert, so the previous shape would
    have crashed with a confusing AttributeError under
    optimised builds.
  • Settings.FIELD_ENCRYPTION_KEYS lives on the instance (set
    in __init__) instead of as a class-level mutable default.
    Singleton-style usage isn't affected; a future test that builds
    a second Settings() won't inherit a shared list.

Tests

  • tests/test_slow_query_setting_v2_6.py β€” SLOW_QUERY_MS
    resolution order, None-disables behaviour,
    configure-driven cache invalidation, cacheable flag.
  • tests/test_v2_6_features.py + test_v2_6_audit_fixes.py
    • test_v2_6_improvements.py β€” query-count guard,
      assertNumQueries, sticky window, lint rules, querylog,
      LocMemCache, row-cache, audit-fix regression tests, and the
      MemoizedSetting / ScopedCollector helper coverage.
  • tests/test_v2_7_auth.py β€” password-hashing helpers + User /
    Group / Permission round-trip, key reuse, salt entropy,
    superuser-flag enforcement, group-permission resolution.
  • tests/test_v2_7_asyncguard.py β€” guard activation modes,
    warn dedup, sync-context inertness, listener teardown.
  • tests/test_v2_7_functions.py β€” SQL-shape pinning for every
    new function and re-export check against dorm.__all__.
  • tests/test_v2_7_parity.py β€” in_bulk,
    Manager.from_queryset, Prefetch(to_attr=…) and
    Meta.managed = False round-trips.
  • tests/test_v2_7_bench_smoke.py β€” benchmark runner survives
    an end-to-end sqlite run.
  • tests/test_v2_8_encrypted.py β€” round-trip, deterministic /
    random nonce semantics, key rotation, tamper detection,
    missing / invalid key handling, field-level prep / from_db_value
    path. Skipped when cryptography isn't installed.
  • tests/test_v2_8_prometheus.py β€” install idempotence, query
    recording, exposition shape, label escaping, uninstall reset.
  • tests/test_v3_audit_fixes.py β€” pins the audit-fix list above:
    QueryRecord.vendor rename, Prometheus exposition without
    alias="", ScopedCollector sender forwarding,
    LocMemCache prefix-index correctness for keys without
    :, UserManager unbound-manager error path, per-instance
    FIELD_ENCRYPTION_KEYS.