feat(python): expose spillway controls (open kwargs + runtime setters)

python/README.md

@@ -189,11 +189,13 @@ print(after.pages_allocated - before.pages_allocated)
 
 ```python
 chisel.open(
-    path,                      # str, os.PathLike, or None for in-memory
-    cache_max_bytes=8_388_608, # bytes; default 8 MiB (= 1024 × 8 KiB pages)
+    path,                                        # str, os.PathLike, or None for in-memory
+    cache_max_bytes=8_388_608,                   # bytes; default 8 MiB (= 1024 × 8 KiB pages)
+    spillway_max_bytes=None,                     # None → 1024 × cache_max_bytes (8 GiB default); 0 disables
+    drain_insertion=chisel.DrainInsertion.LruTail,
     create_if_missing=True,
     read_only=False,
-    superblock_count=2,        # 2..=16, only consulted on create
+    superblock_count=2,                          # 2..=16, only consulted on create
 )
 ```
 
@@ -203,6 +205,38 @@ chisel.open(
 
 `chisel.open(None)` produces an in-memory database (same engine, `Vec<u8>`-backed I/O, no file, no lock, lost on close).
 
+### Cache and spillway
+
+The page cache is bounded strictly by `cache_max_bytes`. When the cache is full of dirty pages (nothing evictable), overflow spills into a sidecar file `<path>.spillway` rather than failing. The spillway is bounded by `spillway_max_bytes` and is truncated at every commit and rollback. It is never `fsync`ed — its contents are uncommitted by definition, so a crash with a non-empty spillway is recovered by discarding it.
+
+- `spillway_max_bytes=None` (default) scales the spillway cap to `1024 × cache_max_bytes` — 8 GiB at the 8 MiB cache default. This matches the Rust API's default and is the recommended setting for normal workloads.
+- `spillway_max_bytes=0` disables the spillway entirely and restores `CacheFullError`-at-cap semantics. Useful for tests that want deterministic "cache is full" failures, or for memory-constrained deployments where you'd rather error than spill to disk.
+- Any positive integer caps the spillway in bytes. When both the cache and the spillway are exhausted, the engine raises `SpillwayFullError`; commit or roll back to drain.
+
+`drain_insertion` controls where rehydrated spillway pages re-enter the cache when the spillway drains at commit:
+
+- `chisel.DrainInsertion.LruTail` (default) places them at the LRU end, so they're the next pages evicted under normal use. Right for workloads that don't re-read pages they just wrote.
+- `chisel.DrainInsertion.Mru` places them at the MRU end. Right for workloads that re-read recently-written pages — the rehydrated cache entries stay warm.
+
+In-memory mode (`chisel.open(None)`) uses an in-memory spillway buffer instead of a sidecar file; everything else behaves the same.
+
+### Runtime configuration
+
+All three options above can also be changed *between* transactions on a live `Chisel` handle:
+
+```python
+with chisel.open("db.chisel") as db:
+    db.set_cache_max_bytes(4 * 1024 * 1024)              # halve to 4 MiB
+    db.set_spillway_max_bytes(0)                          # disable spillway
+    db.set_drain_insertion(chisel.DrainInsertion.Mru)
+    with db.transaction() as tx:
+        ...  # uses the new config
+```
+
+Each setter operates ONLY on between-transactions state: calling any of them while a transaction is active raises `TransactionInProgressError`. The engine guards this because shrinking the cache or spillway mid-transaction would either need to reject pinned dirty pages or silently overflow them, neither of which is a clean story — commit or roll back first.
+
+The setters take effect immediately after they return. A subsequent `db.transaction()` uses the new caps and policy; the previous transaction (already committed or rolled back) was unaffected.
+
 ## Errors
 
 All Chisel errors inherit from `chisel.ChiselError`, which splits into two tiers.
@@ -216,16 +250,16 @@ Catch and continue.
 | `InvalidHandleError` | Unknown or deleted handle passed to `read` / `update` / `delete` |
 | `NoActiveTransactionError` | Mutation attempted outside a transaction |
 | `TransactionAlreadyActiveError` | `begin()` called while one is already running |
-| `TransactionInProgressError` | Configuration mutator called while a transaction is active (reserved for future use; not raised by the v1 binding because the runtime config setters are not exposed) |
+| `TransactionInProgressError` | `set_cache_max_bytes` / `set_spillway_max_bytes` / `set_drain_insertion` called while a transaction is active; commit or roll back first |
 | `SavepointNotFoundError` | `rollback_to` / `release` on an unknown savepoint name |
 | `DuplicateSavepointError` | `savepoint(name)` reused an active name |
 | `ReadOnlyModeError` | Write attempted on a read-only handle |
 | `DatabaseFileNotFoundError` | `create_if_missing=False` and file absent |
 | `InvalidRootNameError` | Named-root name is empty, too long, or not valid UTF-8 |
 | `RootNameTableFullError` | All named-root slots are in use |
 | `InvalidSuperblockCountError` | `superblock_count` outside `2..=16` |
-| `CacheFullError` | Page cache hit its strict `cache_max_bytes` cap with every cached page dirty (no clean page available for eviction); commit or rollback to drain. The Python binding currently disables the spillway sidecar, so this is the only buffer-full path you'll see. |
-| `SpillwayFullError` | Spillway sidecar's `spillway_max_bytes` cap was reached (reserved for future use; not raised by the v1 binding because the spillway is disabled — `spillway_max_bytes` is hard-coded to 0) |
+| `CacheFullError` | Page cache hit its strict `cache_max_bytes` cap with every cached page dirty (no clean page available for eviction) AND the spillway is disabled (`spillway_max_bytes=0`); commit or roll back to drain. When the spillway is enabled (default), the cache overflows into it instead and you'll see `SpillwayFullError` only if the spillway also fills. |
+| `SpillwayFullError` | Spillway sidecar's `spillway_max_bytes` cap was reached during a transaction; commit or roll back to drain the spillway. Database is intact. |
 | `ClosedError` | Call through a `Transaction` / `Savepoint` after `db.close()` |
 | `AlreadyFinishedError` | Second explicit drive on a transaction or savepoint |
 

python/chisel/__init__.py

@@ -5,6 +5,7 @@
     Chisel,
     Transaction,
     Savepoint,
+    DrainInsertion,
     open,
     ChiselError,
     OperationalError,
@@ -103,7 +104,7 @@ class DefragStats:
 
 __all__ = [
     "__version__",
-    "Chisel", "Transaction", "Savepoint", "open",
+    "Chisel", "Transaction", "Savepoint", "DrainInsertion", "open",
     "Stats", "Counters", "DefragOptions", "DefragStats",
     "ChiselError", "OperationalError", "FatalError",
     "InvalidHandleError", "NoActiveTransactionError",

python/chisel/chisel.pyi

@@ -88,10 +88,22 @@ class DefragStats:
     values_moved: int = 0
 
 
+class DrainInsertion:
+    # PyO3 pyclass enum mirroring chisel::DrainInsertion. Users reach for
+    # `chisel.DrainInsertion.LruTail` / `chisel.DrainInsertion.Mru`; the
+    # class-level attributes ARE the singleton instances. Variant names
+    # match the Rust spelling so cross-referencing ARCHITECTURE.md
+    # ADR-5 stays mechanical.
+    LruTail: DrainInsertion
+    Mru: DrainInsertion
+
+
 def open(
     path: str | os.PathLike[str] | None = None,
     *,
     cache_max_bytes: int = 8_388_608,
+    spillway_max_bytes: int | None = None,
+    drain_insertion: DrainInsertion = ...,
     create_if_missing: bool = True,
     read_only: bool = False,
     superblock_count: int = 2,
@@ -131,6 +143,12 @@ class Chisel:
 
     def defrag(self, options: DefragOptions | None = None) -> DefragStats: ...
 
+    # Between-transaction config mutators. Each raises
+    # TransactionInProgressError if called mid-transaction.
+    def set_cache_max_bytes(self, bytes: int) -> None: ...
+    def set_spillway_max_bytes(self, bytes: int) -> None: ...
+    def set_drain_insertion(self, policy: DrainInsertion) -> None: ...
+
 
 class Transaction:
     def __enter__(self) -> Self: ...

python/src/db.rs

@@ -56,6 +56,33 @@ pub struct PyChisel {
     inner: RefCell<Option<Chisel>>,
 }
 
+// `DrainInsertion` mirrors `chisel::DrainInsertion` one-to-one. We use a
+// dedicated PyO3 pyclass enum rather than re-using the Rust enum so the
+// Python type is owned by this binding (its module path, repr, and any
+// future Python-side methods are ours to evolve) and so we don't depend
+// on the Rust enum implementing the PyO3 conversion traits.
+//
+// Variant names match the Rust spelling (`LruTail`, `Mru`) rather than
+// PEP-8 `LRU_TAIL` / `MRU` so cross-referencing ARCHITECTURE.md ADR-5
+// and the Rust source stays mechanical. `eq` / `eq_int` give Python users
+// `chisel.DrainInsertion.LruTail == chisel.DrainInsertion.LruTail` and
+// efficient int-based comparison under the hood.
+#[pyclass(eq, eq_int, module = "chisel._chisel", name = "DrainInsertion")]
+#[derive(Clone, Copy, PartialEq)]
+pub enum PyDrainInsertion {
+    LruTail,
+    Mru,
+}
+
+impl From<PyDrainInsertion> for chisel::DrainInsertion {
+    fn from(v: PyDrainInsertion) -> Self {
+        match v {
+            PyDrainInsertion::LruTail => chisel::DrainInsertion::LruTail,
+            PyDrainInsertion::Mru => chisel::DrainInsertion::Mru,
+        }
+    }
+}
+
 // Keyword-only args after `path` (note the `*`) so users can't
 // positionally supply cache_max_bytes by accident; the defaults mirror
 // `chisel::Options::default()` so `open(None)` and `open("f.db")`
@@ -65,19 +92,37 @@ pub struct PyChisel {
 // sides hard-code them rather than sharing a constant because the Rust
 // side's constants are not exposed through the PyO3 signature syntax.
 // 8_388_608 = 8 MiB = 1024 × 8 KiB pages, matching Options::default().
+//
+// `spillway_max_bytes = None` is the sentinel for "use Rust's computed
+// default of 1024 × cache_max_bytes" (8 GiB at the 8 MiB cache default).
+// Explicit 0 disables the spillway and restores CacheFull-at-cap
+// semantics; any positive integer is the cap in bytes. We resolve None
+// at runtime rather than via the signature literal because the literal
+// would not scale with a user-overridden cache_max_bytes.
 #[pyfunction]
 #[pyo3(signature = (
     path = None,
     *,
     cache_max_bytes = 8_388_608,
+    spillway_max_bytes = None,
+    drain_insertion = PyDrainInsertion::LruTail,
     create_if_missing = true,
     read_only = false,
     superblock_count = 2
 ))]
+// open() has 8 args; clippy warns at 7. All but `path` are keyword-only
+// (note the `*` in the pyo3(signature) above), so the user can never
+// pass them positionally and confuse argument order. Each arg maps to
+// a distinct field of chisel::Options that we want to expose; bundling
+// them into a struct would require Python users to construct one,
+// which is less ergonomic than kwargs.
+#[allow(clippy::too_many_arguments)]
 pub fn open(
     py: Python<'_>,
     path: Option<PyObject>,
     cache_max_bytes: u64,
+    spillway_max_bytes: Option<u64>,
+    drain_insertion: PyDrainInsertion,
     create_if_missing: bool,
     read_only: bool,
     superblock_count: u32,
@@ -103,13 +148,16 @@ pub fn open(
         }
     };
 
-    // Spillway is disabled for v1 of the Python binding (spillway_max_bytes=0);
-    // exposing spillway controls via Python is deferred until the Rust side
-    // has shipped and stabilized. drain_insertion uses the Rust default.
+    // Resolve the spillway cap. None → Rust's 1024 × cache_max_bytes
+    // default (8 GiB at the 8 MiB cache default); explicit 0 disables
+    // and falls back to CacheFull-at-cap. The 1024 multiplier matches
+    // chisel::Options::default() exactly.
+    let resolved_spillway_max_bytes = spillway_max_bytes.unwrap_or(1024 * cache_max_bytes);
+
     let options = chisel::Options {
         cache_max_bytes,
-        spillway_max_bytes: 0,
-        drain_insertion: chisel::DrainInsertion::LruTail,
+        spillway_max_bytes: resolved_spillway_max_bytes,
+        drain_insertion: drain_insertion.into(),
         create_if_missing,
         read_only,
         superblock_count,
@@ -301,6 +349,31 @@ impl PyChisel {
         kwargs.set_item("values_moved", stats.values_moved)?;
         Ok(cls.call((), Some(&kwargs))?.unbind())
     }
+
+    // ── Between-transaction configuration mutators ──────────────────
+    //
+    // The three setters below mirror the same-named methods on
+    // chisel::Chisel. They operate ONLY on between-transactions state:
+    // calling any of them while a transaction is active raises
+    // TransactionInProgressError, because shrinking the cache or
+    // spillway mid-transaction would either reject pinned dirty pages
+    // or silently spill them — neither is a clean story (see
+    // ARCHITECTURE.md ADR-5 + ISSUES.md for the spillway design).
+    //
+    // All three flow through with_inner_mut_io, so a closed handle
+    // raises ClosedError uniformly with every other mutating method.
+
+    fn set_cache_max_bytes(&self, py: Python<'_>, bytes: u64) -> PyResult<()> {
+        self.with_inner_mut_io(py, |c| c.set_cache_max_bytes(bytes))
+    }
+
+    fn set_spillway_max_bytes(&self, py: Python<'_>, bytes: u64) -> PyResult<()> {
+        self.with_inner_mut_io(py, |c| c.set_spillway_max_bytes(bytes))
+    }
+
+    fn set_drain_insertion(&self, py: Python<'_>, policy: PyDrainInsertion) -> PyResult<()> {
+        self.with_inner_mut_io(py, |c| c.set_drain_insertion(policy.into()))
+    }
 }
 
 // Internal helpers — NOT exposed to Python. PyTransaction reaches into
@@ -445,6 +518,7 @@ fn closed_err() -> PyErr {
 
 pub fn register(m: &Bound<'_, PyModule>) -> PyResult<()> {
     m.add_class::<PyChisel>()?;
+    m.add_class::<PyDrainInsertion>()?;
     m.add_function(wrap_pyfunction!(open, m)?)?;
     Ok(())
 }

python/tests/test_open.py

@@ -49,3 +49,69 @@ def test_double_open_same_path_fails(tmp_db):
     with chisel.open(str(tmp_db)):
         with pytest.raises(chisel.LockFailedError):
             chisel.open(str(tmp_db))
+
+
+# ── Spillway and drain_insertion kwargs ─────────────────────────────
+
+
+def test_drain_insertion_has_both_variants():
+    # Smoke check that the pyclass enum is exposed and both variants
+    # are accessible as class attributes.
+    assert hasattr(chisel, "DrainInsertion")
+    assert hasattr(chisel.DrainInsertion, "LruTail")
+    assert hasattr(chisel.DrainInsertion, "Mru")
+
+
+def test_drain_insertion_variants_compare_equal_to_themselves():
+    # The pyclass enum was declared with `eq, eq_int` so identity AND
+    # equality must hold. Catches accidental future regressions where
+    # a variant becomes a wrapper that breaks equality.
+    assert chisel.DrainInsertion.LruTail == chisel.DrainInsertion.LruTail
+    assert chisel.DrainInsertion.Mru == chisel.DrainInsertion.Mru
+    assert chisel.DrainInsertion.LruTail != chisel.DrainInsertion.Mru
+
+
+def test_open_with_default_spillway(tmp_db):
+    # spillway_max_bytes=None (default) resolves to 1024 × cache_max_bytes
+    # inside the Rust binding. Just verifying the open path accepts the
+    # default and succeeds.
+    with chisel.open(str(tmp_db)) as db:
+        assert db is not None
+
+
+def test_open_with_spillway_disabled(tmp_db):
+    # Explicit 0 disables the spillway and restores CacheFull-at-cap.
+    with chisel.open(str(tmp_db), spillway_max_bytes=0) as db:
+        assert db is not None
+
+
+def test_open_with_custom_spillway_size(tmp_db):
+    # A small explicit cap. Engine should accept any non-negative integer.
+    with chisel.open(str(tmp_db), spillway_max_bytes=1024 * 1024) as db:
+        assert db is not None
+
+
+def test_open_with_drain_insertion_lru_tail(tmp_db):
+    # Default value explicit-passed. Mainly verifies that a
+    # DrainInsertion variant survives the round-trip through pyo3 into
+    # chisel::DrainInsertion without panicking or rejecting.
+    with chisel.open(str(tmp_db), drain_insertion=chisel.DrainInsertion.LruTail) as db:
+        assert db is not None
+
+
+def test_open_with_drain_insertion_mru(tmp_db):
+    # The non-default variant. Reaching here means the From<PyDrainInsertion>
+    # impl is wired for both arms.
+    with chisel.open(str(tmp_db), drain_insertion=chisel.DrainInsertion.Mru) as db:
+        assert db is not None
+
+
+def test_open_in_memory_with_spillway_and_drain(tmp_db):
+    # In-memory mode uses an in-memory spillway buffer instead of a
+    # sidecar file; same kwargs should still be accepted.
+    with chisel.open(
+        None,
+        spillway_max_bytes=1024 * 1024,
+        drain_insertion=chisel.DrainInsertion.Mru,
+    ) as db:
+        assert db is not None