fix(ci): run the real Python test suite (3 tests → 182)

[beinan]

The problem

python-test.yml ran pytest python/tests/ with working-directory: python, which resolves to python/python/tests/ — a single stale file with 3 tests. The real suite, python/tests/ (24 files, 179 tests), was never collected. pytest exits 0 on a valid-but-nearly-empty directory, so this failed silently.

Git history dates the split: python/python/tests/ last changed in #59; python/tests/ is current through #158. ~100 PRs merged ungated — rollout store, async wrappers, export, ingestion.

Two further layers of silencing, each sufficient on its own:

• Only bare pytest was installed, never the [tests] extra — so pytest-asyncio was absent.
• No asyncio_mode anywhere, so the 12 async def tests were collected, never awaited, and passed without executing a line.

The same path bug appeared three more times: the doctest step was a permanent no-op (an if on a path that never exists, plus || echo), and ruff format --check python/ / ruff check python/ never linted the 24 test files. .codex/.../run_ci_checks.sh pointed at a rust/ directory that does not exist, so set -e killed it before any Python check ran.

CI changes

• Pin collection with testpaths = ["tests"] so the ambiguous relative path cannot resurface.
• asyncio_mode = "strict" + --strict-markers --strict-config — a mistyped marker now fails instead of silently disarming a test.
• Install [lance-python,tests]; lint . instead of python/; make the doctest step real.
• Point the helper script at the real crates/ layout.

Turning the suite on surfaced 19 real failures

1. Embedded RolloutStore silently discarded every write. This is the significant one. #181 made add durable-but-not-visible, with visibility driven by a server-side sweeper. The embedded path has no sweeper, and flush() was never exposed to Python — so:

s = RolloutStore.open(d)
s.add({'id':'a','rollout_id':'t1'})   # -> {'version': 2, 'ids': ['a'], 'count': 1}
s.list()                              # -> []   ... and still [] after reopen

Data acknowledged, then unreadable forever. This PR exposes flush() through the facade, PyO3, and both sync and async Python wrappers.

2. test_search.py's DummyInner fake had drifted from the real binding signature (missing include_binary/include_embedding), so all 55 tests raised TypeError on contact with current code.

3. Remote rollout tests asserted read-your-write against a server whose default flush interval is 30s. They now set ROLLOUT_FLUSH_INTERVAL_SECS=1 and poll — asserting the row arrives rather than pinning to the interval.

The 3 stale tests are moved into python/tests/ rather than dropped (they cover snapshot/fork/memory:// and still pass), and the orphaned directory is removed so nothing can silently collect it again.

Verification

• 182 passed, 2 skipped (179 real + 3 rescued).
• ruff format --check ., ruff check ., pyright — all clean.
• cargo fmt --all -- --check and clippy clean on the changed crates.
• Confirmed the async tests now genuinely execute by injecting a failing assertion and watching it fail — it would not have before this change.

Note on scope

This is deliberately scoped to making CI honest plus the failures that gating surfaced. It does not address: publish workflows having no needs: on any test job (a release cut from a red main ships to PyPI/crates.io), cargo test covering only 2 of 8 crates and --lib only, or the DummyInner fake asserting hardcoded literals so RRF ranking is still never exercised. Happy to file those separately.

🤖 Generated with Claude Code

[beinan]

CI surfaced a real bug: MemWAL drops storage_options on custom-endpoint S3

The first CI run came back 190 passed, 2 failed — more tests than my local run, because CI installs the [tests] extra and therefore actually runs the moto-backed S3 tests. Both failures are genuine, pre-existing, and not caused by this PR.

Symptom

Context.create against a custom-endpoint S3 succeeds; the first add fails with Bucket '...' not found.

Diagnosis

Reproduced locally against moto with request logging. Every request that reaches moto returns 200, and the failing request never arrives at all — so the WAL writer is talking to a different endpoint than the dataset.

Root cause is upstream in lance 7.0.0:

// lance/src/dataset/mem_wal/api.rs:607
let (store, base_path) = ObjectStore::from_uri(base_uri).await?;

mem_wal_writer builds a fresh object store from the URI alone, discarding the dataset's configured storage_options. aws_endpoint_url is dropped, so the writer targets real AWS. create works because it goes through the options-aware path; add is the first call that opens a shard writer.

No local workaround exists — ShardWriterConfig has no field to carry storage options. The fix is upstream: use from_uri_and_params, which already exists in lance-io.

Blast radius beyond the tests

All three stores open shard writers this way — store.rs:518, rollout_store.rs:632, datagen_store.rs:186. Writes are broken against any custom-endpoint S3: MinIO, Ceph, R2, moto. Only AWS-proper works today. This was invisible precisely because the tests covering it were never executed.

What I did

Marked both xfail(strict=True) — not skipped. They keep running, and flip to a hard failure the moment a lance bump fixes this, so the marker cannot silently rot. The reason string carries the full diagnosis and the upstream file:line.

Full suite with all extras: 194 passed, 2 skipped, 3 xfailed.

This seems worth an upstream issue against lance; happy to file it if you agree with the read.