Snapshot Isolation: ColumnShard #12972
Description
Description (image of the result)
Customers can use Snapshot Isolated transactions for Column Tables.
The following use cases are ready for using:
- Data marts
- tx1 reads a lot from table1, calculates a lot, and writes a bit to a table2
- tx2 concurrently writes a lot to table1 intersecting with what tx1 reads
- Copy table using sql operators
- tx1 reads all the table1, and writes all of the data to table2
- tx2 concurrently writes to table1
DoD
The changes are merged into stable-???-??? and available in cloud.
Changelog entry
Snapshot isolated transactions are available in Column Tables.
Documentation
https://ydb.tech/docs/ru/concepts/transactions?version=main#modes
Dependencies
- KQP team. Supported. Some fixes are in the process... Snapshot Isolation write only optimization error #29454
- SDK Support snapshot-rw isolation mode in query client in Python SDK ydb-python-sdk#727
- YDB CLI Support snapshot-rw isolation mode in ydb cli #29532
- UI Support Snapshot Read-Write transaction mode for queries in UI, EM #29533
- Data Shards team. Not supported. Plans ???
Compatibility
Rolling update no-snapshot-isolation-version -> snapshot-isolation-version. No problems expected. Snapshot isolated transaction on no-snapshot-isolation-version will behave as serializable (default). So, no data loses or unexpected anomalies.
Rolling update snapshot-isolation-version -> no-snapshot-isolation-version. No problems expected. See explanation above.
Compatibility тесты не требуются
Test plan
- these tests for Column Shards
- jepsen тесты https://github.com/ydb-platform/jepsen.ydb, need to support Snapshot Isolation in jepsen.
- 100 of 2 minutes runs
- 1 of 24 hours run
- 1 of 24 hours run with nemesis
Stress tests
- jepsen тесты https://github.com/ydb-platform/jepsen.ydb, need to support Snapshot Isolation in jepsen.
Implementation Plan
Milestone 1. the relaxed version is supported - no read conflicts
Deadline 24.11.2025 ✅ in stable-25-3
Milestone 2. the classic version - no read conflicts, yes write conflicts
Deadline 01.12.2025 ✅ in stable-25-3
Milestone 3. the classic sped up version, do not read data for writes, only detects conflicts
Deadline 08.12.2025 ✅ in stable-25-3
Milestone 4 jepsen passes
Deadline ??? - depends on #28452
Milestone 5 support long transactions
Deadline ??? - depends on the previous milestone
Feature flag
table_service_config. EnableSnapshotIsolationRW
Promotion
Promotion makes sense because the lack of snapshot isolation was a deal breaker for some potential customers.
- explicitly notify the customers that need it
- ???
- post on Atushka
Implementation details
Snapshot Isolation transactions use LockMode = OPTIMISTIC_SNAPSHOT_ISOLATION (https://github.com/ydb-platform/ydb/blame/257ac8b881d2e2485934f6b5347c9d57958232f5/ydb/core/protos/data_events.proto#L69) in EvWrite/EvRead/DataTransaction/Scan.
Writes:
- Locks are taken from the MVCCSnapshot timestamp (not the current timestamp like for serializable isolation).
Reads:
- No locks are taken for reads. LockId still exists here to support uncommitted changes read.
These tests will pass after implementing Snapshot Isolation for shards.
ydb/.github/config/muted_ya.txt
Line 53 in a3f859c