Skip to content

Commit

Permalink
Cleanup and add more write skew tests
Browse files Browse the repository at this point in the history
  • Loading branch information
al8n committed Apr 27, 2024
1 parent 2fc2012 commit bd9a80f
Show file tree
Hide file tree
Showing 21 changed files with 864 additions and 682 deletions.
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ wmark = { path = "wmark", version = "0.1", default-features = false }
txn-core = { path = "txn-core", version = "0.1", default-features = false }
txn = { path = "txn", version = "0.1", default-features = false }
async-txn = { path = "async-txn", version = "0.1", default-features = false }
skipdb-core = { path = "skipdb-core", version = "0.1", default-features = false }
skipdb-core = { path = "skipdb-core", version = "0.2", default-features = false }


[workspace.metadata.docs.rs]
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,13 @@ This repository contains two kinds of in-memory key-value database which support

Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.

Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

2. `OptimisticDb`

Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.

All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

### Features

Expand Down
4 changes: 2 additions & 2 deletions async-skipdb/README-zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,13 @@ This crate contains two kinds of in-memory key-value database:

Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.

Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

2. `OptimisticDb`

Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.

All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

## Features

Expand Down
4 changes: 2 additions & 2 deletions async-skipdb/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,13 @@ This crate contains two kinds of in-memory key-value database:

Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.

Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

2. `OptimisticDb`

Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.

All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

## Features

Expand Down
78 changes: 0 additions & 78 deletions async-skipdb/src/optimistic/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -359,84 +359,6 @@ fn txn_write_skew_smol() {
smol::block_on(txn_write_skew_in::<SmolSpawner>());
}

// https://wiki.postgresql.org/wiki/SSI#Intersecting_Data
async fn txn_write_skew_intersecting_data_in<S: AsyncSpawner>() {
let db: OptimisticDb<&'static str, u64, S> = OptimisticDb::new().await;

// Setup
let mut txn = db.write().await;
txn.insert("a1", 10).unwrap();
txn.insert("a2", 20).unwrap();
txn.insert("b1", 100).unwrap();
txn.insert("b2", 200).unwrap();
txn.commit().await.unwrap();
assert_eq!(1, db.version().await);

let mut txn1 = db.write().await;
let val = txn1
.iter()
.unwrap()
.filter_map(|ele| {
if ele.key().starts_with('a') {
Some(*ele.value())
} else {
None
}
})
.sum::<u64>();
txn1.insert("b3", 30).unwrap();
assert_eq!(30, val);

let mut txn2 = db.write().await;
let val = txn2
.iter()
.unwrap()
.filter_map(|ele| {
if ele.key().starts_with('b') {
Some(*ele.value())
} else {
None
}
})
.sum::<u64>();
txn2.insert("a3", 300).unwrap();
assert_eq!(300, val);
txn2.commit().await.unwrap();
txn1.commit().await.unwrap_err();

let mut txn3 = db.write().await;
let val = txn3
.iter()
.unwrap()
.filter_map(|ele| {
if ele.key().starts_with('a') {
Some(*ele.value())
} else {
None
}
})
.sum::<u64>();
assert_eq!(330, val);
}

#[tokio::test]
#[cfg(feature = "tokio")]
async fn txn_write_skew_intersecting_data_tokio() {
txn_write_skew_intersecting_data_in::<TokioSpawner>().await;
}

#[async_std::test]
#[cfg(feature = "async-std")]
async fn txn_write_skew_intersecting_data_async_std() {
txn_write_skew_intersecting_data_in::<AsyncStdSpawner>().await;
}

#[test]
#[cfg(feature = "smol")]
fn txn_write_skew_intersecting_data_smol() {
smol::block_on(txn_write_skew_intersecting_data_in::<SmolSpawner>());
}

async fn txn_conflict_get_in<S: AsyncSpawner>() {
let set_count = Arc::new(AtomicU32::new(0));

Expand Down
21 changes: 9 additions & 12 deletions async-skipdb/src/serializable/optimistic.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use async_txn::{error::WtmError, PwmComparableRange};
use skipdb_core::rev_range::WriteTransactionRevRange;

use std::{convert::Infallible, future::Future, ops::Bound};
use std::{convert::Infallible, future::Future};

use super::*;

Expand Down Expand Up @@ -183,18 +183,14 @@ where
&mut self,
) -> Result<TransactionIter<'_, K, V, BTreeCm<K>>, TransactionError<Infallible, Infallible>> {
let version = self.wtm.version();
let (mut marker, pm) = self
let (marker, pm) = self
.wtm
.blocking_marker_with_pm()
.ok_or(TransactionError::Discard)?;

let start: Bound<K> = Bound::Unbounded;
let end: Bound<K> = Bound::Unbounded;
marker.mark_range((start, end));
let committed = self.db.inner.map.iter(version);
let pendings = pm.iter();

Ok(TransactionIter::new(pendings, committed, None))
Ok(TransactionIter::new(pendings, committed, Some(marker)))
}

/// Iterate over the entries of the write transaction in reverse order.
Expand All @@ -204,18 +200,19 @@ where
) -> Result<WriteTransactionRevIter<'_, K, V, BTreeCm<K>>, TransactionError<Infallible, Infallible>>
{
let version = self.wtm.version();
let (mut marker, pm) = self
let (marker, pm) = self
.wtm
.blocking_marker_with_pm()
.ok_or(TransactionError::Discard)?;

let start: Bound<K> = Bound::Unbounded;
let end: Bound<K> = Bound::Unbounded;
marker.mark_range((start, end));
let committed = self.db.inner.map.iter_rev(version);
let pendings = pm.iter().rev();

Ok(WriteTransactionRevIter::new(pendings, committed, None))
Ok(WriteTransactionRevIter::new(
pendings,
committed,
Some(marker),
))
}

/// Returns an iterator over the subset of entries of the database.
Expand Down
78 changes: 0 additions & 78 deletions async-skipdb/src/serializable/optimistic/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -358,84 +358,6 @@ fn txn_write_skew_smol() {
smol::block_on(txn_write_skew_in::<SmolSpawner>());
}

// https://wiki.postgresql.org/wiki/SSI#Intersecting_Data
async fn txn_write_skew_intersecting_data_in<S: AsyncSpawner>() {
let db: SerializableDb<&'static str, u64, S> = SerializableDb::new().await;

// Setup
let mut txn = db.optimistic_write().await;
txn.insert("a1", 10).unwrap();
txn.insert("a2", 20).unwrap();
txn.insert("b1", 100).unwrap();
txn.insert("b2", 200).unwrap();
txn.commit().await.unwrap();
assert_eq!(1, db.version().await);

let mut txn1 = db.optimistic_write().await;
let val = txn1
.iter()
.unwrap()
.filter_map(|ele| {
if ele.key().starts_with('a') {
Some(*ele.value())
} else {
None
}
})
.sum::<u64>();
txn1.insert("b3", 30).unwrap();
assert_eq!(30, val);

let mut txn2 = db.optimistic_write().await;
let val = txn2
.iter()
.unwrap()
.filter_map(|ele| {
if ele.key().starts_with('b') {
Some(*ele.value())
} else {
None
}
})
.sum::<u64>();
txn2.insert("a3", 300).unwrap();
assert_eq!(300, val);
txn2.commit().await.unwrap();
txn1.commit().await.unwrap_err();

let mut txn3 = db.optimistic_write().await;
let val = txn3
.iter()
.unwrap()
.filter_map(|ele| {
if ele.key().starts_with('a') {
Some(*ele.value())
} else {
None
}
})
.sum::<u64>();
assert_eq!(330, val);
}

#[tokio::test]
#[cfg(feature = "tokio")]
async fn txn_write_skew_intersecting_data_tokio() {
txn_write_skew_intersecting_data_in::<TokioSpawner>().await;
}

#[async_std::test]
#[cfg(feature = "async-std")]
async fn txn_write_skew_intersecting_data_async_std() {
txn_write_skew_intersecting_data_in::<AsyncStdSpawner>().await;
}

#[test]
#[cfg(feature = "smol")]
fn txn_write_skew_intersecting_data_smol() {
smol::block_on(txn_write_skew_intersecting_data_in::<SmolSpawner>());
}

async fn txn_conflict_get_in<S: AsyncSpawner>() {
let set_count = Arc::new(AtomicU32::new(0));

Expand Down
2 changes: 1 addition & 1 deletion skipdb-core/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "skipdb-core"
version.workspace = true
version = "0.2.0"
edition.workspace = true
rust-version.workspace = true
repository.workspace = true
Expand Down
4 changes: 2 additions & 2 deletions skipdb/README-zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,13 @@ This crate contains two kinds of in-memory key-value database:

Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.

Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

2. `OptimisticDb`

Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.

All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

## Features

Expand Down
4 changes: 2 additions & 2 deletions skipdb/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,13 @@ This crate contains two kinds of in-memory key-value database:

Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.

Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

2. `OptimisticDb`

Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.

All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. https://wiki.postgresql.org/wiki/SSI#Intersecting_Data.

## Features

Expand Down

0 comments on commit bd9a80f

Please sign in to comment.