Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a citus_use_snapshot function for basic distributed snapshot isolation #6489

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

marcocitus
Copy link
Member

@marcocitus marcocitus commented Nov 11, 2022

DESCRIPTION: Adds a citus_use_snapshot UDF for basic distributed snapshot isolation

This PR implements a basic form of distributed snapshot isolation in a way that's similar to citus_create_restore_point. The citus_use_snapshot UDF:

  1. Locks pg_dist_transaction on all nodes
  2. Calls pg_export_snapshot on all nodes
  3. Unlocks pg_dist_transaction
  4. Uses the exported snapshots for all remote transaction blocks

By taking the pg_dist_transaction lock, we ensure that all ongoing 2PCs have finished when we do pg_export_snapshot, so the queries in the transaction block will not see any partially committed transactions. The lock is released immediately so it only briefly interrupts multi-shard writes in their final stages of commit.

The citus_use_snapshot UDF can only be used in REPEATABLE READ (or SERIALIZABLE) mode at the start of a transaction block. Currently, multi-shard writes are not supported due to a postgres limitation: cannot PREPARE a transaction that has exported snapshots.

Usage:

DROP TABLE IF EXISTS test;
CREATE TABLE test (key int primary key, value int);
INSERT INTO test SELECT s, 0 FROM generate_series(1,100) s;
SELECT create_distributed_table('test','key');

BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
SELECT citus_use_snapshot();
SELECT sum(value) FROM test; 
END;

generate concurrent multi-shard updates that should leave sum at 0:

$ cat test.sql 
\set aid random(1, 100)
\set bid random(1, 100)
BEGIN;
UPDATE test SET value = value + 1 WHERE key = :aid;
UPDATE test SET value = value - 1 WHERE key = :bid;
END;
$ pgbench -c 20 -j 4 -f test.sql -P 10 -T 60 -n

(will have some regular deadlocks)

@marcocitus marcocitus force-pushed the marcocitus/distributed-snapshot-isolation-v0 branch from 62aef3a to b03a1e3 Compare November 11, 2022 15:19
@marcocitus marcocitus force-pushed the marcocitus/distributed-snapshot-isolation-v0 branch from b03a1e3 to 21b0500 Compare November 11, 2022 15:37
@marcocitus marcocitus changed the title Add a citus_use_snapshot UDF for basic distributed snapshot isolation Add a citus_use_snapshot function for basic distributed snapshot isolation Nov 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants