Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve MySQL ReadRelationships paginated throughput #1276

Merged
merged 1 commit into from
Apr 20, 2023

Conversation

vroldanbet
Copy link
Contributor

@vroldanbet vroldanbet commented Apr 20, 2023

Same experiment as #1275 with MySQL

It performed well in all scenarios. Test with 153295 tuples, local MySQL and SpiceDB
Overall regression is still obvious in 2/3 scenarios

zed relationship read authzed/resource reader authzed/user --consistency-full | wc -l
No pagination: 1495.851
with tuple comparison: 47135.61ms
with expanded logic: 9718.667ms

zed relationship read authzed/resource reader --consistency-full | wc -l
No pagination: 1241.65
with tuple comparison: 40956.867ms
with expanded logic: 34398.637ms

zed relationship read authzed/resource --consistency-full | wc -l
No pagination: 1279.036
with tuple comparison: 44472.297ms
with expanded logic: 1653.474ms

current state in main

EXPLAIN ANALYZE SELECT namespace, object_id, relation, userset_namespace, userset_object_id, userset_relation, caveat_name, caveat_context
FROM relation_tuple
WHERE created_transaction <= 5 AND (deleted_transaction = 9223372036854775807 OR deleted_transaction > 9223372036854775807)
AND namespace = 'authzed/resource' AND relation = 'reader' AND ((userset_namespace = 'authzed/user'))
AND (object_id, relation, userset_namespace, userset_object_id, userset_relation) > ('1', 'reader', 'authzed/user', '38dfc848-df6f-11ed-b1b6-0242ac130002', '...')
ORDER BY namespace, object_id, relation, userset_namespace, userset_object_id, userset_relation
LIMIT 1000
"EXPLAIN"
"-> Limit: 1000 row(s)  (cost=1049.65 rows=634) (actual time=158.641..161.601 rows=1000 loops=1)
    -> Filter: ((relation_tuple.object_id,relation_tuple.relation,relation_tuple.userset_namespace,relation_tuple.userset_object_id,relation_tuple.userset_relation) > ('1','reader','authzed/user','38dfc848-df6f-11ed-b1b6-0242ac130002','...'))  (cost=1049.65 rows=634) (actual time=158.640..161.557 rows=1000 loops=1)
        -> Index lookup on relation_tuple using uq_relation_tuple_namespace (namespace='authzed/resource'), with index condition: ((relation_tuple.userset_namespace = 'authzed/user') and (relation_tuple.relation = 'reader') and (relation_tuple.created_transaction <= 5) and (relation_tuple.deleted_transaction = 9223372036854775807))  (cost=1049.65 rows=76084) (actual time=0.025..149.079 rows=50000 loops=1)
"

changes in this PR

EXPLAIN ANALYZE SELECT namespace, object_id, relation, userset_namespace, userset_object_id, userset_relation, caveat_name, caveat_context
FROM relation_tuple
WHERE created_transaction <= 5 AND (deleted_transaction = 9223372036854775807 OR deleted_transaction > 9223372036854775807)
AND namespace = 'authzed/resource' AND relation = 'reader' AND ((userset_namespace = 'authzed/user'))
AND (object_id > '1' OR (object_id = '1' AND relation > 'reader') OR (object_id = '1' AND relation = 'reader' AND userset_namespace > 'authzed/user')
     OR (object_id = '1' AND relation = 'reader' AND userset_namespace = 'authzed/user' AND userset_object_id > '38dfc848-df6f-11ed-b1b6-0242ac130002')
     OR (object_id = '1' AND relation = 'reader' AND userset_namespace = 'authzed/user' AND userset_object_id = '38dfc848-df6f-11ed-b1b6-0242ac130002' AND userset_relation > '...'))
ORDER BY namespace, object_id, relation, userset_namespace, userset_object_id, userset_relation
LIMIT 1000
"EXPLAIN"
"-> Limit: 1000 row(s)  (cost=67187.29 rows=1000) (actual time=0.033..3.458 rows=1000 loops=1)
    -> Index range scan on relation_tuple using uq_relation_tuple_namespace over (namespace = 'authzed/resource' AND object_id = '1' AND relation = 'reader' AND userset_namespace = 'authzed/user' AND userset_object_id = '38dfc848-df6f-11ed-b1b6-0242ac130002' AND '...' < userset_relation) OR (namespace = 'authzed/resource' AND object_id = '1' AND relation = 'reader' AND userset_namespace = 'authzed/user' AND '38dfc848-df6f-11ed-b1b6-0242ac130002' < userset_object_id) OR (namespace = 'authzed/resource' AND '1' < object_id), with index condition: ((relation_tuple.userset_namespace = 'authzed/user') and (relation_tuple.relation = 'reader') and (relation_tuple.namespace = 'authzed/resource') and (relation_tuple.created_transaction <= 5) and (relation_tuple.deleted_transaction = 9223372036854775807) and ((relation_tuple.object_id > '1') or ((relation_tuple.object_id = '1') and (relation_tuple.userset_object_id > '38dfc848-df6f-11ed-b1b6-0242ac130002')) or ((relation_tuple.userset_object_id = '38dfc848-df6f-11ed-b1b6-0242ac130002') and (relation_tuple.object_id = '1') and (relation_tuple.userset_relation > '...'))))  (cost=67187.29 rows=76086) (actual time=0.032..3.384 rows=1000 loops=1)

It performed well in all scenarios. Test with 153295 tuples, local MySQL and SpiceDB

zed relationship read authzed/resource reader authzed/user --consistency-full | wc -l
No pagination: 1495.851
with tuple comparison: 47135.61
with expanded logic: 9718.667

zed relationship read authzed/resource reader --consistency-full | wc -l
No pagination: 1241.65
with tuple comparison: 40956.867
with expanded logic: 34398.637

zed relationship read authzed/resource --consistency-full | wc -l
No pagination: 1279.036
with tuple comparison: 44472.297
with expanded logic: 1653.474
@vroldanbet vroldanbet requested a review from a team as a code owner April 20, 2023 12:21
@github-actions github-actions bot added the area/datastore Affects the storage system label Apr 20, 2023
@vroldanbet vroldanbet assigned vroldanbet and jakedt and unassigned jakedt Apr 20, 2023
@vroldanbet vroldanbet requested a review from jakedt April 20, 2023 12:27
@vroldanbet vroldanbet changed the title use expanded tuple comparison query condition in MySQL Improve MySQL ReadRelationships paginated throughput Apr 20, 2023
@vroldanbet vroldanbet merged commit 750035e into main Apr 20, 2023
24 checks passed
@vroldanbet vroldanbet deleted the use-expanded-tuple-comparison-mysql branch April 20, 2023 17:22
@github-actions github-actions bot locked and limited conversation to collaborators Apr 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/datastore Affects the storage system
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants