Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[store] one hashset for child ref checks #7654

Merged
merged 3 commits into from
Apr 1, 2024

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Mar 27, 2024

When writes add chunks at execution time, we currently add their direct child references to the memory table's pending ref queue to check for GC'd refs. We do this by constructing a hash map to pass every chunk's child refs to the memtable's pending queue. This PR avoids creating a new hashset per chunk by using a callback to delay reading out the refs. We make one child ref set per memtable, which has the added benefit of deduplicating child refs between chunks.

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_median to_latency_median is_faster
tpcc-scale-factor-1 123.28 121.08 0
test_name server_name server_version tps test_name server_name server_version tps is_faster
tpcc-scale-factor-1 dolt 801a82a 16.46 tpcc-scale-factor-1 dolt ef4341b 17.21 0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
ef4341b ok 5937457
version total_tests
ef4341b 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency_median to_latency_median is_faster
covering_index_scan 3.02 2.97 0
groupby_scan 17.32 17.32 0
index_join 5.09 5.18 0
index_join_scan 2.26 2.26 0
index_scan 62.19 63.32 0
oltp_point_select 0.49 0.49 0
oltp_read_only 8.13 8.28 0
select_random_points 0.78 0.78 0
select_random_ranges 0.94 0.95 0
table_scan 63.32 63.32 0
types_table_scan 173.58 176.73 0
write_tests from_latency_median to_latency_median is_faster
oltp_delete_insert 7.04 7.04 0
oltp_insert 3.49 3.43 0
oltp_read_write 16.12 15.83 0
oltp_update_index 3.62 3.55 0
oltp_update_non_index 3.55 3.49 0
oltp_write_only 8.13 7.84 0
types_delete_insert 7.84 7.7 0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
30427de ok 5937457
version total_tests
30427de 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
4d90526 ok 5937457
version total_tests
4d90526 5937457
correctness_percentage
100.0

Copy link
Contributor

@reltuk reltuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@max-hoffman max-hoffman merged commit 053e6ca into main Apr 1, 2024
20 checks passed
@max-hoffman max-hoffman deleted the max/aggregate-child-ref-hashset branch April 1, 2024 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants