Skip to content

Refresh Postgres results #66

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rschu1ze merged 1 commit into from
Apr 8, 2025
Merged

Refresh Postgres results #66

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 40 additions & 37 deletions postgresql/results/_index_usage/_m6i.8xlarge_bluesky_1000m_lz4.index_usage
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,33 @@
------------------------------------------------------------------------------------------------------------------------
Index usage for query Q1:

Sort (cost=69519427.00..69519427.03 rows=13 width=40)
Sort (cost=69172569.47..69172569.50 rows=12 width=40)
Sort Key: (count(*)) DESC
-> Finalize GroupAggregate (cost=69519423.40..69519426.75 rows=13 width=40)
-> Finalize GroupAggregate (cost=69172566.16..69172569.26 rows=12 width=40)
Group Key: (((data -> 'commit'::text) ->> 'collection'::text))
-> Gather Merge (cost=69519423.40..69519426.43 rows=26 width=40)
-> Gather Merge (cost=69172566.16..69172568.96 rows=24 width=40)
Workers Planned: 2
-> Sort (cost=69518423.37..69518423.40 rows=13 width=40)
-> Sort (cost=69171566.13..69171566.16 rows=12 width=40)
Sort Key: (((data -> 'commit'::text) ->> 'collection'::text))
-> Partial HashAggregate (cost=69518422.94..69518423.13 rows=13 width=40)
-> Partial HashAggregate (cost=69171565.74..69171565.92 rows=12 width=40)
Group Key: ((data -> 'commit'::text) ->> 'collection'::text)
-> Parallel Seq Scan on bluesky (cost=0.00..67846651.20 rows=334354347 width=32)
-> Parallel Seq Scan on bluesky (cost=0.00..67495020.80 rows=335308987 width=32)
JIT:
Functions: 7
Options: Inlining true, Optimization true, Expressions true, Deforming true

------------------------------------------------------------------------------------------------------------------------
Index usage for query Q2:

Sort (cost=534467372.01..534467372.04 rows=13 width=48)
Sort (cost=531988506.66..531988506.69 rows=12 width=48)
Sort Key: (count(*)) DESC
-> GroupAggregate (cost=438967879.95..534467371.77 rows=13 width=48)
-> GroupAggregate (cost=436041090.60..531988506.44 rows=12 width=48)
Group Key: (((data -> 'commit'::text) ->> 'collection'::text))
-> Gather Merge (cost=438967879.95..526916003.64 rows=755136794 width=569)
-> Gather Merge (cost=436041090.60..524401719.92 rows=758678634 width=566)
Workers Planned: 2
-> Sort (cost=438966879.92..439753480.75 rows=314640331 width=569)
-> Sort (cost=436040090.57..436830380.82 rows=316116098 width=566)
Sort Key: (((data -> 'commit'::text) ->> 'collection'::text)), ((data ->> 'did'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..71927510.45 rows=314640331 width=569)
-> Parallel Seq Scan on bluesky (cost=0.00..71590418.69 rows=316116098 width=566)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text))
JIT:
Functions: 8
Expand All @@ -36,15 +36,15 @@ Index usage for query Q2:
------------------------------------------------------------------------------------------------------------------------
Index usage for query Q3:

Finalize GroupAggregate (cost=134562855.16..206828438.13 rows=473420435 width=72)
Finalize GroupAggregate (cost=134041870.72..206094379.64 rows=472024562 width=72)
Group Key: (EXTRACT(hour FROM to_timestamp(((((data ->> 'time_us'::text))::bigint / 1000000))::double precision))), (((data -> 'commit'::text) ->> 'collection'::text))
-> Gather Merge (cost=134562855.16..188483396.27 rows=394517030 width=72)
-> Gather Merge (cost=134041870.72..187803427.86 rows=393353802 width=72)
Workers Planned: 2
-> Partial GroupAggregate (cost=134561855.14..142945342.03 rows=197258515 width=72)
-> Partial GroupAggregate (cost=134040870.70..142399638.99 rows=196676901 width=72)
Group Key: (EXTRACT(hour FROM to_timestamp(((((data ->> 'time_us'::text))::bigint / 1000000))::double precision))), (((data -> 'commit'::text) ->> 'collection'::text))
-> Sort (cost=134561855.14..135055001.43 rows=197258515 width=64)
-> Sort (cost=134040870.70..134532562.95 rows=196676901 width=64)
Sort Key: (EXTRACT(hour FROM to_timestamp(((((data ->> 'time_us'::text))::bigint / 1000000))::double precision))), (((data -> 'commit'::text) ->> 'collection'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..77718225.92 rows=197258515 width=64)
-> Parallel Seq Scan on bluesky (cost=0.00..77369022.11 rows=196676901 width=64)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text) AND (((data -> 'commit'::text) ->> 'collection'::text) = ANY ('{app.bsky.feed.post,app.bsky.feed.repost,app.bsky.feed.like}'::text[])))
JIT:
Functions: 11
Expand All @@ -53,39 +53,42 @@ Index usage for query Q3:
------------------------------------------------------------------------------------------------------------------------
Index usage for query Q4:

Limit (cost=116709176.13..116709176.13 rows=3 width=40)
-> Sort (cost=116709176.13..116885878.13 rows=70680802 width=40)
Limit (cost=94926225.45..94926225.46 rows=3 width=40)
-> Sort (cost=94926225.45..94926749.83 rows=209751 width=40)
Sort Key: (min(('1970-01-01 00:00:00+00'::timestamp with time zone + ('00:00:00.000001'::interval * (((data ->> 'time_us'::text))::bigint)::double precision))))
-> Finalize GroupAggregate (cost=106788266.40..115795640.01 rows=70680802 width=40)
-> Finalize GroupAggregate (cost=94869849.75..94923514.46 rows=209751 width=40)
Group Key: ((data ->> 'did'::text))
-> Gather Merge (cost=106788266.40..114617626.65 rows=58900668 width=40)
-> Gather Merge (cost=94869849.75..94918795.06 rows=419502 width=40)
Workers Planned: 2
-> Partial GroupAggregate (cost=106787266.37..107818028.06 rows=29450334 width=40)
Group Key: ((data ->> 'did'::text))
-> Sort (cost=106787266.37..106860892.21 rows=29450334 width=569)
Sort Key: ((data ->> 'did'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..72935592.23 rows=29450334 width=569)
-> Sort (cost=94868849.72..94869374.10 rows=209751 width=40)
Sort Key: ((data ->> 'did'::text))
-> Partial HashAggregate (cost=90626411.32..94844573.00 rows=209751 width=40)
Group Key: (data ->> 'did'::text)
Planned Partitions: 32
-> Parallel Seq Scan on bluesky (cost=0.00..72597573.04 rows=29166978 width=566)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text) AND (((data -> 'commit'::text) ->> 'collection'::text) = 'app.bsky.feed.post'::text))
JIT:
Functions: 13
Functions: 10
Options: Inlining true, Optimization true, Expressions true, Deforming true

------------------------------------------------------------------------------------------------------------------------
Index usage for query Q5:

Limit (cost=117901914.66..117901914.66 rows=3 width=64)
-> Sort (cost=117901914.66..118078616.66 rows=70680802 width=64)
Limit (cost=95439986.96..95439986.96 rows=3 width=64)
-> Sort (cost=95439986.96..95440511.33 rows=209751 width=64)
Sort Key: ((EXTRACT(epoch FROM (max(('1970-01-01 00:00:00+00'::timestamp with time zone + ('00:00:00.000001'::interval * (((data ->> 'time_us'::text))::bigint)::double precision))) - min(('1970-01-01 00:00:00+00'::timestamp with time zone + ('00:00:00.000001'::interval * (((data ->> 'time_us'::text))::bigint)::double precision))))) * '1000'::numeric)) DESC
-> Finalize GroupAggregate (cost=106788266.40..116988378.54 rows=70680802 width=64)
-> Finalize GroupAggregate (cost=95380989.36..95437275.96 rows=209751 width=64)
Group Key: ((data ->> 'did'::text))
-> Gather Merge (cost=106788266.40..115133007.49 rows=58900668 width=48)
-> Gather Merge (cost=95380989.36..95429934.68 rows=419502 width=48)
Workers Planned: 2
-> Partial GroupAggregate (cost=106787266.37..108333408.91 rows=29450334 width=48)
Group Key: ((data ->> 'did'::text))
-> Sort (cost=106787266.37..106860892.21 rows=29450334 width=569)
Sort Key: ((data ->> 'did'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..72935592.23 rows=29450334 width=569)
-> Sort (cost=95379989.34..95380513.71 rows=209751 width=48)
Sort Key: ((data ->> 'did'::text))
-> Partial HashAggregate (cost=91136833.44..95354995.11 rows=209751 width=48)
Group Key: (data ->> 'did'::text)
Planned Partitions: 32
-> Parallel Seq Scan on bluesky (cost=0.00..72597573.04 rows=29166978 width=566)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text) AND (((data -> 'commit'::text) ->> 'collection'::text) = 'app.bsky.feed.post'::text))
JIT:
Functions: 13
Options: Inlining true, Optimization true, Expressions true, Deforming true
Functions: 10
Options: Inlining true, Optimization true, Expressions true, Deforming true

77 changes: 40 additions & 37 deletions postgresql/results/_index_usage/_m6i.8xlarge_bluesky_1000m_pglz.index_usage
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,33 @@
------------------------------------------------------------------------------------------------------------------------
Index usage for query Q1:

Sort (cost=69897029.86..69897029.89 rows=13 width=40)
Sort (cost=69151118.72..69151118.75 rows=13 width=40)
Sort Key: (count(*)) DESC
-> Finalize GroupAggregate (cost=69897026.26..69897029.62 rows=13 width=40)
-> Finalize GroupAggregate (cost=69151115.12..69151118.48 rows=13 width=40)
Group Key: (((data -> 'commit'::text) ->> 'collection'::text))
-> Gather Merge (cost=69897026.26..69897029.29 rows=26 width=40)
-> Gather Merge (cost=69151115.12..69151118.16 rows=26 width=40)
Workers Planned: 2
-> Sort (cost=69896026.24..69896026.27 rows=13 width=40)
-> Sort (cost=69150115.10..69150115.13 rows=13 width=40)
Sort Key: (((data -> 'commit'::text) ->> 'collection'::text))
-> Partial HashAggregate (cost=69896025.80..69896025.99 rows=13 width=40)
-> Partial HashAggregate (cost=69150114.66..69150114.86 rows=13 width=40)
Group Key: ((data -> 'commit'::text) ->> 'collection'::text)
-> Parallel Seq Scan on bluesky (cost=0.00..68213928.60 rows=336419440 width=32)
-> Parallel Seq Scan on bluesky (cost=0.00..67475856.00 rows=334851733 width=32)
JIT:
Functions: 7
Options: Inlining true, Optimization true, Expressions true, Deforming true

------------------------------------------------------------------------------------------------------------------------
Index usage for query Q2:

Sort (cost=533218266.90..533218266.93 rows=13 width=48)
Sort (cost=532206068.60..532206068.63 rows=13 width=48)
Sort Key: (count(*)) DESC
-> GroupAggregate (cost=437166993.49..533218266.66 rows=13 width=48)
-> GroupAggregate (cost=436209018.58..532206068.36 rows=13 width=48)
Group Key: (((data -> 'commit'::text) ->> 'collection'::text))
-> Gather Merge (cost=437166993.49..525623267.88 rows=759499858 width=568)
-> Gather Merge (cost=436209018.58..524615357.15 rows=759071101 width=568)
Workers Planned: 2
-> Sort (cost=437165993.47..437957139.15 rows=316458274 width=568)
-> Sort (cost=436208018.55..436998717.62 rows=316279625 width=568)
Sort Key: (((data -> 'commit'::text) ->> 'collection'::text)), ((data ->> 'did'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..72319365.77 rows=316458274 width=568)
-> Parallel Seq Scan on bluesky (cost=0.00..71568642.12 rows=316279625 width=568)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text))
JIT:
Functions: 8
Expand All @@ -36,15 +36,15 @@ Index usage for query Q2:
------------------------------------------------------------------------------------------------------------------------
Index usage for query Q3:

Finalize GroupAggregate (cost=134776076.46..206825424.55 rows=472003856 width=72)
Finalize GroupAggregate (cost=134473230.64..207053525.23 rows=475482150 width=72)
Group Key: (EXTRACT(hour FROM to_timestamp(((((data ->> 'time_us'::text))::bigint / 1000000))::double precision))), (((data -> 'commit'::text) ->> 'collection'::text))
-> Gather Merge (cost=134776076.46..188535275.14 rows=393336546 width=72)
-> Gather Merge (cost=134473230.64..188628591.92 rows=396235124 width=72)
Workers Planned: 2
-> Partial GroupAggregate (cost=134775076.44..143133478.04 rows=196668273 width=72)
-> Partial GroupAggregate (cost=134472230.62..142892227.00 rows=198117562 width=72)
Group Key: (EXTRACT(hour FROM to_timestamp(((((data ->> 'time_us'::text))::bigint / 1000000))::double precision))), (((data -> 'commit'::text) ->> 'collection'::text))
-> Sort (cost=134775076.44..135266747.12 rows=196668273 width=64)
-> Sort (cost=134472230.62..134967524.52 rows=198117562 width=64)
Sort Key: (EXTRACT(hour FROM to_timestamp(((((data ->> 'time_us'::text))::bigint / 1000000))::double precision))), (((data -> 'commit'::text) ->> 'collection'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..78105780.64 rows=196668273 width=64)
-> Parallel Seq Scan on bluesky (cost=0.00..77374841.81 rows=198117562 width=64)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text) AND (((data -> 'commit'::text) ->> 'collection'::text) = ANY ('{app.bsky.feed.post,app.bsky.feed.repost,app.bsky.feed.like}'::text[])))
JIT:
Functions: 11
Expand All @@ -53,39 +53,42 @@ Index usage for query Q3:
------------------------------------------------------------------------------------------------------------------------
Index usage for query Q4:

Limit (cost=118661659.29..118661659.29 rows=3 width=40)
-> Sort (cost=118661659.29..118846281.04 rows=73848702 width=40)
Limit (cost=95640269.60..95640269.61 rows=3 width=40)
-> Sort (cost=95640269.60..95640809.49 rows=215958 width=40)
Sort Key: (min(('1970-01-01 00:00:00+00'::timestamp with time zone + ('00:00:00.000001'::interval * (((data ->> 'time_us'::text))::bigint)::double precision))))
-> Finalize GroupAggregate (cost=108296096.38..117707178.66 rows=73848702 width=40)
-> Finalize GroupAggregate (cost=95582225.61..95637478.38 rows=215958 width=40)
Group Key: ((data ->> 'did'::text))
-> Gather Merge (cost=108296096.38..116476366.96 rows=61540584 width=40)
-> Gather Merge (cost=95582225.61..95632619.33 rows=431916 width=40)
Workers Planned: 2
-> Partial GroupAggregate (cost=108295096.35..109372056.57 rows=30770292 width=40)
Group Key: ((data ->> 'did'::text))
-> Sort (cost=108295096.35..108372022.08 rows=30770292 width=568)
Sort Key: ((data ->> 'did'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..73337145.93 rows=30770292 width=568)
-> Sort (cost=95581225.59..95581765.48 rows=215958 width=40)
Sort Key: ((data ->> 'did'::text))
-> Partial HashAggregate (cost=91198626.16..95556183.28 rows=215958 width=40)
Group Key: (data ->> 'did'::text)
Planned Partitions: 32
-> Parallel Seq Scan on bluesky (cost=0.00..72573959.27 rows=30130907 width=568)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text) AND (((data -> 'commit'::text) ->> 'collection'::text) = 'app.bsky.feed.post'::text))
JIT:
Functions: 13
Functions: 10
Options: Inlining true, Optimization true, Expressions true, Deforming true

------------------------------------------------------------------------------------------------------------------------
Index usage for query Q5:

Limit (cost=119907856.12..119907856.13 rows=3 width=64)
-> Sort (cost=119907856.12..120092477.88 rows=73848702 width=64)
Limit (cost=96170998.45..96170998.45 rows=3 width=64)
-> Sort (cost=96170998.45..96171538.34 rows=215958 width=64)
Sort Key: ((EXTRACT(epoch FROM (max(('1970-01-01 00:00:00+00'::timestamp with time zone + ('00:00:00.000001'::interval * (((data ->> 'time_us'::text))::bigint)::double precision))) - min(('1970-01-01 00:00:00+00'::timestamp with time zone + ('00:00:00.000001'::interval * (((data ->> 'time_us'::text))::bigint)::double precision))))) * '1000'::numeric)) DESC
-> Finalize GroupAggregate (cost=108296096.38..118953375.49 rows=73848702 width=64)
-> Finalize GroupAggregate (cost=96110254.98..96168207.23 rows=215958 width=64)
Group Key: ((data ->> 'did'::text))
-> Gather Merge (cost=108296096.38..117014847.07 rows=61540584 width=48)
-> Gather Merge (cost=96110254.98..96160648.70 rows=431916 width=48)
Workers Planned: 2
-> Partial GroupAggregate (cost=108295096.35..109910536.68 rows=30770292 width=48)
Group Key: ((data ->> 'did'::text))
-> Sort (cost=108295096.35..108372022.08 rows=30770292 width=568)
Sort Key: ((data ->> 'did'::text))
-> Parallel Seq Scan on bluesky (cost=0.00..73337145.93 rows=30770292 width=568)
-> Sort (cost=96109254.96..96109794.85 rows=215958 width=48)
Sort Key: ((data ->> 'did'::text))
-> Partial HashAggregate (cost=91725917.03..96083474.16 rows=215958 width=48)
Group Key: (data ->> 'did'::text)
Planned Partitions: 32
-> Parallel Seq Scan on bluesky (cost=0.00..72573959.27 rows=30130907 width=568)
Filter: (((data ->> 'kind'::text) = 'commit'::text) AND (((data -> 'commit'::text) ->> 'operation'::text) = 'create'::text) AND (((data -> 'commit'::text) ->> 'collection'::text) = 'app.bsky.feed.post'::text))
JIT:
Functions: 13
Options: Inlining true, Optimization true, Expressions true, Deforming true
Functions: 10
Options: Inlining true, Optimization true, Expressions true, Deforming true

Loading