Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize worst db query #246

Merged
merged 1 commit into from
May 15, 2024
Merged

Optimize worst db query #246

merged 1 commit into from
May 15, 2024

Conversation

SuperSandro2000
Copy link
Member

Before:
> EXPLAIN ANALYZE SELECT * FROM assets WHERE next_scrape_at <= NOW()  ORDER BY next_scrape_at ASC, id ASC  FOR UPDATE SKIP LOCKED LIMIT 3;
+-------------------------------------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                                              |
|-------------------------------------------------------------------------------------------------------------------------|
| Limit  (cost=860.37..860.41 rows=3 width=137) (actual time=2.933..2.939 rows=3 loops=1)                                 |
|   ->  LockRows  (cost=860.37..876.21 rows=1267 width=137) (actual time=2.932..2.937 rows=3 loops=1)                     |
|         ->  Sort  (cost=860.37..863.54 rows=1267 width=137) (actual time=2.909..2.910 rows=3 loops=1)                   |
|               Sort Key: next_scrape_at, id                                                                              |
|               Sort Method: quicksort  Memory: 26kB                                                                      |
|               ->  Seq Scan on assets  (cost=0.00..844.00 rows=1267 width=137) (actual time=0.168..2.888 rows=7 loops=1) |
|                     Filter: (next_scrape_at <= now())                                                                   |
|                     Rows Removed by Filter: 9526                                                                        |
| Planning Time: 6.214 ms                                                                                                 |
| Execution Time: 2.976 ms                                                                                                |
+-------------------------------------------------------------------------------------------------------------------------+

After:
> EXPLAIN ANALYZE SELECT * FROM assets WHERE next_scrape_at <= NOW()  ORDER BY next_scrape_at ASC, id ASC  FOR UPDATE SKIP LOCKED LIMIT 3;
+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                                                                                 |
|------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Limit  (cost=0.74..2.21 rows=3 width=137) (actual time=0.977..0.982 rows=3 loops=1)                                                                        |
|   ->  LockRows  (cost=0.74..912.04 rows=1849 width=137) (actual time=0.976..0.980 rows=3 loops=1)                                                          |
|         ->  Incremental Sort  (cost=0.74..893.55 rows=1849 width=137) (actual time=0.096..0.098 rows=13 loops=1)                                           |
|               Sort Key: next_scrape_at, id                                                                                                                 |
|               Presorted Key: next_scrape_at                                                                                                                |
|               Full-sort Groups: 1  Sort Method: quicksort  Average Memory: 30kB  Peak Memory: 30kB                                                         |
|               ->  Index Scan using assets_next_scrape_at_idx on assets  (cost=0.29..810.34 rows=1849 width=137) (actual time=0.019..0.061 rows=33 loops=1) |
|                     Index Cond: (next_scrape_at <= now())                                                                                                  |
| Planning Time: 2.214 ms                                                                                                                                    |
| Execution Time: 1.040 ms                                                                                                                                   |
+------------------------------------------------------------------------------------------------------------------------------------------------------------+

Before:
> EXPLAIN ANALYZE SELECT * FROM assets WHERE next_scrape_at <= NOW()  ORDER BY next_scrape_at ASC, id ASC  FOR UPDATE SKIP LOCKED LIMIT 3;
+-------------------------------------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                                              |
|-------------------------------------------------------------------------------------------------------------------------|
| Limit  (cost=860.37..860.41 rows=3 width=137) (actual time=2.933..2.939 rows=3 loops=1)                                 |
|   ->  LockRows  (cost=860.37..876.21 rows=1267 width=137) (actual time=2.932..2.937 rows=3 loops=1)                     |
|         ->  Sort  (cost=860.37..863.54 rows=1267 width=137) (actual time=2.909..2.910 rows=3 loops=1)                   |
|               Sort Key: next_scrape_at, id                                                                              |
|               Sort Method: quicksort  Memory: 26kB                                                                      |
|               ->  Seq Scan on assets  (cost=0.00..844.00 rows=1267 width=137) (actual time=0.168..2.888 rows=7 loops=1) |
|                     Filter: (next_scrape_at <= now())                                                                   |
|                     Rows Removed by Filter: 9526                                                                        |
| Planning Time: 6.214 ms                                                                                                 |
| Execution Time: 2.976 ms                                                                                                |
+-------------------------------------------------------------------------------------------------------------------------+

After:
> EXPLAIN ANALYZE SELECT * FROM assets WHERE next_scrape_at <= NOW()  ORDER BY next_scrape_at ASC, id ASC  FOR UPDATE SKIP LOCKED LIMIT 3;
+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                                                                                 |
|------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Limit  (cost=0.74..2.21 rows=3 width=137) (actual time=0.977..0.982 rows=3 loops=1)                                                                        |
|   ->  LockRows  (cost=0.74..912.04 rows=1849 width=137) (actual time=0.976..0.980 rows=3 loops=1)                                                          |
|         ->  Incremental Sort  (cost=0.74..893.55 rows=1849 width=137) (actual time=0.096..0.098 rows=13 loops=1)                                           |
|               Sort Key: next_scrape_at, id                                                                                                                 |
|               Presorted Key: next_scrape_at                                                                                                                |
|               Full-sort Groups: 1  Sort Method: quicksort  Average Memory: 30kB  Peak Memory: 30kB                                                         |
|               ->  Index Scan using assets_next_scrape_at_idx on assets  (cost=0.29..810.34 rows=1849 width=137) (actual time=0.019..0.061 rows=33 loops=1) |
|                     Index Cond: (next_scrape_at <= now())                                                                                                  |
| Planning Time: 2.214 ms                                                                                                                                    |
| Execution Time: 1.040 ms                                                                                                                                   |
+------------------------------------------------------------------------------------------------------------------------------------------------------------+
@majewsky majewsky merged commit 292cdbd into master May 15, 2024
7 checks passed
@majewsky majewsky deleted the index branch May 15, 2024 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants