Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
auction-server/src/kernel/workers.rs
Outdated
| _ = delete_history_interval.tick() => { | ||
| let threshold = OffsetDateTime::now_utc() - Duration::from_secs(delete_threshold_secs); | ||
|
|
||
| sqlx::query!( |
There was a problem hiding this comment.
This query may take a long time, I suggest to delete limited number of rows each time. Sth like 1000
|
@anihamde You need to generate the sqlx queries for github ci |
auction-server/src/kernel/workers.rs
Outdated
| let threshold_opportunity = OffsetDateTime::now_utc() - Duration::from_secs(delete_threshold_secs); | ||
| while n_opportunities_deleted.unwrap_or(DELETE_BATCH_SIZE) >= DELETE_BATCH_SIZE { | ||
| n_opportunities_deleted = Some(sqlx::query!( | ||
| "WITH rows_to_delete AS ( |
There was a problem hiding this comment.
I think we need to add chain_id for the opp as well. As far as I remember, we had no index on creation_time here :-?
There was a problem hiding this comment.
Instead of having a while, you can run a single delete query per tick and reduce the deletion tick interval. I dont feel good about this while loop inside the tokio selecct.
There was a problem hiding this comment.
this is a good point. i can tie this to run across all chains, but keeping the same params
There was a problem hiding this comment.
Makes sense. Can you get rid of the while loop and set the limit to 5K and run it every 1 seconds?
You can also create a function for deletion and add instrument around it to make sure the delete query is fast enough
auction-server/src/config.rs
Outdated
| /// How often to delete rows from the database. | ||
| #[arg(long = "delete-interval-seconds")] | ||
| #[arg(env = "DELETE_INTERVAL_SECONDS")] | ||
| #[arg(default_value = "60")] |
danimhr
left a comment
danimhr
left a comment
There was a problem hiding this comment.
Before merging, please make sure you are running the deletion loop every second.
I think you just need to update the default value for deletion loop.
- I think 1000 is very small for deletion. We may have more bid than that per second. I suggest to increase it to 5000 and
2 participants
This PR adds a loop to delete old rows in the Postgres
bidandopportunitytables. The arguments are optional, and, if not provided, will not induce any deletion.