Single JSON serialization for DB and S3 - #4679
Conversation
|
Claude finished @jmg-duarte's task in 2m 48s —— View job PR Review: Single JSON serialization for DB and S3
Nice, clean refactor. Collapsing the two independent serializations (DB via One trade-off worth being aware of (already acknowledged by the code comment): serialization now sits on the run loop's critical path, whereas before both serializations happened entirely in background tasks. The production graph and the Only one minor, optional nit posted inline ( |
MartinquaXD
left a comment
There was a problem hiding this comment.
The idea makes sense to me but the current implementation leaves some performance on the table IIUC.
| }) | ||
| .await | ||
| .context("auction serialization task panicked")??; | ||
| let data = std::str::from_utf8(&json).context("serialized auction is not valid UTF-8")?; |
There was a problem hiding this comment.
I think we could avoid re-checking that this is valid utf8 by passing the AuctionUpload struct directly and making its constructor do the serialization. That way we know the immutable bytes we get from the AuctionUpload struct is not only valid utf8 but even valid JSON.
In the end we can just cast the bytes to a string in the DB query IIRC. At the very least that should at least move the re-validation into postgres if it doesn't skip it outright.
Description
Serialize the raw auction data a single data, instead of one per persistence method
Changes
How to test
Ran it last night in mainnet prod, the dip around 22 was when it was deployed