You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Migrate the TCGPlayer/Marketplace pricing functionality from the scheduled cloud function + Firestore storage to its own server/database to reduce the Firebase cost + Read usage.
Due to usage limits with Firestore, it would be costly to increase the # of times the function pulls all of the card prices and stores them in Firestore. This limitation leads to inaccurate or stale pricing data later in the day (since, currently, prices are only updated at 6 am EST every day.
With a dedicated server devoted to the polling and storage of the pricing data we can update it to do this much more frequently, say like ~1-2 hrs, as well as doing some efficiency optimizations by analyzing which cards prices don't fluctuate that often (like commons, or older cards) and reduce their polling rate to once every day or few. Then we can devote more frequently polling to cards with a more volatile market.
Tasks
Build ktor + postgres server to periodically pull price information and serve with lightweight API
Build caching integration in-app to replace the current Firestore backed implementation
The API returns the timestamp of when the server should run it's price check again and this mechanism should use that to invalidate pricing data.
The text was updated successfully, but these errors were encountered:
Migrate the TCGPlayer/Marketplace pricing functionality from the scheduled cloud function + Firestore storage to its own server/database to reduce the Firebase cost + Read usage.
Due to usage limits with Firestore, it would be costly to increase the # of times the function pulls all of the card prices and stores them in Firestore. This limitation leads to inaccurate or stale pricing data later in the day (since, currently, prices are only updated at 6 am EST every day.
With a dedicated server devoted to the polling and storage of the pricing data we can update it to do this much more frequently, say like ~1-2 hrs, as well as doing some efficiency optimizations by analyzing which cards prices don't fluctuate that often (like commons, or older cards) and reduce their polling rate to once every day or few. Then we can devote more frequently polling to cards with a more volatile market.
Tasks
The text was updated successfully, but these errors were encountered: