Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idempotent bids #13

Merged
merged 4 commits into from
Mar 17, 2020
Merged

Idempotent bids #13

merged 4 commits into from
Mar 17, 2020

Conversation

rsolari
Copy link

@rsolari rsolari commented Mar 17, 2020

Add in-memory idempotency cache to prevent double-sending bids

@rsolari rsolari requested a review from turbomaze March 17, 2020 19:41
@@ -79,6 +80,9 @@ class Wallet extends EventEmitter {

this.txdb = new TXDB(this.wdb);

const capacity = consensus.MAX_BLOCK_OPENS * 10;
this.sendBidResults = new LRU(capacity);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially thought we were just storing the keys but we're also storing the responses. What was your reasoning for this cache size? A number that makes sense to me is 20 blocks worth of bids (current conf time), which would be auctions per block * bids per auction * 20. With the max auctions per block, this would be:

const bidsCacheSizeInBlocks = 20;
const bidsPerAuction = 3; // the average right now is 3.1 ish
const capacity = consensus.MAX_BLOCK_OPENS * bidsPerAuction * bidsCacheSizeInBlocks;

What do you think?

Copy link
Author

@rsolari rsolari Mar 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that we just needed to store the last couple of blocks worth of bids, but I like your reasoning. I think it's fine to provision a little extra. The JSON serialization of a bid is about 2 KB, so 60 * 300 * 2KB = about 36 MB, which is fine.

The main goal here is to just avoid using unbounded memory if the fullnode runs for weeks without ever restarting.

After thinking about it more, we need to provision for peak, which is when a big week's worth of auctions close within a few blocks of each other. I'll increase it to opens * 20 *3.

const idempotencyKey = options ? options.idempotencyKey : null;
if (idempotencyKey) {
if (this.sendBidResults.has(idempotencyKey)) {
return this.sendBidResults.get(idempotencyKey);;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semicolon; does your editor running this repo's eslint?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I'll set that up.

if (idempotencyKey) {
this.sendBidResults.set(idempotencyKey, result);
}
return result

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eslint: semicolon

@turbomaze turbomaze merged commit 68e74ee into production-namebase Mar 17, 2020
@turbomaze turbomaze deleted the raf/idempotent-bids branch March 17, 2020 23:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants