Transactions not added to the pool when making repeated calls to sendRawTransaction from the same account

[fabio-muramatsu]

System information

Geth version: 1.6.0-stable
OS & Version: Arch Linux

Issue description

I'm sending multiple transactions using sendRawTransaction calls asynchronously, but some of them seem to be dropped by geth and are not added to the transaction pool. I thought my issue was related to #14375, but in my tests, I define the transaction nonces manually before sending them to geth.

Expected behaviour

All transactions are received by geth and added to the transaction pool as pending transactions.

Actual behaviour

All transactions are received by geth (they are logged in geth's console), but some of them are not added to the transaction pool. No error messages are displayed.

Steps to reproduce the behaviour

I've uploaded sample scripts in this repository.

[holiman]

I experimented a bit with this, by running it sync, and adding the transactions in reverse, starting at nonce=100. After sending all except the 0: this is the result:

> txpool.status
{
  pending: 0,
  queued: 64
}

So apparently, it caps the queued at 64.
Adding the final one, results in all of them being shoved over to pending:

> txpool.status
{
  pending: 65,
  queued: 0
}

And indeed:

	maxQueuedPerAccount  = uint64(64)    // Max limit of queued transactions per address

So what's happening is that since you're throwing all txs in no particular order, most of them will wind up in queued instead of pending. (They only get to pending if they're includable: no nonce-gaps allowed in pending, whereas they're allowed in queued).

And queued will start evicting the higher-nonces transactions after there's already 64 of them in there.

So it's basically expected behaviour. In this case, I can't see any performance gain in sending async over sync (even if we 'fix' this isuse, you're just shuffling work from the js-side over to the geth-side). Transactions are inherently sequential, what you can benefit from is paralellising signing.

However, if you really want to use this approach, you need to batch it in sizes below 64. I tested a few times running async with 50 instead of 100 transactions, and the result was that no transactions were lost.

[fabio-muramatsu]

Thank you for your response. Actually, I was using asynchronous calls because, at least using web3js in Node.js, the rate at which geth receives transactions is much higher than the synchronous version. Maybe the overhead in waiting for the RPC response before making the next call may explain this.

Nonetheless, I was making quick tests here and found another weird behavior here. Could you take a look into this to see if I'm making a mistake? If it is a bug, I can open a new issue here. The problem is very simple: I sent 50 transactions asynchronously to geth using sendRawTransaction, and all of them were added to the transaction pool, as expected. However, calling getTransactionCount using the pending argument returns an arbitrary value.

> txpool.status
{
  pending: 50,
  queued: 0
}
> Object.keys(txpool.content.pending["0x722bbc43bb665a5570640b0a56af48364eaba495"]).length
50
> eth.getTransactionCount("0x722bbc43bb665a5570640b0a56af48364eaba495", "pending")
2
> eth.getTransactionCount("0x722bbc43bb665a5570640b0a56af48364eaba495")
0

This issue does not seem to happen if I send the transactions synchronously. Is this expected behavior? I was trying to call this function to derive the correct nonce for future transactions.

Thank you again for your help.

[holiman]

> txpool.status
{
  pending: 50,
  queued: 0
}

txpool.status returns the number of entries (in total) in the transaction pools pending and queued, respectively. 50 is correct.

Object.keys(txpool.content.pending["0x722bbc43bb665a5570640b0a56af48364eaba495"]).length
50

Also correct.

> eth.getTransactionCount("0x722bbc43bb665a5570640b0a56af48364eaba495")
0

Now, this can be a cause of confusion. eth.getTransactionCount should really be called getNonce. This call asks "what is the nonce for 0x722..., at the latest block" . So none of it's transactions are executed in any block yet, it's still at 0.

> eth.getTransactionCount("0x722bbc43bb665a5570640b0a56af48364eaba495", "pending")
2

This is almost the same as above, but the question that we're asking is "what's the nonce of 0x722... in the pending block".

Geth cobbles together a pending block, and if you're mining, that's what it's trying to mine. That pending block is updated at certain events, e.g. to include uncles or new transactions. In this particular case, the pending block contains two of your transactions. If you would mine a block, my guess is that the next block after that would contain the other 48 transactions.

So, TLDR; there's a difference between the pending-pool and the pending block.

[fabio-muramatsu]

Thank you for the explanation. I thought getTransactionCount looked on the transaction pool because of the discussion in this issue, but now I realized it is still open. Do you know whether this change is still planned? This would be important to determine the nonce to use if there are still pending transactions in the pool, as I can't access txpool through web3js.

[holiman]

I guess we'll have to ping @fjl about that.

[holiman]

This issue has been resolved as of #14523 .
Running with --txpool.accountqueue 200 , here's my status after batching in 101 transactions in the wrong order:

> txpool.status
{
  pending: 101,
  queued: 0
}