Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

performance optimizations and startup improvements #1844

Merged
merged 5 commits into from
Jul 26, 2021

Conversation

chappjc
Copy link
Member

@chappjc chappjc commented Jul 23, 2021

This includes a number of performance optimizations and startup improvements:

  • Less redundant calls to TxHash and BlockHash, using CachedTxHash when it is a thread-safe location.
  • txhelpers.MsgTxAtomicSwapsInfo: Avoid allocation and computing tx hash unless needed.
    Caller must check for nil result even with nil error. Check inputs before outputs. Skip output loop entirely if outputSpends map is empty.
  • dcrpg: scan with MsgTxAtomicSwapsInfo in separate loop. Move the MsgTxAtomicSwapsInfo loop out of the transaction
    processing loop. All N txns were checked N times each!
  • Do not call getrawtransaction in (*StakeDatabase).applyDiff
  • remove blockgate and improve startup. This removes the unused RPC "blockgate" that became obsolete when
    dcrsqlite was removed. Polish the sync handling, removing ensureSync and moving the legacy
    project fund responsibility to main instead of SyncChainDB.

Stage 1:

[INF] PSQL: Beginning SYNC STAGE 1 of 5 (block data import).
[INF] PSQL: Scanning genesis block into chain db.
[DBG] PSQL: UTXO cache size: 0
[INF] PSQL: Processing blocks 1 to 500...
[INF] PSQL: Processing blocks 501 to 1000...
[INF] PSQL: Processing blocks 1001 to 1500...
[INF] PSQL: Processing blocks 1501 to 2000...
[INF] PSQL: Processing blocks 2001 to 2500...
[INF] PSQL: (110 blk/s, 1547 tx/s, 1738 vin/sec, 5099 vout/s)
...
[INF] PSQL: Processing blocks 575001 to 575201...
[INF] PSQL: Avg. speed: 1119 tx/s, 4771 vout/s
[INF] PSQL: Block import elapsed: 149.55 minutes, 575202 blocks (64.11 blocks/s)

Avoid allocation and computing tx hash unless needed.
Caller must check for nil result even with nil error.

Check inputs before outputs.

Skip output loop entirely if outputSpends map is empty.
Move the MsgTxAtomicSwapsInfo loop out of the transaction
processing loop. All N txns were checked N times each!
This removes the unused RPC "blockgate" that became obsolete when
dcrsqlite was removed.

Polish the sync handling, removing ensureSync and moving the legacy
project fund responsibility to main instead of SyncChainDB.

Add PiProposalsHistory guard to prevent repeated calls from starting
redundant concurrent queries. This is needed because the initial Pi
updates started in main are async, plus the regular updates are on
a timer and they need to know if a previous update is still running.
@chappjc chappjc marked this pull request as ready for review July 26, 2021 13:56
@chappjc chappjc merged commit b1c6abb into decred:master Jul 26, 2021
@chappjc chappjc deleted the optim branch July 26, 2021 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant