concurrent processing of get blocks. batches by 50 default #181
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
leovct
reviewed
Jan 4, 2024
leovct
reviewed
Jan 4, 2024
leovct
reviewed
Jan 4, 2024
leovct
reviewed
Jan 4, 2024
leovct
reviewed
Jan 4, 2024
leovct
requested changes
Jan 4, 2024
leovct
reviewed
Jan 4, 2024
gatsbyz
force-pushed
the
jesse/monitor-process-batch-concurrently
branch
from
5fbfc70
to
dfa825b
Compare
leovct
reviewed
Jan 5, 2024
leovct
reviewed
Jan 5, 2024
leovct
reviewed
Jan 5, 2024
leo changes Co-authored-by: Léo Vincent <28714795+leovct@users.noreply.github.com>
gatsbyz
force-pushed
the
jesse/monitor-process-batch-concurrently
branch
from
61f1fce
to
f849992
Compare
leovct
reviewed
Jan 8, 2024
There was a problem hiding this comment.
$ ./polycli monitor --verbosity 700 --rpc-url https://polygon-rpc.com &> debug.txt
# it crashes after a few seconds
$ cat debug.txt
�[90m10:56AM�[0m �[35mTRC�[0m Starting logger in console mode
�[90m10:56AM�[0m �[33mDBG�[0m Using fake peer count �[36merror=�[0m�[31m"the method net_peerCount does not exist/is not available"�[0m
�[90m10:56AM�[0m �[33mDBG�[0m Fetching blocks �[36mChainID=�[0m137 �[36mGasPrice=�[0m93919755129 �[36mHeadBlock=�[0m52079176 �[36mPeerCount=�[0m0
�[90m10:56AM�[0m �[33mDBG�[0m Auto-adjusted batchSize to 200 based on cache limit
�[90m10:56AM�[0m �[33mDBG�[0m Redrawing �[36mBatchSize=�[0m200 �[36mChainID=�[0m137 �[36mGasPrice=�[0m93919755129 �[36mHeadBlock=�[0m52079176 �[36mLowerBlock=�[0m<nil> �[36mPeerCount=�[0m0 �[36mPendingCount=�[0m64165 �[36mTopDisplayedBlock=�[0m52079176 �[36mUpperBlock=�[0m<nil>
panic: close of closed channel
goroutine 48 [running]:
github.com/maticnetwork/polygon-cli/cmd/monitor.(*monitorStatus).processBatchesConcurrently.func1.1()
/Users/leovct/Documents/work/cli/polycli/cmd/monitor/monitor.go:321 +0x59
github.com/maticnetwork/polygon-cli/cmd/monitor.(*monitorStatus).processBatchesConcurrently.func1(0x0)
/Users/leovct/Documents/work/cli/polycli/cmd/monitor/monitor.go:353 +0x68f
created by github.com/maticnetwork/polygon-cli/cmd/monitor.(*monitorStatus).processBatchesConcurrently in goroutine 67
/Users/leovct/Documents/work/cli/polycli/cmd/monitor/monitor.go:317 +0xd1
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
some chains have limits for the max batch processing number. to solve this, concurrent fetching by batches in 50 is implemented.