Refactor blockchain daemon #2257
Conversation
app/workers/daemons/blockchain.rb
Outdated
|
|
||
| logger.info { "Started processing #{bc.key} block number #{block_id}." } | ||
| def initialize | ||
| super |
There was a problem hiding this comment.
I'm not sure if we still need to call super here clarify with @mnaichuk please
There was a problem hiding this comment.
We need it, because we have to support graceful shutdown, that's why we use running variable.
There was a problem hiding this comment.
@mnaichuk am I right, that we need to call super?
app/workers/daemons/blockchain.rb
Outdated
| end | ||
| end | ||
|
|
||
| def process | ||
| @workers.map(&:join) unless running |
There was a problem hiding this comment.
In this way, we won't have ability to gracefully stop blockchain daemon.
config/blockchains.yml
Outdated
| @@ -0,0 +1,19 @@ | |||
| version: '2' | |||
lib/peatio/bitcoin/client.rb
Outdated
| @@ -43,12 +44,13 @@ def json_rpc(method, params = []) | |||
| private | |||
|
|
|||
| def connection | |||
| Faraday.new(@json_rpc_endpoint).tap do |connection| | |||
| @connection ||= Faraday.new(@json_rpc_endpoint) do |f| | |||
| f.adapter :net_http_persistent | |||
There was a problem hiding this comment.
-
Is it possible to explicitly define number of persistent connections ?
Or add comment for clarifying what is default value -
Is it possible to write spec for this ? For being sure that it works in the way we expect
There was a problem hiding this comment.
- I don't know how to test this kind of stuff, because we stub all calls in the tests.
- I think code related to gem shouldn't be tested in the peatio, it should be tested in the gem.
app/workers/daemons/blockchain.rb
Outdated
| logger.info {"Finished processing #{bc.key} block number #{block_id}."} | ||
| end | ||
| logger.info {"Finished processing #{bc.name} blocks."} | ||
| rescue Peatio::Blockchain::ClientError => e |
There was a problem hiding this comment.
I would rescue StandardError here, to reduce the risk of thread death.
There was a problem hiding this comment.
TODO:
-
Rename process to run and process_block to process
-
Rename config/blockchains.yml -> e.g config/cryptonodes.yml
-
Still I'm not sure if we need
unless runningbefore joining threads -
Add spec for Headers added by net_http_persistent
-
Add spec that faraday returns the same connection instead of initializing new one
-
Try to inspect gem for vulnerabilities for being sure
- Fix issues with timeout. - Split blockchains sync into threads. - Speed up sync using typhoeus adapter. - Use one BlockchainService instance per thread. - Use one connection per thread. - Sleep 10 seconds if synced.
Benchmark in
kovannetwork without deposits/withdraws: 70 blocks/sec.