uses evm tool via server rather than proccess forks

[sudeepdino008]

Tested this with the No_such_log fix PR and it was working smooth. Note that this depends on covalenthq/erigon#11 getting merged.
Need to fix testing and any Dockerfile for this.

[noslav]

This is a pretty big PR and it also introduces a fork from the "binary as a pipeline transformer plugin" direction we had before. I would suggest when we merge this after the comments and some corrections are addressed - and that we release the v0.2.0 of rudder, because of the departure from the earlier build direction or support both in the future - running the transformer binaries on bsps through the porcelain process and/or running the http servers as externally dependent services, it would complicate things a bit and so how to do that of course I would leave to you!

[noslav]

I think we should still have a makefile for rudder setup where we can consolidate different calls to the iex like loading dependencies etc.

[noslav]

@sudeepdino008 thanks for fixing the CI along with this, btw I noticed some performance drops from using the evm server opposed to the plugin, what do you think we can do about this?

use as plugin

rudder        | 17:23:58.972 [info] Counter for bsp_metrics - [execute: 3]
rudder        | 17:23:58.972 [info] LastValue for bsp_metrics - [execute_last_exec_time: 1.65e-4]
rudder        | 17:23:58.972 [info] Sum for bsp_metrics - [execute_total_exec_time: 9.919999999999998e-4]
rudder        | 17:23:58.972 [info] Summary for bsp_metrics  - ***1.65e-4, 4.4199999999999996e-4***

use as server

rudder        | 13:22:58.696 [info] Counter for bsp_metrics - [execute: 3]
rudder        | 13:22:58.696 [info] LastValue for bsp_metrics - [execute_last_exec_time: 5.909999999999999e-4]
rudder        | 13:22:58.696 [info] Sum for bsp_metrics - [execute_total_exec_time: 0.001382]
rudder        | 13:22:58.696 [info] Summary for bsp_metrics  - ***3.4399999999999996e-4, 5.909999999999999e-4***

anything we can do to make it up?

[sudeepdino008]

This is a pretty big PR and it also introduces a fork from the "binary as a pipeline transformer plugin" direction we had before. I would suggest when we merge this after the comments and some corrections are addressed - and that we release the v0.2.0 of rudder, because of the departure from the earlier build direction or support both in the future - running the transformer binaries on bsps through the porcelain process and/or running the http servers as externally dependent services, it would complicate things a bit and so how to do that of course I would leave to you!

running both would be unadvisable - the cost of maintaining and following the documentation and code is too big to justify keeping both approaches in the repo.

[sudeepdino008]

good catch on the perf issues. I'm however comfortable with how we have here. We should analyze the pipeline later when there's a need for it; right now in this PR, the only thing that changed was http overhead instead of process forks. I'll nevertheless explore some simple things which should fix evm server perf and post a separate PR for that.

[sudeepdino008]

@kitti-katy reverted to defp
I'm fine with the current state of check in listener. The thing tha tmight go wrong is if the "isSessionOpen" check fails and so error is reported, then doesn't that stop the block specimen event listener? Seems like it should continue listening (and maybe have 3 restarts allowed in 2 minutes) and not error out and stop.

[kitti-katy]

@kitti-katy reverted to defp I'm fine with the current state of check in listener. The thing tha tmight go wrong is if the "isSessionOpen" check fails and so error is reported, then doesn't that stop the block specimen event listener? Seems like it should continue listening (and maybe have 3 restarts allowed in 2 minutes) and not error out and stop.

@sudeepdino008 there are other places where the listener makes web3 calls (to fetch logs and block height) which might fail if something is wrong with the node, I don't think isSession function will ever have a failing input (it just accesses storage vars), so it will fail if we made an upgrade on the contract side that does not match the old abi.

In the first case, the http adapter already makes 5 retries and in the second case if the contract is changed, I assume they will have to upgrade the rudder too.

[sudeepdino008]

@kitti-katy reverted to defp I'm fine with the current state of check in listener. The thing tha tmight go wrong is if the "isSessionOpen" check fails and so error is reported, then doesn't that stop the block specimen event listener? Seems like it should continue listening (and maybe have 3 restarts allowed in 2 minutes) and not error out and stop.

@sudeepdino008 there are other places where the listener makes web3 calls (to fetch logs and block height) which might fail if something is wrong with the node, I don't think isSession function will ever have a failing input (it just accesses storage vars), so it will fail if we made an upgrade on the contract side that does not match the old abi.

In the first case, the http adapter already makes 5 retries and in the second case if the contract is changed, I assume they will have to upgrade the rudder too.

sounds good @kitti-katy