Error while executing unsigned transaction via eth_call

[bdrhn9]

Hello, I successfully spinned zkevm-node up for testnet. It's fully synced and I'm using current latest release v0.0.3-RC29. While I'm trying to read statically on chain data via eth_call my RPC components throws the following error, normally contracts and backend works great on other chains:

2023-03-25T09:19:53.215Z        INFO    jsonrpc/handler.go:144  failed call: [-32000]failed to execute the unsigned transaction. Params: [{"from":"0x0000000000000000000000000000000000000000","to":"0x3dec619dc529363767dee9e71d8dd1a5bc270d76","data":"0x67a7cfb707bd5ed37a946e4c54f1efb26f013245406fbf760ded5db962a9ce054570aa28"},"latest"]    {"pid": 1, "version": "", "method": "eth_call", "requestId": 66850}
2023-03-25T09:19:53.216Z        ERROR   state/state.go:1315     error processing unsigned transaction  rpc error: code = ResourceExhausted desc = %!(EXTRA string=
/src/log/log.go:140 github.com/0xPolygonHermez/zkevm-node/log.appendStackTraceMaybeArgs()
/src/log/log.go:249 github.com/0xPolygonHermez/zkevm-node/log.Errorf()
/src/state/state.go:1315 github.com/0xPolygonHermez/zkevm-node/state.(*State).internalProcessUnsignedTransaction()
/src/state/state.go:1226 github.com/0xPolygonHermez/zkevm-node/state.(*State).ProcessUnsignedTransaction()
/src/jsonrpc/endpoints_eth.go:98 github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*EthEndpoints).Call.func1()
/src/jsonrpc/dbtxmanager.go:21 github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*dbTxManager).NewDbTxScope()
/src/jsonrpc/endpoints_eth.go:57 github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*EthEndpoints).Call()
/usr/local/go/src/reflect/value.go:556 reflect.Value.call()
/usr/local/go/src/reflect/value.go:339 reflect.Value.Call()
/src/jsonrpc/handler.go:142 github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Handler).Handle()
/src/jsonrpc/server.go:274 github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Server).handleSingleRequest()
/src/jsonrpc/server.go:249 github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Server).handle()
/usr/local/go/src/net/http/server.go:2084 net/http.HandlerFunc.ServeHTTP()
/go/pkg/mod/github.com/didip/tollbooth/v6@v6.1.2/tollbooth.go:311 github.com/didip/tollbooth/v6.LimitHandler.func1()
/usr/local/go/src/net/http/server.go:2084 net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2462 net/http.(*ServeMux).ServeHTTP()
/usr/local/go/src/net/http/server.go:2916 net/http.serverHandler.ServeHTTP()
/usr/local/go/src/net/http/server.go:1966 net/http.(*conn).serve()
)       {"pid": 1, "version": ""}
github.com/0xPolygonHermez/zkevm-node/state.(*State).internalProcessUnsignedTransaction
        /src/state/state.go:1315
github.com/0xPolygonHermez/zkevm-node/state.(*State).ProcessUnsignedTransaction
        /src/state/state.go:1226
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*EthEndpoints).Call.func1
        /src/jsonrpc/endpoints_eth.go:98
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*dbTxManager).NewDbTxScope
        /src/jsonrpc/dbtxmanager.go:21
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*EthEndpoints).Call
        /src/jsonrpc/endpoints_eth.go:57
reflect.Value.call
        /usr/local/go/src/reflect/value.go:556
reflect.Value.Call
        /usr/local/go/src/reflect/value.go:339
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Handler).Handle
        /src/jsonrpc/handler.go:142
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Server).handleSingleRequest
        /src/jsonrpc/server.go:274
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Server).handle
        /src/jsonrpc/server.go:249
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2084
github.com/didip/tollbooth/v6.LimitHandler.func1
        /go/pkg/mod/github.com/didip/tollbooth/v6@v6.1.2/tollbooth.go:311
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2084
net/http.(*ServeMux).ServeHTTP
        /usr/local/go/src/net/http/server.go:2462
net/http.serverHandler.ServeHTTP
        /usr/local/go/src/net/http/server.go:2916
net/http.(*conn).serve
        /usr/local/go/src/net/http/server.go:1966
2023-03-25T09:19:53.216Z        ERROR   jsonrpc/server.go:401   failed to execute the unsigned transaction:rpc error: code = ResourceExhausted desc =  %!s(MISSING)     {"pid": 1, "version": ""}
github.com/0xPolygonHermez/zkevm-node/jsonrpc.rpcErrorResponseWithData
        /src/jsonrpc/server.go:401
github.com/0xPolygonHermez/zkevm-node/jsonrpc.rpcErrorResponse
        /src/jsonrpc/server.go:396
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*EthEndpoints).Call.func1
        /src/jsonrpc/endpoints_eth.go:100
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*dbTxManager).NewDbTxScope
        /src/jsonrpc/dbtxmanager.go:21
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*EthEndpoints).Call
        /src/jsonrpc/endpoints_eth.go:57
reflect.Value.call
        /usr/local/go/src/reflect/value.go:556
reflect.Value.Call
        /usr/local/go/src/reflect/value.go:339
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Handler).Handle
        /src/jsonrpc/handler.go:142
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Server).handleSingleRequest
        /src/jsonrpc/server.go:274
github.com/0xPolygonHermez/zkevm-node/jsonrpc.(*Server).handle
        /src/jsonrpc/server.go:249
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2084
github.com/didip/tollbooth/v6.LimitHandler.func1
        /go/pkg/mod/github.com/didip/tollbooth/v6@v6.1.2/tollbooth.go:311
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2084
net/http.(*ServeMux).ServeHTTP
        /usr/local/go/src/net/http/server.go:2462
net/http.serverHandler.ServeHTTP
        /usr/local/go/src/net/http/server.go:2916
net/http.(*conn).serve
        /usr/local/go/src/net/http/server.go:1966
2023-03-25T09:19:53.216Z        INFO    jsonrpc/handler.go:144  failed call: [-32000]failed to execute the unsigned transaction. Params: [{"from":"0x0000000000000000000000000000000000000000","to":"0x3dec619dc529363767dee9e71d8dd1a5bc270d76","data":"0x67a7cfb7bf1e2b084c65043809d1ab0e1a4096a8fb0d4b81bb9f2d929400f95e3bc2f11e"},"latest"]    {"pid": 1, "version": "", "method": "eth_call", "requestId": 66912}
2023-03-25T09:19:54.627Z        INFO    state/state.go:1608     new l2 blocks detected, Number 422723, Hash 0x02d79414df5491739c86c337e22492b184bb6fbbe436173e95d6631ace44c900  {"pid": 1, "version": ""}

[bdrhn9]

It fails when it try to make eth_call requests more than 500. While sending single request, there is no problem at all. Tuned MaxRequestsPerIPAndSecond to 30000, didn't help.

[bdrhn9]

Solved the problem with tuning prover configuration like following:

"maxExecutorThreads":` 20 -> 900,
"dbNumberOfPoolConnections": 28 -> 950

But I don't think it is proper solution, because allocating ~1000 connections to postgres (max_connections) is hard almost any hardware.

[arnaubennassar]

The issue is related to performance as you've already figured out. We're working on improving performance for the different components of the system, and how we deal with scaling.

[0xtakamaka]

@bdrhn9 can you point-out where is setting "dbNumberOfPoolConnections" is located?

I get "failed to execute the unsigned transaction:not enough keccak counters to continue the execution" error, when using eth_call.

[bdrhn9]

@bdrhn9 can you point-out where is setting "dbNumberOfPoolConnections" is located?

I get "failed to execute the unsigned transaction:not enough keccak counters to continue the execution" error, when using eth_call.

Hey, here it is.

[0xtakamaka]

Thanks! I did not found it, as it's not included in latest release (https://github.com/0xPolygonHermez/zkevm-node/releases/tag/v0.0.3-RC36) :)

[RedCuckoo]

Is not reproducible anymore, request executes successfully on production endpoint of mainnet now.

curl --location --request POST 'https://rpc.ankr.com/polygon_zkevm' --header 'Content-Type: application/json' --data-raw '[{     "jsonrpc":"2.0",     "method":"eth_call",     "params":[{"from":"0x0000000000000000000000000000000000000000","to":"0x3dec619dc529363767dee9e71d8dd1a5bc270d76","data":"0x67a7cfb7bf1e2b084c65043809d1ab0e1a4096a8fb0d4b81bb9f2d929400f95e3bc2f11e"},"latest"]    ,     "id":1 }]

Returns result

[{"jsonrpc":"2.0","id":1,"result":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}]

I have filed a new issue with a more detailed overview of the problem, as it is not gone for the contracts which have to return a lot of data.