Move dispatcher failed dispatch method from ERROR to WARN log severity

[sergerad]

Description

We are running a cluster of validators and when users are hitting the RPC endpoints we get a huge amount of ERROR logs for user-initiated RPC requests. Would like to treat these as WARN instead.

If we could move to DEBUG, would also prefer it over WARN.

Changes include

• Bugfix (non-breaking change that solves an issue)
• Hotfix (change that solves an urgent issue, and requires immediate attention)
• New feature (non-breaking change that adds functionality)
• Breaking change (change that is not backwards-compatible and/or changes current functionality)

Checklist

• I have assigned this PR to myself
• I have added at least 1 reviewer
• I have added the relevant labels
• I have updated the official documentation
• I have added sufficient documentation in code

Testing

• I have tested this code with the official test suite
• I have tested this code manually

[Stefan-Ethernal]

LGTM.

Btw can you provide a bit more context? Are you guys running a public chain and you are facing incorrect input being sent by end-users to JSON RPC endpoints?

I don't have a strong opinion about lowering the log level to DEBUG (possibly I'm more leaning towards not doing so than doing it). The rationale is that logInternalError is triggered in two cases:

• user input is invalid,
• it is not possible to marshal response to JSON.
  Because of the second reason, not sure is it correct to lower logging to DEBUG, because in that case, we would lack those logs by default.

[sergerad]

Btw can you provide a bit more context? Are you guys running a public chain and you are facing incorrect input being sent by end-users to JSON RPC endpoints?

We have 2 chains ATM, both are being used by internal dev teams. In future, we will have 3 chains, 2 of them publicly available.

Here are examples of ERRORs we get from user RPC:

message:"2023-05-18T07:28:37.670Z [ERROR] polygon.server.dispatcher: failed to dispatch: method=eth_call err=\"execution was reverted\""

message:"2023-05-18T03:44:31.392Z [ERROR] polygon.server.dispatcher: failed to dispatch: method=eth_estimateGas err=\"not enough funds to cover gas costs\""

message:"2023-05-18T06:23:38.646Z [ERROR] polygon.server.dispatcher: failed to dispatch: method=eth_sendRawTransaction err=\"cannot extract signature\""

Each of these can happen many times, depending the (user) client's flow.

I understand that the system can perform RPC actions but I think we can ERROR on those lower in the stack than the dispatcher

[Stefan-Ethernal]

I understand that the system can perform RPC actions but I think we can ERROR on those lower in the stack than the dispatcher

We could, although it would require quite of changes (for each endpoint and each function, we would need to add logging there). However, it still wouldn't resolve the original problem and we would still be spammed with a lot of errors, and that problem wouldn't vanish. Because of that, I still find the dispatcher a convenient place because only there we are logging all possible errors.