Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix log and change rpc in response to json rpc error #474

Merged
merged 3 commits into from
Apr 13, 2024

Conversation

bxue-l2
Copy link
Contributor

@bxue-l2 bxue-l2 commented Apr 11, 2024

Why are these changes needed?

Checks

  • I've made sure the lint is passing in this PR.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, in that case, please comment that they are not relevant.
  • Testing Strategy
    • Unit tests
    • Integration tests
    • This PR is not tested :(

@@ -25,7 +25,7 @@ const (
func (f *FailoverController) handleHttpError(httpRespError rpc.HTTPError) (NextEndpoint, ImmediateAction) {
sc := httpRespError.StatusCode
// Default to rotation the current RPC, because it allows a higher chance to get the query completed.
f.Logger.Info("[HTTP Response Error]", "Status Code", sc, "Error", httpRespError)
f.Logger.Info("[HTTP Response Error]", "Curr Rpc Fault", f.numberRpcFault, "Status Code", sc, "Error", httpRespError)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not use spaces as attribute labels
f.Logger.Info("[HTTP Response Error]", "numRPCFfault", f.numberRpcFault, "statusCode", sc, "err", httpRespError)

@bxue-l2 bxue-l2 marked this pull request as ready for review April 11, 2024 18:50
f.Logger.Info("[JSON RPC Response Error]", "Error Code", ec, "Error", rpcError)
// we always attribute JSON RPC error as sender's fault, i.e no connection rotation
return CurrentRPC, Return
f.Logger.Warn("[JSON RPC Response Error]", "numRPCFfault", f.numberRpcFault, "errorCode", ec, "err", rpcError)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to extract top level domain (ie strip api token) from url and include it in message so that oncall knows exactly which provider failed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed, please take a look

}

// If no http response or no rpc response is returned, it is a connection issue,
// since we can't accurately attribute the network issue to neither sender nor receiver
// side. Optimistically, switch rpc client
f.Logger.Info("[Default Response Error]", err)
f.Logger.Warn("[Default Response Error]", "numRPCFfault", f.numberRpcFault, "err", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it logged anywhere when it switches to a different RPC endpoint? That'd be a useful information to see from logs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, logged

@@ -51,7 +51,8 @@ func (f *FailoverController) handleHttpError(httpRespError rpc.HTTPError) (NextE
// If the error is http, non2xx error would generate HTTP error, https://github.com/ethereum/go-ethereum/blob/master/rpc/http.go#L233
// but a 2xx http response could contain JSON RPC error, https://github.com/ethereum/go-ethereum/blob/master/rpc/http.go#L181
// If the error is Websocket or IPC, we only look for JSON error, https://github.com/ethereum/go-ethereum/blob/master/rpc/json.go#L67

//
// This function is protected by mutex from the outside
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "This function assumes the mutex protection from caller to ensure thread safety"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will remove

Copy link
Collaborator

@pschork pschork left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏

@bxue-l2 bxue-l2 merged commit 66a9a6f into Layr-Labs:master Apr 13, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants