Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle JsonRpcService startup failure #27075

Merged
merged 1 commit into from Aug 12, 2022

Conversation

jbiseda
Copy link
Contributor

@jbiseda jbiseda commented Aug 11, 2022

Problem

In JsonRpcService::new if start_http fails close_handle_sender will be dropped without sending a value to the channel. Subsequent close_handle_receiver.recv().unwrap() will fail on the closed channel.

Summary of Changes

Send error value over the channel to gracefully handle the failure. Handle errors gracefully in Validator::new rather than using validator::abort().

See #25400 for original change which was reverted by #26130 due to #25941.

Description of issue with original change from #25941:

Problem seems to be that Validator::new fails using abort() which in tests calls panic(). This is handled by the test framework and can leave rocksdb background threads accessing partially torn down state. This is exacerbated by tests in program.rs and commands.rs running parallel tests which call Validator::new. This can be mitigated by running tests serially and using rocksdb DB::cancel_all_background_work.

From a cursory examination of rocksdb code it looks like background threads are cleaned up when the DB object is torn down. One solution is to have Validator::new return a Result and fail without using abort() which will tear down state cleanly.

Fixes #

@jbiseda jbiseda marked this pull request as ready for review August 11, 2022 09:27
Copy link
Contributor

@lijunwangs lijunwangs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@jbiseda jbiseda merged commit e50013a into solana-labs:master Aug 12, 2022
codemonkey6969 added a commit to codemonkey6969/solana that referenced this pull request Aug 12, 2022
xiangzhu70 pushed a commit to xiangzhu70/solana that referenced this pull request Aug 17, 2022
@jbiseda jbiseda deleted the jsonrpcservice-failure-handling branch August 17, 2022 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants