feat: add `reconnecting-rpc-client` #1396

niklasad1 · 2024-01-23T17:10:31Z

This PR adds a reconnecting-rpc-client behind a feature flag --feature unstable-reconnecting-rpc-client and re-exports it from subxt.

The reason why it's added in subxt is for ease-of-use because of orphan-rules etc for anyone that will use it i.e, everyone would need to implement the RpcClientT but won't work without a new type pattern which makes the example much more complicated.

Resolves #1190, resolves #249

…ing-rpc-client

lexnv · 2024-01-26T10:53:46Z

subxt/src/backend/rpc/rpc_client_t.rs

@@ -101,3 +101,94 @@ impl<T: RpcClientT> RpcClientT for Box<T> {
        (**self).subscribe_raw(sub, params, unsub)
    }
 }
+
+#[cfg(feature = "reconnecting-rpc-client")]
+impl RpcClientT for reconnecting_jsonrpsee_ws_client::Client {


I would probably place this under a src/client/reconnecting_client module. Maybe within a ReconnectingClientRpC(reconnecting_jsonrpsee_ws_client::Client).

Ideally we would have a ReconnectingClient<T: Config> that has a ReconnectingClient::builder method and implements OnlineClientT and OfflineClientT traits.
Then we could also expose on the ReconnectingClient a reconnect_count method; and other methods of interest.
This would tidy up a bit the user API. Although the tricky part would be that we'll duplicate the reconnecting_jsonrpsee_ws_client::Client::builder methods on the ReconnectingClient::builder.

Considering this is under a feature-flag at the moment, I'd leave that refactoring as a follow-up as probably would need some extra tought.
What do you think?

My gut feeling is that for now it should just be a standalone optional RPC client that you can use to instantiate the existing OnlineClient, but at some point when we've added more special reconnect logic in places etc we can just use it by default in the current OnlineClient instead of the non-reconnecting-client :)

I agree with the separate module though; something like src/backend/rpc/reconnecting_jsonrpsee_impl.rs maybe to sit next to the current jsonrpsee_impl.rs?

lexnv

This looks good to me! One small nit about moving the RpcClientT implementation for the reconnecting client to a dedicated module. In the future, we could introduce a more complete API (some initial thoughts #1396 (comment))

jsdw · 2024-01-26T11:27:25Z

subxt/examples/reconnecting_rpc_client.rs

+            }
+            // In this example we just ignore when reconnections occurs
+            // but it's technically possible that we can lose
+            // messages on the subscription such as `InFinalizedBlock`


I like this example overall, but when submitting a tx it's particularly tricky to handle as you will lose the subscription anyway won't you and won't get any new messages after reconnecting?

So perhaps a better (and simpler, anyway) example is just subscribing to finalized blocks?

I can't remember how your reconnecting client handles subscriptions etc; does it automatically re-subscribe behind the scenes? That area is an interesting one because on one hand, it might be better if the reconnecting client just reconnects but doesn't try to re-establish anything (so that the Subxt backend's can choose how best to restart stuff), but on the other hand having the client do it is simpler at least (but I fear may not always do what you want). Ultimately we'd need to agree on a "standard" for what is expected from a reconnecting client when it's used in Subxt

I can't remember how your reconnecting client handles subscriptions etc; does it automatically re-subscribe behind the scenes?

Yes, it re-subscribes to subscriptions and re-transmits pending method calls.

It's just a PITA if one have plenty of storage subscriptions opened and manually have to re-subscribe to them.
For rpc-v2 then it's a different story but not really thought of in this use-case.

Let's just make it optional and folks can try it out and complain what's bad :)

Yeahh for now it's behind a feature flag so we can experiment!

My gut feeling is that before it becomes "stable" in Subxt, we'll prob end up wanting the Backend impls to decide when/how to resubscribe etc themselves (eg maybe for TXs we don't try re-sending but for following new/final blocks we just restart it or whatever

Alex had a few cool ideas about distinguish between "stateful (submit_and_watch)" and "stateless (subscribe_storage)" subscriptions but still I think this reconnecting rpc client won't really work that well with the rpc-v2 spec.

One would need to call a few methods for the chainHead_unstable_follow to make emit interesting stuff..

So perhaps a better (and simpler, anyway) example is just subscribing to finalized blocks?

I want to show how to ignore "DisconnectWillReconnect" and continue if a reconnection occurs. If you just call wait_for_finalized_success it will quit with that error but we could wait_for_finalized_success_and_reconnect... I guess.

Added a few comments in the example now hopefully clearer now.

I think this conversation is good start to open another issue how to deal with it reconnections in subxt properly especially with the rpc v2 spec in mind which is particular tricky.

In general, I think just reconnect is probably the way to go and not re-subscribe and so on but we let's try my crate first and collect some feedback what's the best way forward :)

I want to show how to ignore "DisconnectWillReconnect" and continue if a reconnection occurs. If you just call wait_for_finalized_success it will quit with that error but we could wait_for_finalized_success_and_reconnect... I guess.

What I mean is just that it's less code to subscribe to finalied blocks and then show how you can ignore the DisconnectedWillReconnect error. Probably closer to being a valid thing that we can handle nicely too (we can just start the subscription up again as your client does atm anyway)

With transactions we probably(?) don't want to re-submit automatically so showing it as an example sortof hints at a behaviour that I don't think we'll keep, but anyway it's also just more effort building the tx etc when the point of the example is just handling the DisconnectWillReconnect error :)

In general, I think just reconnect is probably the way to go and not re-subscribe and so on but we let's try my crate first and collect some feedback what's the best way forward :)

Yup I agree; I'm happy with this for now since it's behind a feature flag, but before we stabilise it I thnk we should make the RPC client not try to do anything clever and only reconnect. Instead we can make the backend impls decide how to manage disconnects in followup PRs

What I mean is just that it's less code to subscribe to finalied blocks and then show how you can ignore the DisconnectedWillReconnect error. Probably closer to being a valid thing that we can handle nicely too (we can just start the subscription up again as your client does atm anyway)

With transactions we probably(?) don't want to re-submit automatically so showing it as an example sortof hints at a behaviour that I don't think we'll keep, but anyway it's also just more effort building the tx etc when the point of the example is just handling the DisconnectWillReconnect error :)

Got it, makes sense

subxt/examples/reconnecting_rpc_client.rs

subxt/src/backend/rpc/rpc_client_t.rs

jsdw · 2024-02-01T16:24:44Z

subxt/Cargo.toml

@@ -42,6 +42,9 @@ web = [
    "instant/wasm-bindgen"
 ]

+# Enable this to use the reconnecting rpc client
+reconnecting-rpc-client = ["dep:reconnecting-jsonrpsee-ws-client"]


Should we prefix with "unstable" for now until we've settled on the semantics of it (ie probably that the client will reconnect butnot auto resubscribe etc) and handled the DisconnectedWillReconnect?

jsdw · 2024-02-01T16:27:11Z

Mostly LGTM; prooobably we should stick "unstable-" in front of the feature flag so that we can keep playing with the semantics without caring about breaking/changing behaviour between releases, and I still think we could simplify the example to subscribe to finalized blocks instead to show the current reconnect behaviour :)

It also occurs to me that the example isn't referenced in the book anywhere, but for now I don't mind that; maybe we'll hold off until we stabilise it more!

…ing-rpc-client

jsdw

Looks great to me; nice one!

…ing-rpc-client

lexnv

Looks good to me! Nice job here!

For some reason, I keep thinking of having a bit more control over the RetryPolicy.
As a first step, having a global policy makes a lot of sense.

We could look in the future at overriding the policy at a local level of subscriptions / methods. For example, having an API like:

let rpc = Client::builder()
// This is the global default policy, unless specified elsewhere this is used.
.retry_policy(ExponentialBackoff::from_millis(100).max_delay(Duration::from_secs(10)))
// For the `tx_submitAndWatch` we don't attempt at resubmitting the tx again; and let the user handle this if wanted
.local_policy(subxt::rpc::Method::TransactionSubmitAndWatch, Policy::DoNotRetry)

Nothing to worry about in this PR, nicely done again!

niklasad1 added 4 commits January 23, 2024 18:09

initial commit

7c28327

update to reconnecting-ws-client v0.2

bafcb78

re-export: reconnecting_rpc_client behind feature

eb7fcd3

add helper function for reconnect

cf88872

niklasad1 changed the title ~~[WIP]: add reconnecting-rpc-client~~ feat: add reconnecting-rpc-client Jan 25, 2024

niklasad1 marked this pull request as ready for review January 25, 2024 15:26

niklasad1 requested a review from a team as a code owner January 25, 2024 15:26

niklasad1 added 2 commits January 25, 2024 16:47

fix nit in example

b85a547

Merge remote-tracking branch 'origin/master' into integrate-reconnect…

d9f4718

…ing-rpc-client

lexnv reviewed Jan 26, 2024

View reviewed changes

lexnv approved these changes Jan 26, 2024

View reviewed changes

jsdw reviewed Jan 26, 2024

View reviewed changes

subxt/examples/reconnecting_rpc_client.rs Outdated Show resolved Hide resolved

jsdw reviewed Jan 26, 2024

View reviewed changes

subxt/src/backend/rpc/rpc_client_t.rs Outdated Show resolved Hide resolved

jsdw reviewed Jan 26, 2024

View reviewed changes

subxt/src/backend/rpc/rpc_client_t.rs Outdated Show resolved Hide resolved

jsdw reviewed Jan 26, 2024

View reviewed changes

subxt/src/backend/rpc/rpc_client_t.rs Outdated Show resolved Hide resolved

niklasad1 added 3 commits January 26, 2024 17:13

simplify code without weird error fmt

282f86a

address grumbles

57141cd

Merge branch 'master' into integrate-reconnecting-rpc-client

6efcc28

jsdw reviewed Feb 1, 2024

View reviewed changes

niklasad1 added 2 commits February 1, 2024 18:04

Merge remote-tracking branch 'origin/master' into integrate-reconnect…

38b30d4

…ing-rpc-client

address grumbles

c348bb0

jsdw approved these changes Feb 8, 2024

View reviewed changes

niklasad1 added 3 commits February 8, 2024 11:57

update reconnecting-ws-client 0.3

4b6c296

Merge remote-tracking branch 'origin/master' into integrate-reconnect…

1406ec6

…ing-rpc-client

cleanup error message

3a8b520

lexnv approved these changes Feb 8, 2024

View reviewed changes

niklasad1 merged commit cb67f94 into master Feb 8, 2024
12 checks passed

niklasad1 deleted the integrate-reconnecting-rpc-client branch February 8, 2024 12:19

prabal-banerjee mentioned this pull request Feb 23, 2024

Better WS Retry Logic succinctlabs/vectorx#117

Closed

jsdw mentioned this pull request Mar 21, 2024

Prep to release 0.35 #1489

Merged

This was referenced May 7, 2024

Add changes to fix issues running the Spacewalk client in Kubernetes pendulum-chain/spacewalk#519

Merged

Use reconnecting RPC client pendulum-chain/spacewalk#521

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add `reconnecting-rpc-client` #1396

feat: add `reconnecting-rpc-client` #1396

niklasad1 commented Jan 23, 2024 •

edited

Loading

lexnv Jan 26, 2024

jsdw Jan 26, 2024

jsdw Jan 26, 2024

lexnv left a comment

jsdw Jan 26, 2024

niklasad1 Jan 26, 2024

niklasad1 Jan 26, 2024

jsdw Jan 26, 2024

niklasad1 Jan 26, 2024 •

edited

Loading

niklasad1 Jan 26, 2024 •

edited

Loading

niklasad1 Jan 26, 2024

jsdw Feb 1, 2024

niklasad1 Feb 1, 2024

jsdw Feb 1, 2024

jsdw commented Feb 1, 2024

jsdw left a comment

lexnv left a comment

feat: add reconnecting-rpc-client #1396

feat: add reconnecting-rpc-client #1396

Conversation

niklasad1 commented Jan 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lexnv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

niklasad1 Jan 26, 2024 • edited Loading

Choose a reason for hiding this comment

niklasad1 Jan 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsdw commented Feb 1, 2024

jsdw left a comment

Choose a reason for hiding this comment

lexnv left a comment

Choose a reason for hiding this comment

feat: add `reconnecting-rpc-client` #1396

feat: add `reconnecting-rpc-client` #1396

niklasad1 commented Jan 23, 2024 •

edited

Loading

niklasad1 Jan 26, 2024 •

edited

Loading

niklasad1 Jan 26, 2024 •

edited

Loading