multi: handle trade suspension messages.

[dnldd]

This adds the suspension handler to the client core.

[buck54321]

We've got an issue. The spec for suspension is still based off of a simpler time when we had a single epoch duration for the entire DEX. That was changed a while back. The suspension route can no longer send the epoch. It should probably be changed to either a timestamp, or a mapping of market->epoch or market->timestamp.

---

How do we reset the suspension? The docs do say

Users should expect to lose connection during suspension.

but I don't know if that's a strong enough guarantee that we could then hook into handleReconnect to catch resumption of trade and perform the necessary actions.

[chappjc]

I think the server should send a market->epoch map in the suspension message. one message per market

But as far as this PR goes, I think we should put it on hold and first address the issue from the server side to be sure it is practical.

[chappjc]

It's worth saying when this suspension is going to happen. (suspension epoch * epoch duration (ms)) We'd need to get this from the market map, but as @buck54321 pointed out, there will need to be a suspension epoch for each market.

[chappjc]

I'll wrap up the server-side trade suspension PR in a day or two. After we're sure what info will come from the server we can resume this one. Thanks for hanging in there on this one.

[dnldd]

No worries, dcrpool work has been keeping me busy.

[chappjc]

msgjson.TradeSuspension message is implemented server side now. a47e2cd#diff-9a36b4e081b2c1cc9032524815e5d57c

dcrdex/dex/msgjson/types.go

Lines 710 to 717 in 73a3f78

	// TradeSuspension is the SuspensionRoute notification payload. It is part of
	// the orderbook subscription.
	type TradeSuspension struct {
	MarketID string `json:"marketid"`
	FinalEpoch uint64 `json:"finalepoch"`
	SuspendTime uint64 `json:"suspendtime"`
	Persist bool `json:"persistbook"`
	}

[chappjc]

Also, unless all of the markets are suspended, I don't think the client should expect a lost connection.

[dnldd]

noted, will be back at it this weekend.

[chappjc]

To clarify the current market suspend operation from the point of view of the client:

• The server operator commands a suspend at as soon as a time or asap
• dcrdex returns a final epoch and suspend time, and notifies connected clients immediately of the upcoming trade suspension with a msgjson.TradeSuspension.
• Final epoch is the last epoch. It will be completed.
• Suspend time is the time at which the market is scheduled to begin rejecting orders for that market.
• Final epoch [start, end), where end is suspend time.

Also, TradeResumption is not implemented.

I imagine the client will simply prevent submission of orders to suspended markets.

Note that the config message also had this data, so clients that were offline when TradeSuspension is sent still have this info.

[dnldd]

Noted, thank you.

[dnldd]

Apologies, I edited the existing commit instead of adding a new one, will remember not to in the updates to come. @chappjc please provide some feedback when you can. Also let me know if a unified type that would hold the market suspension time and resumption time for a market would be preferable here, instead of using suspension and resumption message maps.

[chappjc]

Will have to review to give suggestion on the client implementation, but what to do really depends on server capabilities. I just put up a draft of the swapper suspend at #406 and it's looking feasible for the server to resend certain messages and to resume coin waiters. Will have a look at this tomorrow.

[JoeGruffins]

I left a few concerns about concurrency. There's lots of mutexes.

[JoeGruffins]

A couple more.

[chappjc]

Thanks for the patience on this one. I felt we needed to get a better idea of what server is going to do before going too far down the suspend/resume road on the client side. I think this all makes sense for suspend. Just a few comments and questions. Will move on to the resume PR #442

[chappjc]

I'm going to outline a plan for "connectivity robustness" in a new issue tomorrow because we need to look at the bigger picture to decide what needs to be done.

"connectivity robustness" will cover any disruption, either planned or unplanned:

• Planned trade suspensions, which may include market config changes and which may involve connectivity loss if all the markets on a DEX are suspended and the operator need to stop the dex (the spec says to expect this!). This PR starts to deal with this, but: (1) we've not talked about what happens when a client connects after the initial trade suspension message was already sent - the config response could create a pending suspend, (2) we've tossed around the idea of a second TradeSuspension at the instant of suspend just in case the client failed to figure out that a suspend is scheduled, and (3) trade suspension may be followed by connectivity loss - then what?
• Unplanned loss of connectivity
  • client shuts down, server still up (what does server do while client is gone, and what do they both do on reconnect? Client resends certain requests or acks, even if the client sent them before going down? specifically, what? Wait for server to resend certain responses if relevant for active swaps?)
  • server shuts down, client still up (what does client do when server is gone, and what do they both do on reconnect? e.g. I've been dealing with this a lot in terms of the swap state, book restore, etc., but there is more to discuss.)
  • both client and server are still alive, just lost connection (there is no software that has a "startup recovery sequence", just a reconnect plan. should the server recognize mass outage? In general, I believe the client should bear most the burden of resuming swaps and checking on their orders, but certain server actions are still required. SendWhenConnected and RequestWhenConnected in the swap state PR touch on this )

[chappjc]

I'm still pondering if a final TradeSuspension message should also be sent from the server at the moment of market suspend, which would need handling client side of course, but I'm good with this as a start.

I would also like to call out the different scenarios mentioned in #269 (comment), namely what happens when a client connects when a suspension is already scheduled. Does the client create a pendingSuspension or flag a market as suspended based on the 'config' response? If not, it will need to.

[chappjc]

Once @buck54321's reviews are addressed and he approves, please rebase/resolve/squash.