Throttling v2 provider migration

[shaspitz]

Note this issue is only relevant to the provider!

Once #713 is fully implemented, we'll need to decide how we deal with any queued slash packets that exist at the time of the upgrade in functionality. Note that the throttling mechanism/behavior will be changed for both the provider and all consumers, see relevant ADR for details.

As I see it, we have two options

Option 1 - Handle or drop queued slash packets at upgrade time

We could handle all slash packets that are queued up (if any) at upgrade time, this route is slightly more risky, as it could enable the jailing attack that we've spent so much time preventing. However, this route is the most simple, and as long as we trust the consumers enough that exist at upgrade time, this route wouldn't be a problem.

We could also just drop all the queued slash packets that exist at upgrade time, therefore the jailing attack would not be possible. Worst case scenario here is that some validators who were down near upgrade time would not be punished. I think this decision is the way to go.

Note that it's most likely there won't be any queued slash packets at upgrade time, as throttling is not engaged during nominal CCV operations. We're considering worst case scenarios here.

Option 2 - Multi stage upgrade process

We could upgrade the consumers and provider in a clever way to gracefully change the throttling functionality.

First all consumers would need to upgrade to enable their own slash packet queueing/bouncing/blocking behavior. This is described in more detail in ADR.

Next, the provider upgrade would look like:

• Provider keeps old throttle functionality long enough to process all queued slash packets. During this time, ALL new slash packets are bounced.
• Once the slash packet queues are emptied, the provider is toggled over to the new functionality where slash packets are bounced if neccessary.
• A future provider upgrade could remove all the legacy code that won't be used again.

Downsides to this route are that it'd be really annoying to coordinate, especially without go.mod split. Also, we have to write more code to deal with this toggling behavior. This route is not my preference.

[jtremback]

We could also just drop all the queued slash packets that exist at upgrade time, therefore the jailing attack would not be possible. Worst case scenario here is that some validators who were down near upgrade time would not be punished. I think this decision is the way to go.

Yes, this is the way to go

[shaspitz]

Changing the name to reflect that this will just involve a provider migration

[shaspitz]

Closed by #1317