feat!: throttle with retries provider changes #1230

shaspitz · 2023-08-22T18:29:24Z

Description

Closes: Provider changes for throttle v2 #1102, part of EPIC: Improve throttling mechanism with retries #713
Updates/adds UTs and integration tests as needed to close 1102
Implicitly reverts fix: limit vsc matured packets handled per endblocker #1004
Depreciates provider's QueryThrottledConsumerPacketData query, an equivalent query will be added to consumer, tracked as Replace provider throttle state query with a consumer query #1233
Updates throttling related e2e tests to pass. Note retry logic is not fully e2e tested in this PR, and will be fully addressed, tracked by E2e tests for throttle v2 #1103
Migration is NOT addressed in this PR, see Throttling v2 provider migration #1032

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

confirmed the correct type prefix in the PR title
confirmed ! in the type prefix if API or client breaking change
confirmed this PR does not introduce changes requiring state migrations, OR confirmed migration code has been added to consumer and/or provider modules
confirmed all author checklist items have been addressed
reviewed state machine logic
reviewed API design and naming
reviewed documentation is accurate
reviewed tests and test coverage

shaspitz · 2023-08-24T18:05:57Z

tests/integration/stop_consumer.go

@@ -80,33 +79,6 @@ func (s *CCVTestSuite) TestStopConsumerChain() {
 				return nil
 			},
 		},
-		{
-			func(suite *CCVTestSuite) error {


Any test or test case that was removed, is a test that is only relevant to provider queuing logic which is removed in this PR

shaspitz · 2023-08-24T18:12:15Z

tests/integration/throttle_retry.go

-// via new result acks from the provider.
-//
-// TODO: This test will need updating once provider changes are made.
+// TestSlashRetries tests the throttling v2 retry logic at an integration level.
 func (s *CCVTestSuite) TestSlashRetries() {


This test has been adjusted to validate the full throttling v2 logic (provider/consumer both having v2 throttling), previously it just tested that a consumer running the v2 throttling protocol is compatible with a provider running the v1 throttling protocol

shaspitz · 2023-08-24T18:20:15Z

x/ccv/provider/keeper/relay.go

-	// This ensures the meter value is replenished, and not greater than the allowance (max value)
-	// for the block, before the throttling logic is executed.
+// BeginBlockCIS contains the BeginBlock logic needed for the Consumer Initiated Slashing sub-protocol.
+func (k Keeper) BeginBlockCIS(ctx sdk.Context) {


Slash meter replenishment now happens on beginblock, since we want to replenish the meter before slash packets are handled DURING the block. Whereas previously all slash packets were handled in endblocker after the replenishment call

shaspitz · 2023-08-24T18:21:27Z

x/ccv/provider/keeper/relay.go

-	// to enforce order of handling between the two packet data types.
-	if err := k.QueueThrottledSlashPacketData(ctx, chainID, packet.Sequence, data); err != nil {
-		return ccv.NewErrorAcknowledgementWithLog(ctx, fmt.Errorf("failed to queue slash packet data: %s", err.Error()))
+	meter := k.GetSlashMeter(ctx)


Meat of provider changes is here. If slash meter is negative, bounce packet, otherwise decrement meter and handle packet

This reverts commit a10a239.

shaspitz · 2023-09-06T14:20:57Z

do you know why the labels indicate the PR contains changes to the consumer?

Not sure, I don't see any changes to the consumer, it might have to do with merging main on this branch and how many times I had to change the base (feat) branch to reflect changes in main

…-changes" This reverts commit 6bdfff9, reversing changes made to d8f5690.

…ith-retries-provider-changes

jtremback

LGTM but I didn't really review the tests. Great work!

p-offtermatt

First part of review, more reviewing to come

p-offtermatt · 2023-09-13T15:04:38Z

tests/e2e/actions.go

 			break
 		}

 		if time.Now().After(timeout) {
-			panic(fmt.Sprintf("\n\n\nwaitForSlashThrottleDequeuemethod has timed out after: %s\n\n", action.timeout))
+			panic(fmt.Sprintf("\n\nwaitForSlashMeterReplenishment has timed out after: %s\n\n", action.timeout))
 		}

 		time.Sleep(500 * time.Millisecond)


Can we replace this with tr.waitBlocks/tr.waitTime?

I think it should be waitTime from my understanding of the slashMeter (it fills by timestamp, not by blockcount?)

Nice catch 53c02ea

tests/e2e/steps_downtime.go

p-offtermatt · 2023-09-13T15:11:22Z

tests/e2e/state.go

@@ -667,9 +654,10 @@ func (tr TestRun) getProviderAddressFromConsumer(consumerChain chainID, validato
 	return addr
 }

-func (tr TestRun) getGlobalSlashQueueSize() uint {
+func (tr TestRun) getSlashMeter() int64 {


Should this take the chain as an argument instead of always using provi?

Imo no because the slash meter is only relevant to the provider. If we ever plan to have multiple providers in these tests then that's a different story, but I don't see that happening

p-offtermatt · 2023-09-13T15:12:28Z

tests/e2e/steps_downtime.go

 						validatorID("carol"): 500,
 					},
 				},
 			},
 		},
+		// TODO(Shawn): Improve this test to have the consumer retry it's downtime slash, and to assert queue size on consumer.


Just to confirm, we are removing the Queue size from the chain state, despite this TODO, right? It reads to me as if that would reintroduce it, but maybe I'm missing context

p-offtermatt · 2023-09-13T15:13:23Z

tests/e2e/steps_downtime.go

-				// for meter to become positive again. 3*20 = 60 seconds + buffer = 80 seconds
-				timeout: 80 * time.Second,
+				// for meter to become positive again. 3*20 = 60 seconds + buffer = 100 seconds
+				timeout: 100 * time.Second,


Just curious, was there a reason the buffer needs to be increased?

I think this is an artifact of #1304 and trying to get this to work with cometmock

p-offtermatt

I added some more comments. Long list of removed code, nothing I was looked seriously problematic, so I'll approve. I mostly looked at the tests, but it was hard for me to tell whether something that should be tested is untested

x/ccv/provider/client/cli/query.go

p-offtermatt · 2023-09-13T15:43:53Z

x/ccv/provider/keeper/grpc_query.go

@@ -150,24 +149,6 @@ func (k Keeper) QueryThrottleState(goCtx context.Context, req *types.QueryThrott
 	candidate := k.GetSlashMeterReplenishTimeCandidate(ctx) // always UTC
 	packets := []*types.ThrottledSlashPacket{}


This serves no purpose anymore, right? I assume we are not changing the return type here to not break protobufs?

I just noticed we are already removing a message type, can't we adjust this type as well?

Yea my original intention here was to avoid breaking the query API. But you make a good point that this PR is already "query breaking", might as well make things clean -> 1ceddcb

To be consistent I should also remove this deprecated provider query too -> e228953 and 7d3dd64

p-offtermatt · 2023-09-13T15:46:55Z

x/ccv/provider/keeper/grpc_query.go


 	return &types.QueryThrottledConsumerPacketDataResponse{
 		ChainId:             req.ChainId,
-		Size_:               k.GetThrottledPacketDataSize(ctx, req.ChainId),
+		Size_:               0,


Same here, this type seems irrelevant now?

I removed this entire query

p-offtermatt · 2023-09-13T15:56:29Z

x/ccv/provider/keeper/proposal_test.go

@@ -525,28 +524,6 @@ func TestStopConsumerChain(t *testing.T) {
 			},
 			expErr: true,
 		},
-		{
-			description: "valid stop of consumer chain, throttle related queues are cleaned",


Should we really remove the whole test case here, not just the fact that it checks for cleaned queues? Not sure how easy this is to separate

I believe the entire intention of this test case was to confirm the provider queues are cleared. Since those queues no longer exist it seems neccessary to remove the whole test case

p-offtermatt · 2023-09-14T06:58:00Z

tests/e2e/actions.go

-	if initialGlobalQueueSize != action.currentQueueSize {
-		panic(fmt.Sprintf("wrong initial queue size: %d - expected global queue: %d\n", initialGlobalQueueSize, action.currentQueueSize))
+	if initialSlashMeter >= 0 {
+		panic(fmt.Sprintf("No need to wait for slash meter replenishment, current value: %d", initialSlashMeter))


Is this a bad idea to panic on? It makes the tests flakey, because if they happen to run slowly, the slash meter might be already-full, right?

Hmm I don't see a good way to decouple the logic of the slash meter away from time. Any test for throttling must have a pretty good and mostly deterministic control over chain time

Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com>

…#1321) * merge main into throttle feat branch (#1267) * build(deps): bump actions/checkout from 3 to 4 (#1257) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps)!: bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 (#1258) * build(deps): bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 Bumps [github.com/cosmos/ibc-go/v7](https://github.com/cosmos/ibc-go) from 7.2.0 to 7.3.0. - [Release notes](https://github.com/cosmos/ibc-go/releases) - [Changelog](https://github.com/cosmos/ibc-go/blob/main/CHANGELOG.md) - [Commits](cosmos/ibc-go@v7.2.0...v7.3.0) --- updated-dependencies: - dependency-name: github.com/cosmos/ibc-go/v7 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.4 to 0.47.5 (#1259) * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.3 to 0.47.5 Bumps [github.com/cosmos/cosmos-sdk](https://github.com/cosmos/cosmos-sdk) from 0.47.3 to 0.47.5. - [Release notes](https://github.com/cosmos/cosmos-sdk/releases) - [Changelog](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) - [Commits](cosmos/cosmos-sdk@v0.47.3...v0.47.5) --- updated-dependencies: - dependency-name: github.com/cosmos/cosmos-sdk dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * chore: Separate semver (#1217) separate semver * docs: cleanup changelog (#1260) fix changelog * fix!: validate MsgTransfer before calling Transfer() (#1244) * validate MsgTransfer * add TestSendRewardsToProvider * update DefaultConsumerUnbondingPeriod to 14 days * update changelog * fix linter * fix test * apply review suggestions * update changelog * docs: Create adr-012-separate-releasing.md (#1229) * Create adr-011-separate-releasing.md * Update docs/docs/adrs/adr-011-separate-releasing.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * adr 12 not 11 * correct that we use postfix not prefix * explanation on example release flow --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * fix: remove addr validation for provider fee pool addr param (#1262) * fix: remove validation for provider chain address since we cannot validate it properly on consumer * add changelog entry --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> * feat!: throttle with retries provider changes (#1230) * wip, tests not fixed yet * rm packet query on provider * rm unneeded UTs * rm tests from relay_test * rm query and more tests * rm more tests * builds again and rm debug tests * lint * fix handling of slash packet and integration test * Fix TestMultiConsumerSlashPacketThrottling * fix two more slashing integration tests * Update TestSlashRetries, cleanup neededc * cleaned up TestSlashRetries * UT for TestOnRecvDowntimeSlashPacket * cleans * use helper in throttle test * lintz * Revert "rm packet query on provider" This reverts commit a10a239. * cmd file too * fully restore query * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 73db33b, reversing changes made to 78a8269. * make e2e test pass, with todos * clean * Revert "Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes"" This reverts commit 5bfccc3. * lint * Update CHANGELOG.md * slightly longer buffer * build(deps): bump actions/checkout from 3 to 4 (#1257) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps)!: bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 (#1258) * build(deps): bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 Bumps [github.com/cosmos/ibc-go/v7](https://github.com/cosmos/ibc-go) from 7.2.0 to 7.3.0. - [Release notes](https://github.com/cosmos/ibc-go/releases) - [Changelog](https://github.com/cosmos/ibc-go/blob/main/CHANGELOG.md) - [Commits](cosmos/ibc-go@v7.2.0...v7.3.0) --- updated-dependencies: - dependency-name: github.com/cosmos/ibc-go/v7 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.4 to 0.47.5 (#1259) * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.3 to 0.47.5 Bumps [github.com/cosmos/cosmos-sdk](https://github.com/cosmos/cosmos-sdk) from 0.47.3 to 0.47.5. - [Release notes](https://github.com/cosmos/cosmos-sdk/releases) - [Changelog](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) - [Commits](cosmos/cosmos-sdk@v0.47.3...v0.47.5) --- updated-dependencies: - dependency-name: github.com/cosmos/cosmos-sdk dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * chore: Separate semver (#1217) separate semver * docs: cleanup changelog (#1260) fix changelog * fix!: validate MsgTransfer before calling Transfer() (#1244) * validate MsgTransfer * add TestSendRewardsToProvider * update DefaultConsumerUnbondingPeriod to 14 days * update changelog * fix linter * fix test * apply review suggestions * update changelog * docs: Create adr-012-separate-releasing.md (#1229) * Create adr-011-separate-releasing.md * Update docs/docs/adrs/adr-011-separate-releasing.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * adr 12 not 11 * correct that we use postfix not prefix * explanation on example release flow --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * fix: remove addr validation for provider fee pool addr param (#1262) * fix: remove validation for provider chain address since we cannot validate it properly on consumer * add changelog entry * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 6bdfff9, reversing changes made to d8f5690. * fmt * Update steps_downtime.go * Update tests/e2e/steps_downtime.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * waittime instead of sleep * Update x/ccv/provider/client/cli/query.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * Update x/ccv/provider/client/cli/query.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * rm throttled packets from query * rm provider query * whoopsies forgot to rm some boilerplate --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> * feat: consumer throttle state query (#1285) * wip, tests not fixed yet * rm packet query on provider * rm unneeded UTs * rm tests from relay_test * rm query and more tests * rm more tests * builds again and rm debug tests * lint * fix handling of slash packet and integration test * Fix TestMultiConsumerSlashPacketThrottling * fix two more slashing integration tests * Update TestSlashRetries, cleanup neededc * cleaned up TestSlashRetries * UT for TestOnRecvDowntimeSlashPacket * cleans * use helper in throttle test * lintz * Revert "rm packet query on provider" This reverts commit a10a239. * cmd file too * fully restore query * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 73db33b, reversing changes made to 78a8269. * make e2e test pass, with todos * clean * Revert "Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes"" This reverts commit 5bfccc3. * lint * Update CHANGELOG.md * slightly longer buffer * build(deps): bump actions/checkout from 3 to 4 (#1257) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps)!: bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 (#1258) * build(deps): bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 Bumps [github.com/cosmos/ibc-go/v7](https://github.com/cosmos/ibc-go) from 7.2.0 to 7.3.0. - [Release notes](https://github.com/cosmos/ibc-go/releases) - [Changelog](https://github.com/cosmos/ibc-go/blob/main/CHANGELOG.md) - [Commits](cosmos/ibc-go@v7.2.0...v7.3.0) --- updated-dependencies: - dependency-name: github.com/cosmos/ibc-go/v7 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.4 to 0.47.5 (#1259) * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.3 to 0.47.5 Bumps [github.com/cosmos/cosmos-sdk](https://github.com/cosmos/cosmos-sdk) from 0.47.3 to 0.47.5. - [Release notes](https://github.com/cosmos/cosmos-sdk/releases) - [Changelog](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) - [Commits](cosmos/cosmos-sdk@v0.47.3...v0.47.5) --- updated-dependencies: - dependency-name: github.com/cosmos/cosmos-sdk dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * chore: Separate semver (#1217) separate semver * docs: cleanup changelog (#1260) fix changelog * fix!: validate MsgTransfer before calling Transfer() (#1244) * validate MsgTransfer * add TestSendRewardsToProvider * update DefaultConsumerUnbondingPeriod to 14 days * update changelog * fix linter * fix test * apply review suggestions * update changelog * docs: Create adr-012-separate-releasing.md (#1229) * Create adr-011-separate-releasing.md * Update docs/docs/adrs/adr-011-separate-releasing.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * adr 12 not 11 * correct that we use postfix not prefix * explanation on example release flow --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * fix: remove addr validation for provider fee pool addr param (#1262) * fix: remove validation for provider chain address since we cannot validate it properly on consumer * add changelog entry * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 6bdfff9, reversing changes made to d8f5690. * fmt * protobuf gen * impl query resp * add query assertion to e2e tests * lint * add consumer to pending packets name * add panic for invalid bytes --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> * feat: retry delay period as param (#1308) * boilerplate * fix test * Update proposal_test.go * apply new param to keeper code * Update params_test.go * tests: throttle retry e2e tests (#1312) * boilerplate * fix test * Update proposal_test.go * apply new param to keeper code * wip * add action * incorporate queue assertions. Also make retry delay period 200s * correct queue size assertion * make wait time just 30 seconds * change naming * feat!: throttle v2 provider migration (#1317) * migration.go and incr consensus version * throttle v1 code for migration * old methods * finish test * register those migs * Update migration_test.go * lint * consumer keeper -> provider keeper * some fixes * fix the rest o the shit * lint * Addressing simon's comments * whoopies * height assertion * Update x/ccv/provider/keeper/migration.go Co-authored-by: Simon Noetzlin <simon.ntz@gmail.com> * action boilerplate * feat: remove max throttled packets param (#1332) * rm param * Update proto/interchain_security/ccv/provider/v1/provider.proto Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * merge fixes * update e2e jsons --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Shawn <44221603+smarshall-spitzbart@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> Co-authored-by: Simon Noetzlin <simon.ntz@gmail.com>

…#1321) * merge main into throttle feat branch (#1267) * build(deps): bump actions/checkout from 3 to 4 (#1257) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps)!: bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 (#1258) * build(deps): bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 Bumps [github.com/cosmos/ibc-go/v7](https://github.com/cosmos/ibc-go) from 7.2.0 to 7.3.0. - [Release notes](https://github.com/cosmos/ibc-go/releases) - [Changelog](https://github.com/cosmos/ibc-go/blob/main/CHANGELOG.md) - [Commits](cosmos/ibc-go@v7.2.0...v7.3.0) --- updated-dependencies: - dependency-name: github.com/cosmos/ibc-go/v7 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.4 to 0.47.5 (#1259) * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.3 to 0.47.5 Bumps [github.com/cosmos/cosmos-sdk](https://github.com/cosmos/cosmos-sdk) from 0.47.3 to 0.47.5. - [Release notes](https://github.com/cosmos/cosmos-sdk/releases) - [Changelog](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) - [Commits](cosmos/cosmos-sdk@v0.47.3...v0.47.5) --- updated-dependencies: - dependency-name: github.com/cosmos/cosmos-sdk dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * chore: Separate semver (#1217) separate semver * docs: cleanup changelog (#1260) fix changelog * fix!: validate MsgTransfer before calling Transfer() (#1244) * validate MsgTransfer * add TestSendRewardsToProvider * update DefaultConsumerUnbondingPeriod to 14 days * update changelog * fix linter * fix test * apply review suggestions * update changelog * docs: Create adr-012-separate-releasing.md (#1229) * Create adr-011-separate-releasing.md * Update docs/docs/adrs/adr-011-separate-releasing.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * adr 12 not 11 * correct that we use postfix not prefix * explanation on example release flow --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * fix: remove addr validation for provider fee pool addr param (#1262) * fix: remove validation for provider chain address since we cannot validate it properly on consumer * add changelog entry --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> * feat!: throttle with retries provider changes (#1230) * wip, tests not fixed yet * rm packet query on provider * rm unneeded UTs * rm tests from relay_test * rm query and more tests * rm more tests * builds again and rm debug tests * lint * fix handling of slash packet and integration test * Fix TestMultiConsumerSlashPacketThrottling * fix two more slashing integration tests * Update TestSlashRetries, cleanup neededc * cleaned up TestSlashRetries * UT for TestOnRecvDowntimeSlashPacket * cleans * use helper in throttle test * lintz * Revert "rm packet query on provider" This reverts commit a10a239. * cmd file too * fully restore query * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 73db33b, reversing changes made to 78a8269. * make e2e test pass, with todos * clean * Revert "Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes"" This reverts commit 5bfccc3. * lint * Update CHANGELOG.md * slightly longer buffer * build(deps): bump actions/checkout from 3 to 4 (#1257) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps)!: bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 (#1258) * build(deps): bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 Bumps [github.com/cosmos/ibc-go/v7](https://github.com/cosmos/ibc-go) from 7.2.0 to 7.3.0. - [Release notes](https://github.com/cosmos/ibc-go/releases) - [Changelog](https://github.com/cosmos/ibc-go/blob/main/CHANGELOG.md) - [Commits](cosmos/ibc-go@v7.2.0...v7.3.0) --- updated-dependencies: - dependency-name: github.com/cosmos/ibc-go/v7 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.4 to 0.47.5 (#1259) * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.3 to 0.47.5 Bumps [github.com/cosmos/cosmos-sdk](https://github.com/cosmos/cosmos-sdk) from 0.47.3 to 0.47.5. - [Release notes](https://github.com/cosmos/cosmos-sdk/releases) - [Changelog](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) - [Commits](cosmos/cosmos-sdk@v0.47.3...v0.47.5) --- updated-dependencies: - dependency-name: github.com/cosmos/cosmos-sdk dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * chore: Separate semver (#1217) separate semver * docs: cleanup changelog (#1260) fix changelog * fix!: validate MsgTransfer before calling Transfer() (#1244) * validate MsgTransfer * add TestSendRewardsToProvider * update DefaultConsumerUnbondingPeriod to 14 days * update changelog * fix linter * fix test * apply review suggestions * update changelog * docs: Create adr-012-separate-releasing.md (#1229) * Create adr-011-separate-releasing.md * Update docs/docs/adrs/adr-011-separate-releasing.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * adr 12 not 11 * correct that we use postfix not prefix * explanation on example release flow --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * fix: remove addr validation for provider fee pool addr param (#1262) * fix: remove validation for provider chain address since we cannot validate it properly on consumer * add changelog entry * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 6bdfff9, reversing changes made to d8f5690. * fmt * Update steps_downtime.go * Update tests/e2e/steps_downtime.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * waittime instead of sleep * Update x/ccv/provider/client/cli/query.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * Update x/ccv/provider/client/cli/query.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * rm throttled packets from query * rm provider query * whoopsies forgot to rm some boilerplate --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> * feat: consumer throttle state query (#1285) * wip, tests not fixed yet * rm packet query on provider * rm unneeded UTs * rm tests from relay_test * rm query and more tests * rm more tests * builds again and rm debug tests * lint * fix handling of slash packet and integration test * Fix TestMultiConsumerSlashPacketThrottling * fix two more slashing integration tests * Update TestSlashRetries, cleanup neededc * cleaned up TestSlashRetries * UT for TestOnRecvDowntimeSlashPacket * cleans * use helper in throttle test * lintz * Revert "rm packet query on provider" This reverts commit a10a239. * cmd file too * fully restore query * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 73db33b, reversing changes made to 78a8269. * make e2e test pass, with todos * clean * Revert "Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes"" This reverts commit 5bfccc3. * lint * Update CHANGELOG.md * slightly longer buffer * build(deps): bump actions/checkout from 3 to 4 (#1257) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps)!: bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 (#1258) * build(deps): bump github.com/cosmos/ibc-go/v7 from 7.2.0 to 7.3.0 Bumps [github.com/cosmos/ibc-go/v7](https://github.com/cosmos/ibc-go) from 7.2.0 to 7.3.0. - [Release notes](https://github.com/cosmos/ibc-go/releases) - [Changelog](https://github.com/cosmos/ibc-go/blob/main/CHANGELOG.md) - [Commits](cosmos/ibc-go@v7.2.0...v7.3.0) --- updated-dependencies: - dependency-name: github.com/cosmos/ibc-go/v7 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.4 to 0.47.5 (#1259) * build(deps): bump github.com/cosmos/cosmos-sdk from 0.47.3 to 0.47.5 Bumps [github.com/cosmos/cosmos-sdk](https://github.com/cosmos/cosmos-sdk) from 0.47.3 to 0.47.5. - [Release notes](https://github.com/cosmos/cosmos-sdk/releases) - [Changelog](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) - [Commits](cosmos/cosmos-sdk@v0.47.3...v0.47.5) --- updated-dependencies: - dependency-name: github.com/cosmos/cosmos-sdk dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * add changelog entries --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> * chore: Separate semver (#1217) separate semver * docs: cleanup changelog (#1260) fix changelog * fix!: validate MsgTransfer before calling Transfer() (#1244) * validate MsgTransfer * add TestSendRewardsToProvider * update DefaultConsumerUnbondingPeriod to 14 days * update changelog * fix linter * fix test * apply review suggestions * update changelog * docs: Create adr-012-separate-releasing.md (#1229) * Create adr-011-separate-releasing.md * Update docs/docs/adrs/adr-011-separate-releasing.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * adr 12 not 11 * correct that we use postfix not prefix * explanation on example release flow --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * fix: remove addr validation for provider fee pool addr param (#1262) * fix: remove validation for provider chain address since we cannot validate it properly on consumer * add changelog entry * Revert "Merge branch 'main' into shawn/throttle-with-retries-provider-changes" This reverts commit 6bdfff9, reversing changes made to d8f5690. * fmt * protobuf gen * impl query resp * add query assertion to e2e tests * lint * add consumer to pending packets name * add panic for invalid bytes --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> * feat: retry delay period as param (#1308) * boilerplate * fix test * Update proposal_test.go * apply new param to keeper code * Update params_test.go * tests: throttle retry e2e tests (#1312) * boilerplate * fix test * Update proposal_test.go * apply new param to keeper code * wip * add action * incorporate queue assertions. Also make retry delay period 200s * correct queue size assertion * make wait time just 30 seconds * change naming * feat!: throttle v2 provider migration (#1317) * migration.go and incr consensus version * throttle v1 code for migration * old methods * finish test * register those migs * Update migration_test.go * lint * consumer keeper -> provider keeper * some fixes * fix the rest o the shit * lint * Addressing simon's comments * whoopies * height assertion * Update x/ccv/provider/keeper/migration.go Co-authored-by: Simon Noetzlin <simon.ntz@gmail.com> * action boilerplate * feat: remove max throttled packets param (#1332) * rm param * Update proto/interchain_security/ccv/provider/v1/provider.proto Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> --------- Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * merge fixes * update e2e jsons --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Shawn <44221603+smarshall-spitzbart@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mpoke <marius.poke@posteo.de> Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> Co-authored-by: Dmitry Kolupaev <dmitry.klpv@gmail.com> Co-authored-by: Simon Noetzlin <simon.ntz@gmail.com> (cherry picked from commit 88499b7) # Conflicts: # CHANGELOG.md # tests/e2e/action_rapid_test.go # tests/e2e/actions.go # tests/e2e/json_utils.go # tests/e2e/main.go # tests/e2e/state.go # tests/e2e/state_rapid_test.go # tests/e2e/steps_downtime.go # tests/e2e/tracehandler_testdata/changeover.json # tests/e2e/tracehandler_testdata/democracy.json # tests/e2e/tracehandler_testdata/democracyRewardsSteps.json # tests/e2e/tracehandler_testdata/happyPath.json # tests/e2e/tracehandler_testdata/multipleConsumers.json # tests/e2e/tracehandler_testdata/shorthappy.json # tests/e2e/tracehandler_testdata/slashThrottle.json # tests/integration/common.go # tests/integration/slashing.go # tests/integration/throttle.go # tests/integration/throttle_retry.go # x/ccv/consumer/types/query.pb.go # x/ccv/provider/types/provider.pb.go # x/ccv/provider/types/query.pb.go # x/ccv/types/params.go # x/ccv/types/shared_consumer.pb.go

shaspitz added 6 commits August 21, 2023 14:21

wip, tests not fixed yet

7691bf5

rm packet query on provider

a10a239

rm unneeded UTs

8a557b3

rm tests from relay_test

196ce38

rm query and more tests

0f27c31

rm more tests

cf09f5f

github-actions bot added C:Testing Assigned automatically by the PR labeler C:x/provider Assigned automatically by the PR labeler labels Aug 22, 2023

shaspitz added 10 commits August 22, 2023 11:31

Merge branch 'main' into shawn/throttle-with-retries-provider-changes

87ad0f4

builds again and rm debug tests

7e6264f

lint

5e4b845

fix handling of slash packet and integration test

8350956

Fix TestMultiConsumerSlashPacketThrottling

6d20dd1

fix two more slashing integration tests

461878c

Update TestSlashRetries, cleanup neededc

56242a6

cleaned up TestSlashRetries

1d963fa

UT for TestOnRecvDowntimeSlashPacket

e8acd9e

cleans

8ed33f3

shaspitz commented Aug 24, 2023

View reviewed changes

use helper in throttle test

ecac6a4

shaspitz commented Aug 24, 2023

View reviewed changes

shaspitz added 5 commits August 24, 2023 11:26

lintz

f6d4650

Revert "rm packet query on provider"

956e595

This reverts commit a10a239.

cmd file too

db8dc1b

fully restore query

78a8269

Merge branch 'main' into shawn/throttle-with-retries-provider-changes

73db33b

github-actions bot added C:x/consumer Assigned automatically by the PR labeler C:x/types Assigned automatically by the PR labeler labels Aug 24, 2023

Merge branch 'main' into shawn/throttle-with-retries-provider-changes

6bdfff9

github-actions bot added the C:CI Assigned automatically by the PR labeler label Sep 6, 2023

shaspitz added 2 commits September 6, 2023 12:23

Revert "Merge branch 'main' into shawn/throttle-with-retries-provider…

3b27006

…-changes" This reverts commit 6bdfff9, reversing changes made to d8f5690.

Merge branch 'feat/throttle-complete-attempt-2' into shawn/throttle-w…

afa32f4

…ith-retries-provider-changes

github-actions bot removed C:x/consumer Assigned automatically by the PR labeler C:x/types Assigned automatically by the PR labeler C:CI Assigned automatically by the PR labeler C:Docs Assigned automatically by the PR labeler C:ADR Assigned automatically by the PR labeler labels Sep 6, 2023

fmt

6ee88e2

jtremback approved these changes Sep 7, 2023

View reviewed changes

p-offtermatt reviewed Sep 13, 2023

View reviewed changes

p-offtermatt approved these changes Sep 13, 2023

View reviewed changes

p-offtermatt reviewed Sep 14, 2023

View reviewed changes

Update steps_downtime.go

1ed2f56

shaspitz mentioned this pull request Sep 14, 2023

comet/cometmock discrepancy w.r.t downtime related e2e tests #1304

Closed

shaspitz and others added 7 commits September 14, 2023 10:47

Update tests/e2e/steps_downtime.go

fcf0b92

Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com>

waittime instead of sleep

53c02ea

Update x/ccv/provider/client/cli/query.go

e9d745d

Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com>

Update x/ccv/provider/client/cli/query.go

b685958

Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com>

rm throttled packets from query

1ceddcb

rm provider query

e228953

whoopsies forgot to rm some boilerplate

7d3dd64

shaspitz merged commit 727e731 into feat/throttle-complete-attempt-2 Sep 14, 2023
13 of 14 checks passed

shaspitz deleted the shawn/throttle-with-retries-provider-changes branch September 14, 2023 19:20

This was referenced Sep 14, 2023

Provider changes for throttle v2 #1102

Closed

Replace provider throttle state query with a consumer query #1233

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: throttle with retries provider changes #1230

feat!: throttle with retries provider changes #1230

shaspitz commented Aug 22, 2023 •

edited

Loading

shaspitz Aug 24, 2023

shaspitz Aug 24, 2023

shaspitz Aug 24, 2023

shaspitz Aug 24, 2023

shaspitz commented Sep 6, 2023

jtremback left a comment

p-offtermatt left a comment

p-offtermatt Sep 13, 2023

p-offtermatt Sep 13, 2023

shaspitz Sep 14, 2023

p-offtermatt Sep 13, 2023

shaspitz Sep 14, 2023

p-offtermatt Sep 13, 2023

p-offtermatt Sep 13, 2023

shaspitz Sep 14, 2023

p-offtermatt left a comment •

edited

Loading

p-offtermatt Sep 13, 2023

p-offtermatt Sep 13, 2023

shaspitz Sep 14, 2023

shaspitz Sep 14, 2023 •

edited

Loading

p-offtermatt Sep 13, 2023

shaspitz Sep 14, 2023

p-offtermatt Sep 13, 2023

shaspitz Sep 14, 2023 •

edited

Loading

p-offtermatt Sep 14, 2023

shaspitz Sep 14, 2023

		@@ -150,24 +149,6 @@ func (k Keeper) QueryThrottleState(goCtx context.Context, req *types.QueryThrott
		candidate := k.GetSlashMeterReplenishTimeCandidate(ctx) // always UTC
		packets := []*types.ThrottledSlashPacket{}

feat!: throttle with retries provider changes #1230

feat!: throttle with retries provider changes #1230

Conversation

shaspitz commented Aug 22, 2023 • edited Loading

Description

Author Checklist

Reviewers Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shaspitz commented Sep 6, 2023

jtremback left a comment

Choose a reason for hiding this comment

p-offtermatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

p-offtermatt left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shaspitz Sep 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shaspitz Sep 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shaspitz commented Aug 22, 2023 •

edited

Loading

p-offtermatt left a comment •

edited

Loading

shaspitz Sep 14, 2023 •

edited

Loading

shaspitz Sep 14, 2023 •

edited

Loading