Hidden state in SIMD-0232 and SIMD-0249

[ripatel-fd]

Summary

SIMD-0232 and SIMD-0249 introduce new hidden runtime ("Bank") state.
This is undesirable because such state cannot be easily inspected by users, and cannot be verified when loaded from a snapshot.

Background

• SIMD-0249 "Delay Commission Updates" delays when "commission" ratio changes take effect
• SIMD-0232 "Custom Commission Collector Account" allows vote accounts to send inflation and fee block rewards to custom addresses ("collectors"). Like SIMD-0249, collector changes do not immediately take effect.

The relevant part is the "delay of changes", i.e. rug protection.
This is motivated by malicious validators that bait users to stake them through very low commission settings.
Once they gained sufficient stake (locked for an epoch), they can then immediately change commission/collector to steal the stakers' rewards.

Problem

Both SIMDs specify logic along the lines of "go back in time and look at the vote account state as of the last epoch".
Validator clients discard old account revisions after blocks are rooted by consensus. Therefore, validator logic instead adds hidden caches outside of the account database.

These caches are historically error-prone (difficult to test/fuzz) and hard to inspect.

Users do not have an RPC API to see the effective current commission and collectors of a vote account.
They only see the values scheduled for a future activation.

Since this hidden state affects consensus, it must be packed into a snapshot, thus requires snapshot format changes.
Since the cached data is not directly part of the snapshot slot's bank hash, it is impractical to cryptographically verify whether a snapshot contained the correct commission/collector values. (A malicious snapshot could exploit this to force a bank hash mismatch at a chosen slot in the future.)

Original conversation in Discord: https://discord.com/channels/428295358100013066/439194979856809985/1527404830992367849
(Please keep the conversation here instead)

Ideas

Some ideas that were discussed in Discord in the following.

1. Activate SIMD-0232 and SIMD-0249 as-is

Accept the tech debt of runtime-enshrined rug protection as accepted risk and deal with it later.
(Possibly by exposing this hidden state through sysvars, which also solves the snapshot verification issue.)

2. Postpone rug protection

Abandon SIMD-0232 and SIMD-0249 and rework them.

Allow immediate commission and collector changes (stakers disappointed).

3. On-chain rug protection

Abandon SIMD-0232 and SIMD-0249 and rework them.

Move rug protection into program logic which likely requires a new vote state format and migration.

[topointon-jump]

Abandon SIMD-0232 and SIMD-0249 and rework them.

SIMD-0249 is already active on mainnet btw. But we don't have to make the problem worse with SIMD-0232, and we can always undo SIMD-0249.

[topointon-jump]

My vote is we should allow immediate commission and collector changes (2), and then add (3) as a follow-up if we decide it's necessary.

[jstarry]

Yes, this has come up in the past and I agree it's not ideal to dig ourselves deeper into this hole of relying on past epoch stakes. That being said, we already rely on past epoch stakes values for resolving both the epoch stake sysvar and the clock sysvar so the rug protection doesn't make the situation materially worse. I think it's best to ship the two SIMDs as written and at the same time start working on a new SIMD which fully addresses reliance on snapshot for bank hash calculation.

[joncinque]

I thought about this a bit more, and since SIMD-0249 has already been enabled on mainnet, that means we currently use the cached vote accounts to resolve the commission rates. Using the cached accounts for the commission collectors doesn't make things significantly worse in my opinion.

Since SIMD-0232 was already accepted and implemented in Agave as written, I agree with @jstarry -- let's enable SIMD-0232 as is, and put in separate SIMDs to stop using the cached values. The commission collector will be an extremely easy one, but the commission value will require a new vote program /state. I'm happy to draft both follow-up SIMDs.

[sam0x17]

The follow-up SIMD for the snapshot reliance may already largely exist: SIMD-0511 stores the epoch stakes in an on-chain account per epoch, keyed by epoch number, owned by a native program that rejects all instructions. Since the account is under the bank hash, a snapshot with tampered values fails at load instead of silently diverging later.

Following @ripatel-fd's comment over on that discussion, the revised schema now carries the full vote account v4 income state as of the epoch boundary per vote account: inflation_rewards_collector, block_revenue_collector, and both commission rates in bps. That's exactly the state SIMD-0232 and SIMD-0249 read from the caches discussed here, so 0511 would give it a verifiable on-chain home rather than leaving it implicit in the manifest.

@joncinque before you draft something new, might be worth checking whether 0511 as revised already covers what you had in mind. If fields are missing, now is the time to add them, it's cheap before consumers start reading the accounts and expensive after.

[umar-bhatty]

Related to the follow-up discussed here: I left a consumer-side review on #676 (#676 (comment)) covering which rewards epoch each entry prices, a minimum retention window, and a possible interim vote-account field for the effective commission.

[joncinque]

I would like to move forward with writing a SIMD for using the live collector address values. That won't completely remove the cache, but it will remove some of their usage, and doesn't have any dependency on adding the epoch-stakes account.

The only edge case is closed vote accounts that still get leader slots, but we can just burn lamports meant for collectors on a closed vote account.

Any thoughts before I move forward?