Build a WindowPoSt disputer

[arajasek]

Motivation

FIP-0010 necessitates the creation of "auditors" that try to disprove recent WindowPoSt submissions. We need this to be something easily launched, and so it makes sense to build it within a Lotus node.

Design considerations

There are 2 chief designs we could go with:

• Design 1: Periodically audit every miner's PoSt submissions (that are in the OptimisticPoStSubmissionsSnapshot of their deadlines). This is fairly comprehensive, but could be slow unless we have high efficiency (never re-audit a post). We could also modify this to be random, auditing some configurable number (or percentage) of miners in every period.

• Design 2: Subscribe to updates to the head, and listen for WindowPoSt submissions whose windows have closed (and are therefore challenge-able). Try to dispute every single one.

Implementation

This is a first draft that implements Design 1.
Testing

Based on a quick simulation in devnets, it achieves the two key requirements:

• audits every PoSt
• sends a dispute message for bad posts, ignores good ones.

Next big thing to test will be performance.

TODOs

• give it a spin on mainnet to establish a lower bound for performance
• some refactoring / code beautification is possible

Fixes #5285

[magik6k]

Maybe, but reorgs are annoying to handle correctly

[arajasek]

eh, maybe? the cache would basically be remembering "this ID address corresponds / doesn't correspond to a miner actor at a depth of at least 60 epochs"...probably okay to just ignore reorgs?

[arajasek]

Ran this on mainnet, and it seems pretty okay. Building the deadlineMap doesn't seem to take too long.

That's excluding the majority of the work, which is simulating the Dispute calls, though.

[Stebalien]

The approach looks like a good MVP. It needs some tests but nothing's blocking merging.

Note: this is going to be a pretty inefficient and very slow approach. I think it'll be able to keep up with mainnet, but it's not taking advantage of any of the things we can do off-chain.

The next steps are to:

1. Parallelize this. Maybe call the gas estimator instead of actually trying to submit messages so we can try multiple in parallel?
2. Ideally, check before actually trying to submit. We can play a lot of optimization games if we can skip the VM.

[Stebalien]

nit: log something if it actually works.

[Stebalien]

Actually, I don't think this works. It'll fail with an error if we fail to dispute (returning and skipping everything else). We need to inspect the error).

[Stebalien]

NG Design

1. Watch the chain for deadline changes, extracting optimistic proofs and the associated snapshotted partitions (specifically, compute the sectors that should have been proven). Then send this back to a worker queue. Do all this in parallel.
2. In the worker queue, resolve the sector numbers to piece CIDs, caching the results (we can probably just keep this cache in memory).
3. Verify proofs on multiple workers.
4. Later: use batch verification assuming that most of the proofs are correct.

The key part here is parallelism and caching. The parallelism will git us an NCORE speedup and the caching/parallel disk access will cut the time in ~half for large partitions.

[magik6k]

Could be run in parallel (but doesn't really need to be done now)

[Stebalien]

Actually, I don't think this works. It'll fail with an error if we fail to dispute (returning and skipping everything else). We need to inspect the error).

[magik6k]

Just a couple of nits, nothing really blocking

[jennijuju]

Run a Disputer

After your node is fully synced, you can run lotus chain disputer with the following options:
--max-fee is the max amount in FIL that you are willing to spend for a DisputeWindowedPoSt message. Keep the amount DisputeReward in mind when you set this fee.
--from is the account you want to send the messages from.

Note: A DisputeWindowedPoSt costs ~300M in gas.

Start a Window PoSt Disputer

Simply run lotus chain disputer start to start a disputer. It will start dispute proof since the epoch that your disputer is started unless you specify a --start-epoch.

Manual Dispute

You can also send a specific DisputeWindowedPoSt message by running .lotus chain disputer dispute [minerAddress] [index] [postIndex], where:
minerAddress is the miner id that submitted the proof you want to dispute. The same address is also the recipient of the message.
index is the deadline index of the proof you want to dispute for the miner, it should be in [0, 47].
postIndex is the post snapshot index, see the definition here.