Skip to content

Commit

Permalink
Editing
Browse files Browse the repository at this point in the history
  • Loading branch information
ZenGround0 committed Apr 22, 2021
1 parent 457df20 commit 8dc4db6
Showing 1 changed file with 51 additions and 25 deletions.
76 changes: 51 additions & 25 deletions FIPS/fip-0013.md
Expand Up @@ -27,11 +27,11 @@ is currently limiting network growth.
The miner `ProveCommitSector` method only supports committing a single sector at
a time. It is both frequently executed and expensive. This proposal adds a
way for miners to post multiple `ProveCommit`s at once in an aggregated fashion
using a `ProveCommitSectorAggregated`. This method amortize some of the costs
using a `ProveCommitSectorAggregated`. This method amortizes some of the costs
across multiple sectors, removes some redundant but costly checks and
drastically reduces per-sector proof size and verification times taking
advantage of a novel cryptography result. This proposal also increases the delay
between pre-commit an prove-commit to allow miners of all sizes to enjoy the
between pre-commit and prove-commit to allow miners of all sizes to enjoy the
highest gas saving factor.


Expand All @@ -51,8 +51,8 @@ commit. Verification time and size of an aggregated proof scales logarithmically
with the number of proofs being included.

In addition to this primary optimization, there are several secondary
opportunities for improved processing time and gas cost related to batch
processing many prove commits at once:
opportunities for improved processing time and gas cost related to amortizing state acceses costs through batch processing many prove commits at once:

- Using a bitfield to specify sector numbers in the `ProveCommitSector` parameters could reduce message size
- PreCommits loading overhead can be done once per batch
- Power actor claim validation can be done once per batch
Expand All @@ -61,10 +61,11 @@ processing many prove commits at once:
Additionally the `ProveCommitSectorAggregated` method can do away with the
temporary storage and cron-batching currently used to verify individual prove
commits. This opens further cost reduction opportunities:
- PreCommit info can be loaded once per prove commit rather than once for recording, and again for batch verifying in call back
- Sectors proven through `ProveCommitSectorAggregated` will not need to be stored and read from the power actors `ProofValidationBatch` map

If miner operators implemented a relatively short aggregation period (a day),
- PreCommit info can be loaded once per prove commit rather than once for recording, and again for batch verifying in the cron callback
- With no processing needed in the cron callback, sectors proven through `ProveCommitSectorAggregated` will not need to be stored and read from the power actor's `ProofValidationBatch` map

In summary if miner operators implement a relatively short aggregation period,
the `ProveCommitAggregated` method has the potential to reduce gas costs for:

- State operations: some of the costs listed above can be amortized across
Expand All @@ -90,17 +91,17 @@ type ProveCommitSectorAggregatedParams {

Semantics will be similar to those of `ProveCommitSector` with the following proposed changes:

- Sectornumber bitfield in place of a single abi.SectorNumber in parameters
- Aggregate proof in place of single porep proof in parameters
- Use a SectorNumber bitfield in place of a single abi.SectorNumber in parameters
- Include a Aggregate proof in place of single porep proof in parameters
- MaxProveCommitSize parameter will change
- Minimum and maximum number of sectors proven will be enforced
- PreCommitInfos read in batch
- SealVerifyInfos constructed in batch
- Market actor ComputeDataCommittment method changed to compute batches of commDs
- Gas cost for verification will be updated and now computed as a function of the number of sectors aggregated
- No storing proof info in power actor for batched verification at the end of the epoch.
- `ProveCommitSectorAggregated` will call into a new runtime syscall `AggregateVerifySeals` in place of power actor `BatchVerifySeals` call.
- ConfirmSectorProofsValid logic will be copied over to the second half of `ProveCommitSectorAggregated`.
- `ProveCommitSectorAggregated` will call into a new runtime syscall `AggregateVerifySeals` in place of power actor `BatchVerifySeals` call.
- ConfirmSectorProofsValid logic will be copied over to the second half of `ProveCommitSectorAggregated`.

#### Failure handling

Expand All @@ -109,11 +110,25 @@ Semantics will be similar to those of `ProveCommitSector` with the following pro

#### Scale and limits

The number of sectors that may be pre-committed in a single aggregation starts
from 50 (TODO) and a maximum of 203 (TODO)
The number of sectors that may be proven in a single aggregation
is a minimum of 12 and a maximum of 203

MaxProveCommitDuration, the enforced delay between pre-commit and prove-commit is also increased from 1 day + PreCommitChallengeDelay to 6 days + PreCommitChallengeDelay. This does not impact security of the system it only increases the possible time pre-commits remain on chain before expiry. See the section about incentives for motivation.

#### Gas calculations

Similar to existing PoRep gas charges, gas values will be determined from empirical measurements of aggregate proof validation on representative hardware. Each PoRep count corresponding to a power of two snark count will be assigned to a gas table. See the "Proof scheme changes" section for discussion on padding to the next power of two for motivation. The number of sectors proven in a call to `ProveCommitSectorAggregated` will be rounded up to the nearest number in the table to calculate the gas value. Currently the table looks like this (subject to change)

- 12 PoReps (128 snarks): 300M
- 13-25 PoReps (256 snarks): 300M
- 26-51 PoReps (512 snarks): 350M
- 52-102 PoReps (1024 snarks): 350M
- 103-204 PoReps (2048 snarks): 350M

#### State Migrations

Neither changes to the state schema of any actors nor changes to the fields of existing actors are required to make this change. Therefore a state migration is not needed.

The delay between precommit and provecommit is also increased up to 6 days. See
the section about incentives for more information.

### Proof scheme changes

Expand All @@ -133,13 +148,18 @@ A more detailed technical report on the new constructions can be found here (TOD

#### Proofs API

##### RegisteredAggregationProof

The API changes introduce a new type RegisteredAggregationProof for future proofing the aggregation scheme. As of this change there is only one valid member of the RegisteredAggregationProof type corresponding to the new aggregation scheme.

##### Aggregation

The proofs aggregation procedure expects the following inputs:

```rust
pub fn aggregate_seal_commit_proofs(
registered_proof: RegisteredSealProof,
registered_aggregation: RegisteredAggregationProof,
commit_outputs: &[SealCommitPhase2Output],
) -> Result<AggregateSnarkProof>;
```
Expand All @@ -160,6 +180,7 @@ The proofs verification procedure expects the following inputs:
```rust
pub fn verify_aggregate_seal_commit_proofs(
registered_proof: RegisteredSealProof,
registered_aggregation: RegisteredAggregationProof,
aggregated_proofs_len: usize,
aggregate_proof_bytes: AggregateSnarkProof,
commit_inputs: Vec<Vec<Fr>>,
Expand Down Expand Up @@ -248,7 +269,7 @@ smooth operation through the upgrade period.
### Failure handling

Aborting on any precondition failure is chosen for simplicity.
There is no good reason for submitting an invalid pre-commitment, so this should
Submitting an invalid prove commitment should
never happen for correctly-functioning miners. Aborting on failure will provide
a clear indication that something is wrong, which might be overlooked by an
operator otherwise.
Expand All @@ -259,10 +280,12 @@ This would complicate both the actor and node implementations somewhat, though n

### Scale and limits

The goal of the bound on aggregation size is two fold:
* to limit the impact of potentially mistaken or malicious behaviour.
* to gradually increase the scalability, to have time to observe how the network
Each aggregated proof is bounded at 203 sectors. The motivation for the bound on aggregation size is as follows:

- to limit the impact of potentially mistaken or malicious behaviour.
- to gradually increase the scalability, to have time to observe how the network
is growing with this new method.
- to ensure the gas savings are equally accessible to small miners with a sealing rate as low as 1TB/day

A miner may submit multiple batches in a single epoch to grow faster.

Expand Down Expand Up @@ -294,7 +317,7 @@ accompanied by formal security proofs in the same framework as the original
paper. The trusted setup used is the Filecoin Powers of Tau and the ZCash Powers
of Tau: the full key is generated thanks to this
[tool](https://github.com/nikkolasg/taupipp). The cryptographic implementation
is being audited and report will be made available soon.
is being audited and a report will be made available soon.

## Incentive Considerations

Expand All @@ -310,17 +333,18 @@ miner that can aggregate 800 proofs in a day will be able to enjoy a 20x gas
reduction (theoretical numbers) while a miner that can only aggregate 200 proofs
a day will enjoy "only" a 10x gas reduction. Note that the gas reduction here is
_not_ logarithmic because all the states operations performed in the
`ProveCommitSectorAggregated` do not yield a benefit when using them in batch.
`ProveCommitSectorAggregated` do not scale logarithmically with batch size.

In order to support all size of miners on the Filecoin network, this FIP
In order to support similar gas improvements for all sizes of miners on the Filecoin network, this FIP
proposes to:

1. Increase the maximum delay between pre-commit and prove-commit to 6 days
2. Limit the number of aggregated proofs to 203 (`=floor(2048/10)`)

This FIP consider the baseline of 1TB/day of sealing capacity, or 32 sectors,
for a "small" miner. By allowing a larger delay up to 6 days, such a miner can
aggregate up to 192 proofs in 6 days. Using this strategy, all miners can
benefit from a (theoretical) x11 gas reduction.
benefit from a (theoretical) 11x gas reduction.


## Product Considerations
Expand All @@ -329,13 +353,15 @@ This proposal reduces the aggregate cost of committing new sectors to the
Filecoin network.

This will reduce miner costs overall, as well as reduce contention for chain
transaction bandwidth that can crowd out other messages. It enables to have a
larger emboarding rate for the Filecoin network.
transaction bandwidth that can crowd out other messages. It unlocks a
larger storage onboarding rate for the Filecoin network.

## Implementation

* Cryptographic implementation is currently located on the `feat-ipp2` of the bellperson [repo](https://github.com/filecoin-project/bellperson/tree/feat-ipp2/src/groth16/aggregate)
* Integration between Lotus and crypto-land can be found in [rust-fil-proofs](https://github.com/filecoin-project/rust-fil-proofs/tree/feat-aggregation) and the FFI [here](https://github.com/filecoin-project/filecoin-ffi/blob/feat-aggregation).
* Actors changes are in progress here: https://github.com/filecoin-project/specs-actors/pull/1381
* Lotus integration putting everything together is in progress here: https://github.com/filecoin-project/lotus/pull/5769

## Copyright

Expand Down

0 comments on commit 8dc4db6

Please sign in to comment.