Skip to content

Commit

Permalink
fix: systems done
Browse files Browse the repository at this point in the history
  • Loading branch information
hugomrdias committed Jul 14, 2020
1 parent 0152b43 commit 28574ec
Show file tree
Hide file tree
Showing 21 changed files with 81 additions and 119 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: Genesis
description: Genesis - starting the Blockchain
weight: 6
draft: true
---

# Genesis - starting the Blockchain
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ of the altered protocol.

### Progressive Block Validation

- [Blocks](\missing-link) may be validated in progressive stages, in order to minimize resource expenditure.
- [Blocks](block) may be validated in progressive stages, in order to minimize resource expenditure.
- Validation computation is considerable, and a serious DOS attack vector.
- Secure implementations must carefully schedule validation and minimize the work done by pruning blocks without validating them fully.
- `ChainSync` SHOULD keep a cache of unvalidated blocks (ideally sorted by likelihood of belonging to the chain), and delete unvalidated blocks when they are passed by `FinalityTipset`, or when `ChainSync` is under significant resource load.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ Clients that use a message pool include:

The message pool subsystem is made of two components:

- The [Message Syncer](\missing-link) -- which receives and propagates messages.
- [Message Storage](\missing-link) -- which caches messages according to a given policy.
- The [Message Syncer](message_syncer.md) -- which receives and propagates messages.
- [Message Storage](message_storage.md) -- which caches messages according to a given policy.

{{< hint warning >}}
TODOs:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ TODO:

## Message Propagation

Messages are propagated over the libp2p pubsub channel `/fil/messages`. On this channel, every serialised `SignedMessage` is announced (see [Message](\missing-link)).
Messages are propagated over the libp2p pubsub channel `/fil/messages`. On this channel, every serialised `SignedMessage` is announced (see [Message](message)).

Upon receiving the message, its validity must be checked: the signature must be valid, and the account in question must have enough funds to cover the actions specified. If the message is not valid it should be dropped and must not be forwarded.

{{<hint warning>}}
**TODO:** discuss checking signatures and account balances, some tricky bits that need consideration. Does the fund check cause improper dropping? E.g. I have a message sending funds then use the newly constructed account to send funds, as long as the previous wasn't executed the second will be considered "invalid" ... though it won't be at the time of execution.
{{</hint>}}
Original file line number Diff line number Diff line change
Expand Up @@ -7,36 +7,36 @@ bookCollapseSection: true
# Storage Power Consensus
---

The Storage Power Consensus subsystem is the main interface which enables Filecoin nodes to agree on the state of the system. SPC accounts for individual storage miners' effective power over consensus in given chains in its [Power Table](\missing-link). It also runs [Expected Consensus](\missing-link) (the underlying consensus algorithm in use by Filecoin), enabling storage miners to run leader election and generate new blocks updating the state of the Filecoin system.
The Storage Power Consensus subsystem is the main interface which enables Filecoin nodes to agree on the state of the system. SPC accounts for individual storage miners' effective power over consensus in given chains in its [Power Table](storage_power_actor#the-power-table). It also runs [Expected Consensus](expected_consensus) (the underlying consensus algorithm in use by Filecoin), enabling storage miners to run leader election and generate new blocks updating the state of the Filecoin system.

Succinctly, the SPC subsystem offers the following services:

- Access to the [Power Table](\missing-link) for every subchain, accounting for individual storage miner power and total power on-chain.
- Access to [Expected Consensus](\missing-link) for individual storage miners, enabling:
- Access to the [Power Table](storage_power_actor#the-power-table) for every subchain, accounting for individual storage miner power and total power on-chain.
- Access to [Expected Consensus](expected_consensus) for individual storage miners, enabling:

- Access to verifiable randomness [Tickets](\missing-link) as needed in the rest of the protocol.
- Running [Leader Election](\missing-link) to produce new blocks.
- Running [Chain Selection](\missing-link) across subchains using EC's weighting function.
- Identification of [the most recently finalized tipset](\link-to-finality), for use by all protocol participants.
- Access to verifiable randomness [Tickets](storage_power_consensus#tickets) as needed in the rest of the protocol.
- Running [Leader Election](expected_consensus#secret-leader-election) to produce new blocks.
- Running [Chain Selection](chainsync) across subchains using EC's weighting function.
- Identification of [the most recently finalized tipset](expected_consensus#finality-in-ec), for use by all protocol participants.

Much of the Storage Power Consensus' subsystem functionality is detailed in the code below but we touch upon some of its behaviors in more detail.

{{< embed src="storage_power_consensus_subsystem.id" lang="go" >}}
{{<embed src="storage_power_consensus_subsystem.id" lang="go">}}

## Distinguishing between storage miners and block miners

There are two ways to earn Filecoin tokens in the Filecoin network:

- By participating in the [Storage Market](\missing-link) as a storage provider and being paid by clients for file storage deals.
- By participating in the [Storage Market](storage_market) as a storage provider and being paid by clients for file storage deals.
- By mining new blocks on the network, helping modify system state and secure the Filecoin consensus mechanism.

We must distinguish between both types of "miners" (storage and block miners). [Leader Election](\missing-link) in Filecoin is predicated on a miner's storage power. Thus, while all block miners will be storage miners, the reverse is not necessarily true.
We must distinguish between both types of "miners" (storage and block miners). [Leader Election](expected_consensus#secret-leader-election) in Filecoin is predicated on a miner's storage power. Thus, while all block miners will be storage miners, the reverse is not necessarily true.

However, given Filecoin's "useful Proof-of-Work" is achieved through file storage (PoRep and PoSt), there is little overhead cost for storage miners to participate in leader election. Such a [Storage Miner Actor](\missing-link) need only register with the [Storage Power Actor](\missing-link) in order to participate in Expected Consensus and mine blocks.
However, given Filecoin's "useful Proof-of-Work" is achieved through file storage (PoRep and PoSt), there is little overhead cost for storage miners to participate in leader election. Such a [Storage Miner Actor](storage_miner_actor) need only register with the [Storage Power Actor](storage_power_actor) in order to participate in Expected Consensus and mine blocks.

## On Power

Claimed power is assigned to every sector as a static function of its `SectorStorageWeightDesc` which includes `SectorSize`, `Duration`, and `DealWeight`. DealWeight is a measure that maps size and duration of active deals in a sector during its lifetime to its impact on power and reward distribution. A CommittedCapacity Sector (see Sector Types in [Storage Mining Subsystem](\missing-link)) will have a DealWeight of zero but all sectors have an explicit Duration which is defined from the ChainEpoch that the sector comes online in a ProveCommit message to the Expiration ChainEpoch of the sector. In principle, power is the number of votes a miner has in leader election and it is a point in time concept of storage. However, the exact function that maps `SectorStorageWeightDesc` to claimed `StoragePower` and `BlockReward` will be announced soon.
Claimed power is assigned to every sector as a static function of its `SectorStorageWeightDesc` which includes `SectorSize`, `Duration`, and `DealWeight`. DealWeight is a measure that maps size and duration of active deals in a sector during its lifetime to its impact on power and reward distribution. A CommittedCapacity Sector (see Sector Types in [Storage Mining Subsystem](storage_mining)) will have a DealWeight of zero but all sectors have an explicit Duration which is defined from the ChainEpoch that the sector comes online in a ProveCommit message to the Expiration ChainEpoch of the sector. In principle, power is the number of votes a miner has in leader election and it is a point in time concept of storage. However, the exact function that maps `SectorStorageWeightDesc` to claimed `StoragePower` and `BlockReward` will be announced soon.

More precisely,

Expand All @@ -48,16 +48,16 @@ More precisely,

Tickets are used across the Filecoin protocol as sources of randomness:

- The [Sector Sealers](\missing-link) uses tickets as SealSeeds to bind sector commitments to a given subchain.
- The [Storage Miner](\missing-link) likewise uses tickets as PoStChallenges to prove sectors remain committed as of a given block.
- They are drawn by the Storage Power subsystem as randomness in [Leader Election](\missing-link) to determine their eligibility to mine a block
- The [Sector Sealers](sealing) uses tickets as SealSeeds to bind sector commitments to a given subchain.
- The [Storage Miner](storage_mining) likewise uses tickets as PoStChallenges to prove sectors remain committed as of a given block.
- They are drawn by the Storage Power subsystem as randomness in [Leader Election](election_post) to determine their eligibility to mine a block
- They are drawn by the Storage Power subsystem in order to generate new tickets for future use.

Each of these ticket uses may require drawing tickets at different chain epochs, according to the security requirements of the particular protocol making use of tickets. Specifically, the ticket Digest (which is a Blake2b output) is used for randomness.

In Filecoin, every block header contains a single ticket.

You can find the Ticket data structure [here](\link-to-data-structures).
You can find the Ticket data structure [here](data_structures).

### Comparing Tickets in a Tipset and using ticket values

Expand All @@ -67,10 +67,10 @@ Likewise any use of a ticket's value, is that of its VRFDigest's bytes.

## The Ticket chain and drawing randomness

While each Filecoin block header contains a ticket field (see [Tickets](\missing-link)), it is useful to think of a ticket chain abstraction.
While each Filecoin block header contains a ticket field (see [Tickets](storage_power_consensus#tickets)), it is useful to think of a ticket chain abstraction.
Due to the nature of Filecoin's Tipsets and the possibility of using tickets from epochs that did not yield leaders to produce randomness at a given epoch, tracking the canonical ticket of a subchain at a given height can be arduous to reason about in terms of blocks. To that end, it is helpful to create a ticket chain abstraction made up of only those tickets to be used for randomness generation at a given height.

To read more about specifically how tickets are processed for randomness, see [Randomness](\missing-link).
To read more about specifically how tickets are processed for randomness, see [Randomness](randomness).

### Randomness Ticket generation

Expand All @@ -88,7 +88,7 @@ newRandomness = VRF_miner(H(TicketProdDST || index || Serialization(pastTicket,

The VRF's deterministic output adds entropy to the ticket chain, limiting a miner's ability to alter one block to influence a future ticket (given a miner does not know who will win a given round in advance).

We use the [Verifiable Random Functions](\missing-link) for ticket generation in EC (see the `PrepareNewTicket` method below).
We use the [Verifiable Random Functions](vrf) for ticket generation in EC (see the `PrepareNewTicket` method below).

{{< embed src="../../filecoin_mining/storage_mining/storage_mining_subsystem.go" lang="go" >}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,9 @@ Consensus in Filecoin is secured in part by economic incentives enforced by Pled

Pledge collateral amount is committed based on power pledged to the system (i.e. proportional to number of sectors committed and sector size for a miner). It is a system-wide parameter and is committed to the `StoragePowerActor`. Pledge collateral can be posted by the `StorageMinerActor` at any time by a miner and its requirement is dependent on miner's power. Details around pledge collateral will be announced soon.

Pledge Collateral will be slashed when [Consensus Faults](\missing-link) are reported to the `StoragePowerActor`'s `ReportConsensusFault` method, when a miner fails a SurprisePoSt (DetectedFault), or when a miner terminates a sector earlier than its duration.
Pledge Collateral will be slashed when [Consensus Faults](expected_consensus#consensus-faults) are reported to the `StoragePowerActor`'s `ReportConsensusFault` method, when a miner fails a SurprisePoSt (DetectedFault), or when a miner terminates a sector earlier than its duration.

Pledge Collateral is slashed for any fault affecting storage-power consensus, these include:

- faults to expected consensus in particular (see [Consensus Faults](\missing-link)) which will be reported by a slasher to the `StoragePowerActor` in exchange for a reward.
- faults affecting consensus power more generally, specifically uncommitted power faults (i.e. [Storage Faults](\missing-links)) which will be reported by the `CronActor` automatically or when a miner terminates a sector earlier than its promised duration.
- faults to expected consensus in particular (see [Consensus Faults](expected_consensus#consensus-faults)) which will be reported by a slasher to the `StoragePowerActor` in exchange for a reward.
- faults affecting consensus power more generally, specifically uncommitted power faults (i.e. [Storage Faults](faults#storage-faults)) which will be reported by the `CronActor` automatically or when a miner terminates a sector earlier than its promised duration.
12 changes: 5 additions & 7 deletions next/content/systems/filecoin_mining/_index.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
---
title: "Storage Mining"
description: Storage Mining System - proving storage for producing blocks
title: Storage Mining
bookCollapseSection: true
weight: 6
dashboardAudit: 1
dashboardState: permanent
dashboardInterface: stable
---

# Storage Mining
# Storage Mining System - proving storage for producing blocks
---


## Status Overview
{{< dashboard-level name="Storage Mining" open="true">}}

The Storage Mining System is the part of the Filecoin Protocol that deals with storing Client's
data, producing proof artifacts that demonstrate correct storage behavior, and managing the work
involved.
Expand All @@ -30,3 +25,6 @@ important that the **protocol constraints** are satisfied. The **protocol constr
spelled out in clear detail (an unclear, unmentioned constraint is a "spec error"). It is up
to implementers who deviate from the algorithms presented here to ensure their modifications
satisfy those constraints, especially those relating to protocol security.

## Status Overview
{{< dashboard-level open="true">}}
3 changes: 0 additions & 3 deletions next/content/systems/filecoin_mining/sector_index/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,4 @@ bookCollapseSection: true
# Sector Index
---

{{< hint danger >}}
Issue with readfile
{{< /hint >}}
{{<embed src="sector_index_subsystem.id" lang="go" >}}
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,4 @@ title: Sector Builder
# Sector Builder
---

{{< hint danger >}}
Issue with readfile
{{< /hint >}}
{{<embed src="sector_builder.id" lang="go" >}}
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
---
title: SectorStore
title: Sector Store
---

# SectorStore
# Sector Store
---

{{< hint danger >}}
Issue with readfile
{{< /hint >}}
{{<embed src="sector_store.id" lang="go" >}}

{{<hint warning>}}
TODO

- talk about how sectors are stored
{{</hint>}}

0 comments on commit 28574ec

Please sign in to comment.