Skip to content

Commit

Permalink
fix: intro section (#971)
Browse files Browse the repository at this point in the history
  • Loading branch information
yiannisbot committed Jul 6, 2020
1 parent 60bb993 commit 22b83d4
Show file tree
Hide file tree
Showing 3 changed files with 143 additions and 0 deletions.
7 changes: 7 additions & 0 deletions next/content/intro/systems/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
menuTitle: System Decomposition
title: System Decomposition
entries:
- why_systems
- impl_systems
---
59 changes: 59 additions & 0 deletions next/content/intro/systems/impl_systems.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
menuTitle: Implementing Systems
title: Implementing Systems
---

# Implementing Systems
---

# System Requirements

In order to make it easier to decouple functionality into systems, the Filecoin Protocol assumes
a set of functionality available to all systems. This functionality can be achieved by implementations
in a variety of ways, and should take the guidance here as a recommendation (SHOULD).

All Systems, as defined in this document, require the following:

- **Repository:**
- **Local `IpldStore`.** Some amount of persistent local storage for data structures (small structured objects).
Systems expect to be initialized with an IpldStore in which to store data structures they expect to persist across crashes.
- **User Configuration Values.** A small amount of user-editable configuration values.
These should be easy for end-users to access, view, and edit.
- **Local, Secure `KeyStore`.** A facility to use to generate and use cryptographic keys, which MUST remain secret to the
Filecoin Node. Systems SHOULD NOT access the keys directly, and should do so over an abstraction (ie the `KeyStore`) which
provides the ability to Encrypt, Decrypt, Sign, SigVerify, and more.
- **Local `FileStore`.** Some amount of persistent local storage for files (large byte arrays).
Systems expect to be initialized with a FileStore in which to store large files.
Some systems (like Markets) may need to store and delete large volumes of smaller files (1MB - 10GB).
Other systems (like Storage Mining) may need to store and delete large volumes of large files (1GB - 1TB).
- **Network.** Most systems need access to the network, to be able to connect to their counterparts in other Filecoin Nodes.
Systems expect to be initialized with a `libp2p.Node` on which they can mount their own protocols.
- **Clock.** Some systems need access to current network time, some with low tolerance for drift.
Systems expect to be initialized with a Clock from which to tell network time. Some systems (like Blockchain)
require very little clock drift, and require _secure_ time.

For this purpose, we use the `FilecoinNode` data structure, which is passed into all systems at initialization:

{{< readfile file="../../systems/filecoin_nodes/node_base/filecoin_node.id" code="true" lang="go" >}}

{{< readfile file="../../systems/filecoin_nodes/repository/repository_subsystem.id" code="true" lang="go" >}}

# System Limitations

Further, Systems MUST abide by the following limitations:

- **Random crashes.** A Filecoin Node may crash at any moment. Systems must be secure and consistent through crashes.
This is primarily achived by limiting the use of persistent state, persisting such state through Ipld data structures,
and through the use of initialization routines that check state, and perhaps correct errors.
- **Isolation.** Systems must communicate over well-defined, isolated interfaces. They must not build their critical
functionality over a shared memory space. (Note: for performance, shared memory abstractions can be used to power
IpldStore, FileStore, and libp2p, but the systems themselves should not require it.) This is not just an operational
concern; it also significantly simplifies the protocol and makes it easier to understand, analyze, debug, and change.
- **No direct access to host OS Filesystem or Disk.** Systems cannot access disks directly -- they do so over the FileStore
and IpldStore abstractions. This is to provide a high degree of portability and flexibility for end-users, especially
storage miners and clients of large amounts of data, which need to be able to easily replace how their Filecoin Nodes
access local storage.
- **No direct access to host OS Network stack or TCP/IP.** Systems cannot access the network directly -- they do so over the
libp2p library. There must not be any other kind of network access. This provides a high degree of portability across
platforms and network protocols, enabling Filecoin Nodes (and all their critical systems) to run in a wide variety of
settings, using all kinds of protocols (eg Bluetooth, LANs, etc).
77 changes: 77 additions & 0 deletions next/content/intro/systems/why_systems.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
menuTitle: What are Systems?
title: What are Systems? How do they work?
---

# What are systems? How do they work?
---

Filecoin decouples and modularizes functionality into loosely-joined `systems`.
Each system adds significant functionality, usually to achieve a set of important and tightly related goals.

For example, the Blockchain System provides structures like Block, Tipset, and Chain, and provides functionality
like Block Sync, Block Propagation, Block Validation, Chain Selection, and Chain Access. This is
separated from the Files, Pieces, Piece Preparation, and Data Transfer. Both of these systems are separated from
the Markets, which provide Orders, Deals, Market Visibility, and Deal Settlement.

# Why is System decoupling useful?

This decoupling is useful for:

- **Implementation Boundaries:** it is possible to build implementations of Filecoin that only implement a
subset of systems. This is especially useful for _Implementation Diversity_: we want many implementations
of security critical systems (eg Blockchain), but do not need many implementations of Systems that can be
decoupled.
- **Runtime Decoupling:** system decoupling makes it easier to build and run Filecoin Nodes that isolate
Systems into separate programs, and even separate physical computers.
- **Security Isolation:** some systems require higher operational security than others. System decoupling allows
implementations to meet their security and functionality needs. A good example of this is separating Blockchain
processing from Data Transfer.
- **Scalability:** systems, and various use cases, may drive different performance requirements for different opertators.
System decoupling makes it easier for operators to scale their deployments along system boundaries.


# Filecoin Nodes don't need all the systems

Filecoin Nodes vary significantly, and do not need all the systems.
Most systems are only needed for a subset of use cases.

For example, the Blockchain System is required for synchronizing the chain, participating in secure consensus,
storage mining, and chain validation.
Many Filecoin Nodes do not need the chain and can perform their work by just fetching content from the latest
StateTree, from a node they trust.

Note: Filecoin does not use the "full node" or "light client" terminology, in wide use in Bitcoin and other blockchain
networks. In filecoin, these terms are not well defined. It is best to define nodes in terms of their capabilities,
and therefore, in terms of the Systems they run. For example:

- **Chain Verifier Node:** Runs the Blockchain system. Can sync and validate the chain. Cannot mine or produce blocks.
- **Client Node:** Runs the Blockchain, Market, and Data Transfer systems. Can sync and validate the chain. Cannot mine or produce blocks.
- **Retrieval Miner Node:** Runs the Market and Data Transfer systems. Does not need the chain. Can make Retrieval Deals
(Retrieval Provider side). Can send Clients data, and get paid for it.
- **Storage Miner Node:** Runs the Blockchain, Storage Market, Storage Mining systems. Can sync and validate the chain.
Can make Storage Deals (Storage Provider side). Can seal stored data into sectors. Can acquire
storage consensus power. Can mine and produce blocks.

# Separating Systems

> How do we determine what functionality belongs in one system vs another?
Drawing boundaries between systems is the art of separating tightly related functionality from unrelated parts.
In a sense, we seek to keep tightly integrated components in the same system, and away from other unrelated
components. This is sometimes straightforward, the boundaries naturally spring from the data structures or
functionality. For example, it is straightforward to observe that Clients and Miners negotiating a deal
with each other is very unrelated to VM Execution.

Sometimes this is harder, and it requires detangling, adding, or removing abstractions. For
example, the `StoragePowerActor` and the `StorageMarketActor` were a single `Actor` previously. This caused
a large coupling of functionality across `StorageDeal` making, the `StorageMarket`, markets in general, with
Storage Mining, Sector Sealing, PoSt Generation, and more. Detangling these two sets of related functionality
requried breaking apart the one actor into two.

# Decomposing within a System

Systems themselves decompose into smaller subunits. These are sometimes called "subsystems" to avoid confusion
with the much larger, first-class Systems. Subsystems themselves may break down further. The naming here is not
strictly enforced, as these subdivisions are more related to protocol and implementation engineering concerns
than to user capabilities.

0 comments on commit 22b83d4

Please sign in to comment.