Scalable, ACID, Fast, Host

Updated Nov 3, 2016

The Sia host is currently deficient, primarily in that it does not provide true ACID guarantees around the sectors and contracts that it agrees to store, and secondarily around the way it stores data and the fact that it does not have good scaling properties.

A list of requirements:

  • Host must have less than 0.1% storage overhead for managing contracts, metadata, etc.
  • Host must be able to perform deduplication on sectors, allowing for contracts with overlapping data + timeframes without blowing up storage requirements
  • Host must use at most 1/10,000 the amount of RAM as it is storing on disk. e.g. storing 10 TiB of data must have an overhead of at most 1 GiB of RAM. Scaling must be linear or better.
  • Host must be capable of performing 5+ sector add operations per second on spinning disks, while simultaneously handling to 20+ sector read and 5+ sector delete operations per second, on spinning disks
  • Host must be capable of committing with ACID certainty to renewing a file contract with 1 million sectors (4 TiB) in under 1 second, on spinning disks
  • Host must be capable of leveraging a large amount of storage folders.
  • Host must be capable of operating at full speed if an arbitrary number of storage folders are broken (disk failures, etc.)
  • Host must be capable of re-sizing and removing storage folders without losing data
  • Host must be capable of re-sizing and/or removing unrecoverable storage folders while recovering as much data as possible
  • Host must be capable of doing resize, add, and remove storage folder operations while running at full throttle on the live network
  • Host must be able to forcibly delete a sector at the request of the user (to remove offensive/illegal data)

  • Host must be able to display a list of contracts that it is actively serving, including their blockchain progress + financial stats, storage stats, etc.

  • Host must be able to display a list of historic contracts
  • Host must be able to display warnings about failing storage folders
  • All API endpoints in the host that can return a potentially unbounded amount of data must have pagination

  • The host must be able to do uploads + downloads in the same RPC, on the same contract. Potentially renews as well.

  • The host must be able to perform batch operations on RPCs, to minimize the amount of networking overhead required, as contract negotiation requires multiple round trips.

Gateway Upgrades

Updated Oct 27, 2016

The gateway is performing well, but needs some additional upgrades

  • Version exchange should provide a genesis block + self-identifying nonce
  • Gateway should use self-identifying nonce to not-connect to itself
  • The set of bootstrap nodes needs to be monitored and nodes with low uptime should be removed
  • Gateway needs to be using Yamux instead of muxado

  • More research should be done on what we can do to prevent eclispe attacks, DoS attacks, and other network-based attacks.

High Quality Sia Wallet

Updated Nov 3, 2016

The Sia wallet is somewhat hacked together, and while carefully designed not to lose any coins, there are other components of the wallet which leave something to be desired. For example, it takes a really long time to unlock. This project intends to fix all of the serious shortcomings in the Sia wallet with the intention of creating something that operates like a professional cryptocurrency wallet.

A general list of requirements:

  • The wallet must unlock in under a second.
  • The wallet must not consume more than 50MB of RAM.
  • The wallet must be able to process 1000 blocks per second coming from the consensus set (during IBD and rescan).
  • The wallet must be able to display progress during IBD and rescan, including through the API.
  • The wallet must provide durability guarantees to the user. In combination with #3, clever engineering is required.
  • The wallet must be able to track multiple seeds and keys.
  • The wallet must expose to the user which keys + seeds it is tracking, including how many funds are contained in each.
  • The wallet must have methods for 'sweeping' outputs from one seed or key into a seed.
  • The wallet must have the ability to initialize from existing seed.
  • When loading a seed, the wallet must have extremely high probability of finding all addresses with money in them.
  • The wallet must support custom encryption passwords.
  • The wallet must support 128bit seeds.
  • The wallet must be able to create restore-able backups. Per-key and per-seed backups must be supported.
  • The wallet must be able to restore from backups.
  • The wallet must be able to restore from all legacy backups.
  • The wallet must know not to create transactions that violate the IsStandard rules, particularly the 32kb max-transaction-size rule.
  • All API calls to the wallet which return potentially unbounded amounts of data must support pagination.
  • The wallet must avoid address reuse as much as possible.
  • The wallet requirements must be documented in wallet.go
  • The wallet must be able to accurately predict how much the fees on a given transaction will be

Notable requirements that are skipped for the Sia wallet:

  • Privacy guarantees
  • HD Wallet support

While eventually we hope the Sia platform will support all of these, privacy especially is something that will not be well supported without alterations to the Sia consensus protocol, and ultimately a from-the-group-up rethinking of the platform design and user-experience. Neither of these items is on the short-term or medium-term list of things we hope to accomplish.

No results.