Roadmap

Aron edited this page Aug 8, 2018 · 8 revisions

Roadmap 2018

We manage our roadmap in this Github project: Swarm Roadmap

See the Github issues in the project for details.


2017 and Older:

Swarm working groups 2017

As of end of August 2017, development has been streamlined into working groups: Working groups

latest dev progress for Q2 2017

Projects

Tracks

Network layer rewrite

  • Kademlia 3
    • storer, relayer and light nodes
    • advertising
    • throughput based bucket size?
    • bin-based syncing

Streamer/syncer subtrack

Routing, requests and chunk delivery

  • syncer streamer request handler
  • tests, tests, tests
  • anonymous browsing
  • subscribable chunk/hash streams
  • provable data exchange - periodic signing off of root hash, see receipt with inclusion proof
  • multicast, stream broadcast with adaptive transmission channels (see swatch, https://gist.github.com/zelig/74b3486bcd5523a0b61e12d804d3c00d)
  • file push for Golem
  • delivery using PSS
  • partially specified destinations (see recipient masking and plausibility)
  • improvements by colouring and pss switchboards
  • other methods of guaranteed disjoint paths for efficient parallel routing
  • latency improvements: race for first n out of m chunks and erasure coding
  • beeline delivery using UDP?
  • non-chunk requests
    • database network protocol for index lookups -- outsourcing iteration
    • random access with provable challenge (see inclusion proof), e.g., db entries
    • entire files download from one source
    • proof of custody requests, scan and repair

PSS subtrack

  • mailbox/persistance
  • incentivised relaying based on distance the hop spans
  • mailboxes, retained
  • multicast support

SWAP subtrack

  • tit-for-tat accounting https://github.com/ethersphere/swarm/wiki/Swap
  • also applied to PSS messages
  • sending cheques
  • allow user with 0 ether to participate - chequebook deployment swap = setting up wallet as payment
  • transaction and cheque signing automatic - authorised

BZZ protocol spec docs

Network testing, simulation and visualisation framework

  • support for syncer/request/connectivity/discovery tests tests tests
  • network contention/throughput modelling
  • mock chunk storage and syncing
  • implement remote node monitoring adapter
  • model network latency and failures
  • visualiser improvements

API/FS/OS/WWW track

  • API consolidation https://github.com/ethereum/go-ethereum/issues/14872
  • commit style multistep manifest manipulation using non-synced flag
  • multihash/multiformat ? hash and manifest preamble,
  • add resolver endpoint to http API
  • de-re and de dicto defaults
  • irregular files
  • manifest skeleton
  • obfuscation support
  • swarmignore file
  • per-directory indexes
  • resource update notifications (push and pull) mutability support (sub-block ENS)

Storage layer rewrite

Database/Hash format track

Localstore: Memory store, DB store, Netstore rewrite

  • db store reorg and sync db https://github.com/ethereum/go-ethereum/pull/3780
  • chunk marking:
    • retained: marked not to be removed - future its set upon determining insurance expiry. E.g., self-insurance to protect against GC
    • local: not sent to the swarm. E.g., intermediate manifests before 'commit'
    • synced but not itself entering PO hash stream, e.g., the PO stream itself (to avoid spurious infinite db growth) or cached forwarded query responses
    • cached: garbage collection will affect these but when deleted (and insured), enter PO stream?

localstore rewrite

chunker hooks

  • chunker concurrency control

  • backend for progress bar

  • chunk encryption for privacy: symmetric keccak-feistel cypher

  • shannonian obfuscation for plausible deniability

  • n out of m race hook for chunk level erasure coding

  • mock chunk storage and light (payloadless) syncing for network simulations

  • higher level design for hooks API etc (POC4 preparation), then implement those for POC3

  • POC3 feature change suggestion by @nagydani: chunk encryption in POC3, obfuscation in POC4

  • syncer interaction

    • deliveries (reverse flow) should not enter syncpool even when GC-ed
    • synced flag - put to local storage only - no syncing (commit style multistep manifest manipulation)
    • protected chunks expiry indexing (POC4 preparation)

Obfuscation, encryption and transcoding track

  • context
    • plausible deniability
    • end-to-end encryption
    • anchoring proof of custody
    • outsourcing restricted access
  • obfuscation
    • using symmetric per-chunk XOR (salt?)
    • spec out obfuscator pool and select strategy
    • parallel iteration in chunker
  • double/triple masking save under Hash(Hash(Chunk))
  • proof of custody
    • implement CRASH/SMASH proof of custody using plain pss
  • encryption support for privacy with keccak-feistal blockcypher

Contracts/solidity track

SW3 games (POC4 preparation)

  • swap, swear and swindle implement as per 3rd orange paper
    • abstract courtroom contract suite https://github.com/ethersphere/swap-swear-and-swindle
    • standard library of witness contracts (templates)
    • standard library for provable data exchange, finger pointing
    • base contracts (i keep a secret, release secret sequence, i aggregate your data, certified delivery, promise, * chequebook, payment channels, soft channel deposits, subscription etc)
    • updateable, owned by whom
  • go API for litigation support (finger pointing)

Schedule

POC roadmap

POC 0.3

  • https://github.com/ethersphere/swarm/wiki/Q2-achievements
  • pyramid chunker and Binary Merkle Tree Hash for chunk hash (done)
  • FUSE support (done)
  • full FS support for manifests (done)
  • pss messaging (done)
  • network layer rewrite 1: kademlia, discovery, hive (nearly done)
  • network layer rewrite 2: streamer, syncer, request handler
  • network layer rewrite 3: colored kademlia for storer/relayer nodes
  • light modes of operation
  • encryption for privacy protection
  • SWAP rewrite and bootstrapping chequebook, on by default
  • manifest support

POC 0.4

  • erasure coding for efficient client-side configurable redundancy
  • smash/crash proof of custody
  • swear/swindle incentives for storage insurance
  • obfuscation for plausible deniability
  • stream/multicast
  • POT and db support

Phases

T2

  • network simulation
    • visualiser - Fabio
      • node/connection removal
      • edge length reflects proximity order of peer connection
      • kademlia table display (polling nodes endpoint with id as payload) on select
    • scenario test driver - Lewis
    • remote docker adaptor - Aron
  • network layer rewrite
  • Database and chunker
    • pyramid chunker, append and inifite stream - Fabio
    • binary merkle trie hash 32-byte segment inclusion - Oren
  • API/FS
    • importer - Louis
    • FUSE RW - Zahoor
    • encryption - Dani
  • Plausible deniability
    • obfuscation spec - Nagy Dani (+ Elad)
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.