Internet 2.0

This is an overview of a vision for the future of the Internet. It pulls together recent innovations in authentication, cryptography, blockchains, and more. The result is an open platform that's decentralized, secure, resilient, flexible, and designed from the ground up to ensure integrity of data.

Hopefully, the Qbix Platform 2.0 can be a reference implementation of this system, on top of which we can keep carrying out our mission to Empower People and Unite Communities.

Here is how it all works, step by step:

Basic building blocks

1. Users. Users can get ids on networks. Users may be people or communities, etc.

2. Identity. The Qbix auth protocol is used to authenticate with networks and maintain identities across networks. Messages posted by users to streams may be signed using one of their private keys.

3. Streams. Users can publish Streams, if the Network allows them to. The current streams live on a central server, which is the single source of truth. Qbix already built a huge amount around real-time updates (e.g. moves on a chessboard, messages in a chatroom, etc.) and offline notifications. There's an ecosystem of components (chess games, chats, etc.) and installers for plugins and apps, which communities can install. It would be nice if the next version could use all that, but extend the Streams concept to allow total decentralization, replication, etc. while maintaining its current features like access control, roles and permissions, subscriptions, notifications etc.

4. Messages and States. In the next version, Streams will be implemented just like in Git. Every node in the stream is called a "Stream State" ("revision" in git). Each state has a hash ("revision hash" in git). Transitions between them are documented as "Messages" (deltas/diffs in git), which references previous stream states. The signatures are with hmac, so they are resilient to quantum computer innovations etc.

5. Trees. Streams can fork to become Trees. As long as there are no merges, there are just trees. This can be used for conversations branching off, etc. and you don't need global consensus. You can just pull in other streams, and append to the tree. A lot of things can be done with appending trees, but sometimes you need consensus, which is why we need merging.

6. Apps and Stream Types. Each Stream Type would have its own merge rules. Actually, Stream Types themselves would have Streams and versions, they are similar to RDF resources. And Apps that Networks install may support various Stream Types. There is no central repository for anything technology-wise, but there may be prominent repositories for naming things (apps, stream types, etc.) along with their merge rules, and so on. They would have official Streams representing them, and you can subscribe to updates (e.g. when a new version of an App or Stream Type has been released, hopefully backward compatible). They may also be audited by third parties which can sign verified claims on those streams, about whatever desirable or undesirable properties they may have. If those streams are replicated, it will be much harder to erase these Messages (about desirable or undesirable properties) but of course the third parties can also post them elsewhere. Presumably, Stream Types and Apps can also fork from one another, representing inheritance. Multiple inheritance may be possible also through Merging (see below).

7. Access. Every message needs to be validated according to some rules. Each stream has an initial Publisher, which is a User on a Network. When streams are first published, they reference a certain Type of stream, and thus fork that type's Stream Templates for their default fields, access control, notification rules, etc. This establishes the initial Access Rules, which normally would let the original Publisher of the stream do lots of stuff to it. Then the stream may be changed via Messages sent to it by an authorized User, which is usually the Publisher. Each type of message, including those intended to change the Rules, is described in the Stream Type. You can use inheritance of Stream Types, to build up various Rules for Messages. One particular category of Rules is Access Rules. Here you can grant access to certain Capabilities to send certain Messages to the stream. You then send these capabilities to the Users who you allow to do things, and they can further delegate these Capabilities to others (if the Capability does not require a specific User ID). The Stream's Access Rules may later be changed, adding or removing access for some Capabilities. Someone may lose read access and not even be able to find out why, because they can no longer see the stream's Rules past the point where they lost read access to the Rules themselves. Note: Messages represent mutations on the streams, but there should be a standard interface to see the Stream's States and Messages (Qbix already built such an interface). Reading a stream may or may not result in a log message, depending on the Rules.

8. Payments. There should be some native payment system with this whole thing. The Payments may be optionally submitted with Messages, and each payment may be recorded on the Stream. Each Network may have its own currency. A Network may require payment to get a User account, to publish a Stream, to read a stream's State or Messages, to post a Message on a Stream, to validate Rules on a message, or to implement a Merge with other Streams. It may charge different things based on the size of the job and Stream Type. If the client posts a signed, valid Message with a Payment, the network server may reject the Message with a counter-offer of a higher fee for the Message, in which case the client may re-post the Message again with that fee (assuming the fee hadn't changed in the interim, in which case the process would repeat).

Now come the cool parts: Blockchain and Consensus

9. Replication. A Stream can be forked, and every so often the new messages can be merged into the fork (as long as the forking App still has access to the forked Stream). This is a Replica. Of course, whoever is replicating the Stream (such as a search engine, to show local results quickly) needs permission. The nice thing is that they can subscribe to updates (which we already implemented) and not have to crawl the web like they do now. They may get HTTP Web Pushes for example, or hold open a regular socket between servers for more real-time replication.

10. Diverging. Sometimes, instead of just Replication, the forked Streams may get their own messages and therefore diverge. How the Access is determined for this is an open question. If you want to enable strong copyright enforcement, you may want to inherit rules of the original Stream and somehow find and flag Streams that show signs they've been forked from a copyrighted Stream. On the other hand, sometimes if you start a chess club or facebook group and it just becomes "bigger than you", then perhaps governance should move to some group of people. Otherwise, you can close a large club or company on a whim, that many people have come to rely on. The nice thing is, these things can probably be modeled as Rules in a Stream Type. I mused about such political questions 4 years ago but I'm still looking for more discussion on the topic :)

11. Merging. An App running on a Server may initiate a Merge into their Stream, from stream forks on other Servers. Most Stream Types would have a well-defined algorithm for Merging, which could handle almost any type of merge automatically. These algorithms are part of the Stream Type, and are publicly available and run locally by all Servers on Networks which do the merge. (Probably they are written in Javascript and execute in a VM). Merging two or more Streams with a shared history will result in serializing Messages on the merged Streams in a particular order. Merging algorithms must always result in the same ordering of Messages in the resulting Stream after the Merge. This leads to the next point, Blockstreams.

12. Blockstreams. A Blockstream is essentially like a blockchain but more decentralized (no network-wide consensus is required). It can be thought of as a Stream that implements Fan-Out, Fan In messaging pattern. The Network Servers which agree to Replicate the Stream also run a Consensus protocol (such as Ripple consensus). As a result, every so often, the divergent forks must (with rapidly increasing probability) be merged back into a main Blockstream, after which they can fork again. The period between each fork and merge is called a Block in the Blockstream, and the rules are determined by the Consensus protocol. Blockstreams can be used for collaborative documents, distributed ledgers for currencies, and anything else that requires consensus. But they don't necessarily have to be network-wide. They can be e.g. lightweight "companies" formed on a Network that have their own internal accounting for e.g. bugfixing, or ownership, etc. Or they can be local communities with their own currency.

13. Governance. The concept of "Ownership" is broken down to "Access" to streams. A User can publish more than one Stream, and their initial access varies by Stream Type. For most Stream Types, the publisher has special privileges, but they may, for instance, assign governors and publicly repudiate their own elevated privileges to that stream. That way, everyone knows that no one remains who "owns" the stream, and the stream can evolve according to the Rules set out. Packages of these Rules can be merged into Stream Types (multiple inheritance / behaviors) and used for Streams forked from their Stream Templates. New innovations in Governance systems can therefore be done, and used by various people around the world. Anything from Hierarchy to Holacracy to Voting to Polling and other things as well can be used, to achieve verified consensus on policies. And the best part is, everyone can verify that their input was considered and the Rules were applied. This can lead to innovations in achieving and implementing political consensus.

How does all this interoperate together?

14. Web. A Blockstream or any of its States can be retrieved by its content-addressed hash. This can be a more resilient alternative to HTTP and web resources. Removing, or even hacking into, a Stream would be much harder than taking down or defacing a Website. Streams could be Replicated across many different servers, and each Replica could be found by searching a variety of Streams, including for example Indexes that categorize Streams by various criteria. These Indexes can vary from dumb DHTs to entire search engines. The system would not just support pulling resources (like when users surf or search engines crawl the Web) but also pushing updates, resulting in various Rules and possibly many different cascading effects being run, in a distributed event-driven architecture. Native apps could extend WebViews with a protocol, such as stream:// where the native app would intercept and load resources in a decentralized way, and then let the Web Browser handle the rest. It wouldn't have cookies and localStorage and Javascripts would be isolated per Stream. If Streams wanted to share objects in the browser, the user would have to allow it explicitly.

15. Internet. All this can also be done cross-network. The idea of a Network (which I started with) usually is associated with a Domain, in which there are specific policy rules about how computers connect and share information. But with the internet protocol suite, it's possible for any two computers to do Forking, Merging, etc. provided they are both online. Each operation is done in a client-server fashion, but computers can switch roles anytime, e.g. first A forks from B, then B forks from A. These computers can be Servers on some network, or they can even be local clients. In fact, "offline access" can be properly modeled as a local client fork, where messages are validated to the best of the client's ability and then later during merging it may be discovered that the client needs to pull the latest changes, do Operational Transformation on the messages, and resubmit. In git, this workflow is essentially "pull+merge, rebase, push", and it's repeated until merging can be done automatically without the client.

16. Smart Contracts. So now, what about Smart Contracts, etc? Well, right now they're run in e.g. the Solidity Virtual Machine that's implemented on a global consensus network called Ethereum. I don't think we need global consensus, and I don't think we need a special language for the VM. It can just be (a subset of) Javascript, written in Stream Types and Apps. Networks can download and run these Apps in Virtual Machines, isolated from everything else (just like PhantomJS, these sandboxes have been tested very well for holes over time). These Apps all out to the VM using plain Javascript methods exposed for doing operations with Streams. But living in a VM, how do they find out anything about the outside world, e.g. the weather, stock prices, etc. that they're supposed to enforce?

17. An Internet of Streams. What if, instead of focusing on Servers like we do now, we focus on the logical Streams between them. Streams have a history, and evolve as I described above (9-12). They can reference States of other Streams, subject to all the Access rules, etc. and Networks can execute via Apps that post Messages to Streams. In some way, Streams are similar to "Genetic Code" and these references are similar to actual interactions between animals! Well, perhaps it's not so grand ... it's just that References to States in some other Streams establish that you knew about that State (e.g. the Blockstream of the Stock Market set a certain price for IBM, or the Blockstream of some weather bureaus said it was such-and-such temperature in such-and-such place). Until recently, if you want to prove that X happened in a secret way, you had develop some kind of a zero-knowledge proof. Now with Blockchains, we can have consensus about what happened. And with Blockstreams, we can have ad-hoc consensus that evolves over time as more Streams may agree to merge into a mighty Blockstream, i.e. a blockchain.