Johan Pouwelse edited this page Dec 12, 2016 · 85 revisions

Multi-Year Aim: create a censorship-free Internet

Tribler offers a Youtube style service. Bittorrent-compatible streaming, fast search, thumbnail previews and comments.

For the past 9 years we have been building a very robust Peer-to-Peer system. Today Tribler is robust: "the only way to take Tribler down is to take The Internet down" (but a single software bug could end everything).

We are now aiming to make privacy, strong cryptography and authentication the Internet norm.

Current items under active development

Topic and open Github issue Researcher
PageRank-like trust model with Sybil-attack resilience, math foundation Pim Otte
Blockchain-regulated marketplace to bid/ask/cancel bandwidth credits Martijn de Vos
Blockchain enhancement: self-reinforcing trust with collection of credit records based on live edge traversal Pim Veldhuisen
Blockchain: earn credits with anonymous seeding and detect investments Ardhi Putra
Blockchain: detect freeriders, refuse service; anon compatible Ewout Bongers
Fast anonymous streaming with Tor-like onion routing Quinten Stokkink
Blockchain-regulated markets and fault-tolerance Bas ijzendoorn
initial Sybil attack survey Kelong Cong
Scalability: donating TeraBytes to crowdsourcing projects Wouter Smit
Crowdsourcing and investments Bart Gout
Attack-resilient social media on mobile devices, using LibTribler Paul Brussee
Dissemination videos to millions of people in seconds without servers on Android Chengxin Ma
Adversarial search: blockchain-based spam resilience in Youtube-like systems Jelle Licht
Blockchain walker with attack-resilience and integrated NAT puncturing, trusted peer discovery Changliang
Connecting banks to decentral markets through PSD2 open APIs Kypianou
re-use our decentral market platform for real-world business case, crowdsourcing real-estate 4 bsc students
Understanding the impact of latency from Tor-like tunnels on Libtorrent throughput Van Den Heuvel & Van Deursen

Open projects for new master thesis students: Tor-like streaming, relevance ranking of search results (+swarm popularity), perfect metadata through distributed crowdsourcing, self-reinforcing trust(2017 only), and perfect network connectivity using NAT/Firewall traversal.

Main documentation pointers

Dive directly into the Python source code documentation

Overview of Tribler (.html 5 pages). Tribler is the first client which continuously tries to improve upon the basic BitTorrent implementation by addressing some of the flaws. It implements, amongst others, remote search, streaming, channels and reputation-management. All these features are implemented in a completely distributed manner, not relying on any centralized component. Still, Tribler manages to remain fully backwards compatible with BitTorrent. The 2006 overview of Tribler (.pdf 6 pages) featuring taste groups, friends, friends-of-friends and faster downloads by donating bandwidth to friends (protocol spec of friend boosting). Note that the Tribler protocol specification (.pdf 47 pages) is now mostly outdated, as we switched to our new synchronization protocol called Dispersy (see below).

Scientific aim: Push the boundaries of self-organising systems, robust reputation systems and craft collaborative systems with millions of active participants under continuous attack from spammers and other adversarial entities.

Background Tribler started in 2005 and has been Internet deployed for 8 years now. Roughly 10 to 15 scientists and engineers work full-time on it. Our ambition is to make darknet technology, security and privacy the default. As of 2014 we have 25+ contributors and 158,233 lines of code.

Tor-inspired onion routing and privacy protection

As of December 2014 Tribler has a build-in version of a Tor-like anonymity system. This is completely disconnected from 'The' Tor network. It is still ongoing work. It gives you probably superior protection than a VPN, but no protection against resourceful spying agencies.

We have implemented the main parts of the Tor wire protocol within Tribler. Instead of the TCP protocol that 'the' Tor network uses, we use UDP. The enables us to do NAT puncturing and traversal. We have created our own network using this Tor variant, our code is not compatible with normal Tor. We have successfully conducted a small trial in December 2013 with anonymous downloading of a Bittorrent swarm. Essential part of our work is that everybody who downloads anonymously also becomes a relay. This brings the Bittorrent tit-for-tat idea to darknets. With this ongoing work we aim to offer in 2014 with Tribler V7.0 proxied downloading for any Bittorrent swarm.

December 2013 trial with TOR-like onion routing for anonymous swarm downloading

Lengthy documentation in the form of two master thesis documents is available. First is a general documentation of the tunnel and relay mechanism, Anonymous HD video streaming, .pdf 68 pages. Second is focused on encryption part, called Anonymous Internet: Anonymizing peer-to-peer traffic using applied cryptography, .pdf 85 pages. In addition, there are the specifications for the protocols for anonymous downloading and hidden seeding on this wiki.

Robust Reputations

Tribler includes a deployed reputation system which we have been enhancing over the years (.pdf 8 pages). Improving download speed through providing a reward for seeding has been a key quest of the Tribler team since work began in 2005. Our original BarterCast protocol was deployed in 2007, it uses a maxflow-based reputation mechanism. It provides peers with a (partial) view of the network in terms of the data transfers that have occurred between peers. The DropEdge enhancement was proposed to BarterCast by Harvard and Berkeley scientists (.pdf 35 pages) which makes the mechanism harder to attack. They also present in this in-depth work a general model of distributed work systems, security proofs and simulations. We have studied reducing the storage cost of our reputation system. Key BarterCast reputations deployment evaluation: A Network Science Perspective of a Distributed Reputation Mechanism (.pdf 9 pages), from 2013.

overlay protocol for synchronization

The foundation of Tribler is the Dispersy overlay. Dispersy functionality includes: making connections, sending messages, puncturing NAT boxes, and distributed database synchronization. Every 5 seconds Dispersy sends out a message to establish a new connection or re-connect to a known peer.

Overlay communication, peer discovery and content discovery (keyword search) are essential building blocks of a peer-to-peer system. Tribler preserves the content and peers it discovered in the past. Every Tribler client runs a full SQL database engine. Several times per second each Tribler peer sends and receives updates for this database. Our protocol for distributed database synchronization is called Dispersy. See a simple messaging client written with just a few lines of code as a simple tutorial example; outdated broken tutorial.

The detailed wire protocol specification: introduction-request-1 introduction-request-2

Dispersy is a fully decentralized system for synchronization (.pdf), capable of running in challenged network environments. Key features of Dispersy are stateless synchronization using Bloomfilters, decentralized NAT traversal, and data bundle selection algorithms that allow the system to scale over 100,000 bundles in the presence of high churn and high-load scenario's.

Dispersy uses a simple database schema, with the sync table containing the data bundles to synchronise across peers in the packet field. dispersy1

Cybercurrency: bandwidth-as-a-currency

Since 2007 we have been deploying and improving our Bandwidth-as-a-Currency vision. We are fans of Bitcoin, but also showed in an early analysis the flaws in this concept. Bitcoin differs from our work, but also has many similarities (our work pre-dates Bitcoin).

The foundation of our approach is making repeated successful interactions between actors explicit and durable. Cryptographically signed records of successful encounters serve as proof-of-work certificates. The validity and value of these certificates is determined by our reputation system, explained above. Our Bartercast protocol is designed to distribute proof-of-work certificates. It was first deployed in 2007 and improved over the years, for instance, it can now be used to give priority to cooperators. Relaying for anonymity and seeding in Tribler constitutes work which is rewarded with a signed certificate. Our current implementation has been tested extensively on our server-cluster and should be ready for deployment within Tribler soon. Our long-term focus is ensuring perpetual scalable of proof-of-work systems.

Screenshot of our credit mining Beta code from November 2013, showing the boosting of various swarms The credit mining part of our proof-of-work system is ready in Beta form. See the above screenshot from November 2013 showing the boosting of various swarms. Note the investment yields of "struck gold" and "poor" in the right column.

Android port of LibTribler

Android porting teams are working on the downloading and Tor-like protocol part of Tribler and the overlay, channels and search portions. As of June 2014 there is initial running code. The focus is on stability and creating a mature build environment using Jenkins. See below two actual screenshot of current running code. Download the alpha .APK here: http://jenkins.tribler.org/job/Build_Test-TSAP_Android_master/lastStableBuild/

LibTribler running as an CML-RPC service on Android

TOR-like tunnels on Android with Bittorrent downloading

Stealth app for Android

The following work is ongoing. We have an operational Android app that can spread itself via NFC. The app can spread viral via friends, even if it is blocked from a central app store.

Original student assignment: The aim is to create an Open Source Android smartphone app to help bypass restrictions by non-democratic governments. The Arab Spring showed the importance of video recording of mass protests. However, possession of a video recording on your phone of human rights violations and mass uprisings brings grave danger. The idea is to make this app “check-point-proof”, meaning that a somewhat knowledgeable person will not detect the presence of the app and will not discover any video content. The app itself should be hidden, you can make a “stealth” app by somehow removing the app icon from your app list (sadly it simply still shows up in the uninstall app list). The app is activated simply by “dialing” a secret telephone number or other method your deem secure. Starting point for your work can be found here: http://stackoverflow.com/questions/5921071/how-to-create-a-stealth-like-android-app. Your Stealth app need to be able to virally spread and be able to bypass an government restrictions on the official app store. Include the feature for NFC and direct-wifi transfer of the .apk with an easy on-screen manual and steps. Thus users can pass your app along to their friends.

NAT Traversal: 80% success rate

Peer-to-Peer (P2P) networks work on the presumption that all nodes in the network are connectable. However, NAT boxes and firewalls prevent connections to many nodes on the Internet. We created a method to puncture NATs which does not require a server. Our method is therefore a simple no-server-needed alternative to the complex STUN, TURN and ICE approaches. We conducted one of the largest measurements of NAT/Firewall behavior and puncture efficiency in the wild. Our method is a UDP hole-punching technique. We measured the success rate using volunteers running Tribler. Number of users in our trials are 907 and 1531 people. Our results show that UDP hole punching is an effective method to increase the connectability of peers on the Internet: approximately 64% of all peers are behind a NAT box or firewall. More than 80% of hole punching attempts between these peers succeed.

Brief description of our UDP puncture method in IETF draft

Jenkins test of NAT puncturing in Dispersy: http://jenkins.tribler.org/job/Experiment_NAT_1H/2/

Lengthy thesis work on UDP puncturing from 2005

Various other bits

Cybercurrency vision, in collaboration with Harvard We are working since 2007 on turning bandwidth into a currency. We have been working on cybercurrency before Bitcoin appeared and solved the double spending problem. Impressive work, but see our deep analysis of Bitcoin design flaws that need to be fixed such as security and scalability.

For many year we did emulations of swarm health, scalability and flashcrowds http://www.st.ewi.tudelft.nl/~lucia/publications/flashcrowd_p2p.pdf

100+ anon articles: http://freehaven.net/anonbib/date.html http://dx.doi.org/10.1109/HPCC.2012.111

thesis work on rewarding seeding/goodness/anon-proxy: https://github.com/Tribler/tribler/issues/5

Our aim is creating a censorship-free Internet. Tor has proved itself over the past 10 years, however it is struggling to keep up with demand. Onion routing alone cannot overcome the threat of government-imposed Internet shutdown. The challenge is to design a censorship-free Internet sustainable even when an adversary controls the underlying infrastructure. In 2012 we announced our new aim and many news organisations, like Fox and Russian Today called us the the new weapon in the battle for Internet liberty.

Our 2-year in-depth measurement and analysis of Bittorrent (.pdf 25 pages), largest measurement to date in 2004.
Our study covers eight months of the BitTorrent/Suprnova.org file sharing ecosystem. In particular, we show measurement results of the popularity and the availability of BitTorrent, of its download performance, of the content lifetime, and of the structure of the community responsible for verifying uploaded content. We used 100 nodes of our Distributed ASCI Supercomputer to measure every minute the download bandwidth of more than 100,000 downloads of popular content such as the ”Matrix Revolutions”, ”Lord of the Rings III”, and ”Beyond Good and Evil”, and to obtain the statistics of over 60,000 swarms.

Private Bittorrent communities and understanding sharing ratio enforcement effects (.pdf 5 pages). Measurements of 508,269 peers in 444 swarms of five BitTorrent communities, ranging from public to highly elite. We observe download performance, connectability, seeder/leecher ratios, seeding duration, and statistics regarding the resource supply.

Before the rise of Napster we studied such mechanisms on a small scale. Several years before Wikipedia emerged we deployed a music encyclopedia with unconstrained write access, it never became popular because we focused too much on software, instead community growth.

Grand Vision

Bits moving across the Internet are vulnerable to surveillance and censorship on an unprecedented scale. Today, both Internet providers and governments possess the ability to monitor the moves of their digital citizens from central infrastructure points — an ability that creates significant potential for abuse, and a threat that goes beyond the scope of mere monitoring or filtering.

Internet kill switches seem to exist, governments have demonstrated their ability to disable communications networks in times of crisis. During the 2011 Arab Spring, Egyptian authorities demanded that telecommunication companies sever their broadband connections and mobile networks—both local and European operators were forced to comply, and, as a result, digital Egypt vanished. Despite the country’s decentralized infrastructure, an Internet blackout was relatively easy to carry out. We aim to end such blackouts with fully self-organizing systems. By removing all servers (HTTP, DNS, Bittorrent trackers and video portals) we can create a unique rich media landscape. Today Tribler is robust: "the only way to take Tribler down is to take The Internet down".

By using smartphones we aim to even bypass Internet kill switches in the future. Today we have a fully operational app on Android market which is our proof-of-principle defense against Internet kill switches, note that it is stable but far away from being usable.

Our grand vision in a 1+ hour lecture given at Stanford University, via their Youtube channel. We want more then bypass Internet kill switches. Our grand vision is liberating both media and money. See the talk Abstract and slides (.pdf 78 pages). Keywords: transform money, “Bank-of-Bits”, global financial meltdown isolation. Use cooperation&stability, not volatility&greed. Alter the essence of capitalism (rich get richer) by abolishing compound interest rate and facilitation of safe zero-cost money transfers & lending. We aim for a direct assault on the essence of capitalism, aiming even further then the Bitcoin accomplishment (bypassing the central bank).

Tribler in the Media

Technical Tribler Pages

Tribler development pointers / starting point for new developers

We use Jenkins for continuous integration, unit tests, installer builders and performance testing. You will find a lot of automatic running scripts there for things like correctness, NAT puncture performance and GUI tests.

Running Tribler from sources in Eclipse

Systemtap installation pointers

Tribler history

  • 2014: test only: build-in anonymous Tor-like network, exclusively for tribler users (not connected in any with with 'the' Tor project)
  • 2013: Anonymous Tor-like download trial with forum users
  • 2012: Tribler Mobile live streaming from a phone camera to potentially thousands
  • 2011: Libswift accepted as an upcoming IETF Internet Standard
  • 2010: Wikipedia.org uses our technology for live trial
  • 2009: Large HD streaming trial with BBC
  • 2008: Social network without servers and "easy" invites
  • 2007: Our reputation system launched in the wild
  • 2006: Tribler 1st release
  • 2005: First Tribler code = social Bittorrent
  • 2004: Slashdot for first time with largest Bittorrent study