Johan Pouwelse edited this page Oct 18, 2018 · 125 revisions

Tribler: an attack-resilient micro-economy for media

  • Anonymous Tor-like downloads and fast search
  • Earn seeding tokens
  • Reward content creators

Tribler Search V7.0

Tribler is a Bittorrent-compatible alternative to Youtube. It is designed to protect your privacy, build a web-of-trust, be attack-resilient, and reward content creators directly. We are building a micro-economy without banks, without advertisers, and without any government. Together with Harvard University, the Tribler team deployed one of the first fully distributed ledgers in August 2007, see BBC News coverge and a New Scientist article. In coming years we will further expand our micro-economy based on bandwidth tokens. We aim to become the key place where audiences find their torrents, creative talents get discovered, and artists get financial rewards from their fans. Tribler is the place where 100 percent of the money goes to artists and the people that run the infrastructure.

Our mission: re-inventing media and money.

Over 2 million people have used Tribler over the years. The Tribler project was started in 2005 at Delft University of Technology and over 100+ developers contributed code to it. We are continuously improving it and further expanding the scientific developers team.

Technical foundations of Tribler are the Bittorrent protocol, an overlay for P2P communication across NAT/firewalls, gradual building of trust in public keys with Bittorrent seeding, and our token economy with incentives for Tor-like relaying and hidden seeding. For 12 years we have been building a very robust self-organising Peer-to-Peer system. Today Tribler is robust: "the only way to take Tribler down is to take The Internet down" (but a single software bug could end everything).

Current items under active development

This wiki page contains our main technical documentation, highlights:

  • Trustchain: our 10.000 transactions per second ledger
  • Token economy and decentral market
Topic and open Github issue Researcher
Incremental PageRank-based trust model for attack-resilience Alexander Stannat
Transitive trust based on blockchain constructs Quinten Stokkink
Youtube-like scale: Gigachannels with 1 billion magnet links Vadim
A live token economy and distributed marketplace for bandwidth tokens Martijn de Vos
Distributed Apps: autonomous code execution using IPv8 plugins Mitchell Olsthoorn
Blockchain: detect freeriders, refuse service; anon compatible Ewout Bongers
Walker infrastructure with 48 NAT boxes and automated NAT puncturing analysis Remko Naber
self-sovereign identity+trust: overview, biometric validation, boosted privacy, and voting pass 8 students
Low-level debugging of Tor-like tunnels on Libtorrent and performance in general Vadim
Autonomous self-replicating code buy servers with Bitcoins 4 students
Prototype projects "Blockchain Engineering" Master course around threshold encryption, trustchain, self-sovereign ID, etc 45 master students

Open projects for new TUDelft master thesis students: Tor-like streaming, self-sovereign identity and authentication on Android, relevance ranking of search results (+swarm popularity), perfect metadata through distributed crowdsourcing, self-reinforcing trust, and perfect network connectivity using NAT/Firewall traversal. Speculative projects with long-term focus: prediction market for climate change. A market designed against frontrunners and high-frequency trading abusers in general.

Project: Waiting for new developers Prior Dev
Bottom-up consensus model with full scalability using checkpointing Kelong Cong
PageRank-like trust model with Sybil-attack resilience Pim Otte
Towards global consensus on trust within the Tribler micro-economy Jan-Gerrit Harms
Decentral market primitives: market order and execution engine fairness Marc Juchli
Decentral market: privacy for traders and spam-resilience Bas ijzendoorn
Secure hardware storage of keys using PUF hardware Ade Ade Setyawan Sajim
Blockchain: self-reinforcing trust with collection of credit records Pim Veldhuisen
Fast anonymous streaming with Tor-like onion routing
Attack-resilient social media on mobile devices, using LibTribler Paul Brussee
Blockchain walker with attack-resilience and integrated NAT puncturing, trusted peer discovery Changliang
Blockchain: earn credits with seeding on Kodi-like devices Bohao Zhang
crowdsourcing of rich metadata Stijn van Schooten
Determine popularity+age of content with spam and attack resilience, swarm size community Chengxin Ma
Adversarial search: blockchain-based spam resilience in Youtube-like systems Jelle Licht
Scalability: donating TeraBytes to crowdsourcing projects Wouter Smit
Connecting banks to decentral markets through PSD2 open APIs Kypianou
Crowdsourcing and investments Bart Gout
re-use our decentral market platform for real-world business case, crowdsourcing real-estate 4 bsc students
Establish + Real-time display of Blockchain trust 20 Context project students

Aim: solving trust

Social media today is obsessed with profit, filled with advertisements, overflowing with falsehoods, and infested with fake news. We're trying to fix these hard problems in a unique way: by building trust. Our audacious ambition is a clean-slate re-creation of The Internet itself with foundations of trust. Craiglist and eBay showed us in 1995 that trustworthy trade was possible online. Uber, Etsy, and AirBnB show that entire industries can be disrupted by a single platform with a natural monopoly.

For the past 18 years we have build and deployed platforms to create trust. Before Wikipedia and Youtube existed we studied the mechanisms behind trust and user-generated content on a small scale. Several years before Wikipedia emerged we deployed a music encyclopedia with unconstrained write access, it never became popular because we focused too much on software, instead community growth.

Today we keep a narrow focus and continuously expand Tribler with trustworthy decentralized technology. We launched sub-second keyword search for Bittorrent swarms without any server back in 2010 (see our old Google Tech Talk on this topic). One of our operational trust browsing prototypes: trust browser

Further reading:

  • Our work from 2004, 2-year in-depth measurement and analysis of Bittorrent (.pdf 25 pages), largest measurement to date. Covers eight months of the BitTorrent/Suprnova.org file sharing ecosystem. In particular, we show measurement results of the popularity and the availability of BitTorrent, of its download performance, of the content lifetime, and of the structure of the community responsible for verifying uploaded content.

Tribler features and innovations

Tribler supports torrent search without websites, anonymous downloading, torrent streaming, channels of torrents, and sharing content for tokens. Overview of Tribler (.html 5 pages). All Tribler features are implemented in a completely distributed manner, not relying on any centralized component. Still, Tribler manages to remain fully backwards compatible with BitTorrent. The 2006 overview of Tribler (.pdf 6 pages) featuring taste groups, friends, friends-of-friends and faster downloads by donating bandwidth to friends (protocol spec of friend boosting). Note that the 2006-2009 Tribler protocol specification (.pdf 47 pages) is now mostly outdated, as we switched to our new synchronization protocol called Dispersy (see below).

Trust in social media content is essential for a sustainable ecosystem. We introduced channels of Bittorrent swarms in 2009 with the Tribler 4.x release. Each user can vote on channels to increase their visibility and tell everybody the channel owner is not a spammer and not spreading fake items. The reputation of both the voters and channel owner are important.

Tribler protects your privacy by not storing anything on any server. To protect your privacy even more, we have prototyped search algorithms based on homomorphic cryptography. We presented a new algorithm system for privacy-respecting scalable Gnutella-like search in 2014. Our approach to scalability is a similarity function in the encrypted domain (homomorphic), enabling semantic clustering with privacy.

Back in 2006 we introduced long-lived identities to separate trustworthy peers from freeriders and spammers (PermID). To protect your privacy further we also devised an alternative to onion routing which potentially could have stronger security guarantees (correlation attack). See the details in this thesis on Multi-core architecture for anonymous Internet streaming which includes a performance analysis of running code.

Further reading for developers:

Our primitive 2007 distributed ledger and Trustchain (2012)

We deployed one of the worlds first fully distributed ledgers in August of 2007. For over a decade we meticulously measured, analysed, improved, and enhanced this live system. Today it defines the state-of-the-art in blockchain research, but in the early days it barely functioned at all. A total of five Ph.D. students of Delft contributed key parts and upgrades.

At launch we called our initiative "bandwidth-as-a-currency". Today we have specific terminology for what we did: a token economy. We are making Internet bandwidth a tradable commodity without any middleman or need for any centralised governance. Our efforts span over a decade, making us the veterans in the field. Our ledger provides an incentive for Bittorrent seeding and Tor-like relaying. For numerous years the tit-for-tat algorithm provided the only incentive for contributions in Bittorrent. No incentive for seeding existed, except when central servers kept track of your uploads and downloads. We measured closed invite-only communities for numerous years and mathematically showed their rich-get-richer properties. For details see Fast download but eternal seeding: the reward and punishment of sharing ratio enforcement and our measurement paper understanding bandwidth economics and ratio enforcement (.pdf 5 pages). We measured 508,269 peers in 444 swarms within five BitTorrent communities, ranging from public to highly elite. We observe download performance, connectability, seeder/leecher ratios, seeding duration, and statistics regarding the resource supply.

We got inspiration for a novel blockchain design based on operating our own ledger and studying token economies. Our current work is called Trustchain, a unique design from 2012 where all participants have their own personal blockchain and create their own genesis block. Our older work used a graph-based approach and graph-based reputation algorithms. Trustchain records transactions in a tamper-proof and scalable manner. It does not require mining and does not try to solve the double spending problem. Our primitive 2007 ledger pre-dates Bitcoin, additionally our 2012 DAG-based approach pre-dates IOTA and the Texas DAG patents.

We are fans of Bitcoin, but also showed in an early analysis the flaws in this concept. Our approach to digital signatures is the essential difference which sets us apart from others. Mono-signatures form the foundation of all other projects we have seen in the past decade. Meaning, in systems such as Bitcoin a transaction is already valid with a single signature. Our Trustchain design does not permit transactions with merely a single signature. Trustchain only supports multi-party agreement recording, others are not valid. We believe that we created a more powerful system by removing single-signature transactions. Only time can tell the usefullness of this academically-pure and minimal design.

The foundation of our approach is making repeated successful interactions between actors explicit and durable. Cryptographically signed records of successful encounters serve as proof-of-work certificates. The validity and value of these certificates is determined by a trust and reputation system. Relaying for anonymity and seeding in Tribler constitutes work which is rewarded with a signed certificate. Helping others and uploading in Bittorrent swarms is rewarded with bandwidth tokens (e.g. signed certificates). Mining in our system becomes download parts of a swarm and uploading them to multiple interested parties. In 2013 we got the credit mining part of our system operational in early Beta. The screenshot below from November 2013 shows the boosting of various swarms. Note the investment yields of "struck gold" and "poor" in the right column.

Screenshot of our credit mining Beta code from November 2013, showing the boosting of various swarms

Further reading:

Our methodology: keep focus and dream big

For our narrow focus of a Bittorrent client we are exploring the fundamentals of identity, trust, and trade. With over 1 billion users of Youtube and Bittorrent we know there is a mass audience ready for something better.

Our approach has very boring foundations, when compared to newer and more sexy work, like IPFS, FileCoin, or Storj. We first measured Bittorrent in 2002, it is a flourishing mature ecosystem and ready for an upgrade. Bootstrapping an ecosystem is hard, we designed and deployed a superior alternative to Bittorrent. It became an official IETF Internet Standard, but completely flopped. This formed our preference for simplicity, elegance and our allergy for bloatware, clean-slate work, and over-engineering. Numerous other projects try to create a generic approach using an ICO for funding and promising the early adopters a dazzling return-on-investment. Tribler is different. rant warning. We are non-profit academics. We do not want to replace the old elite with a new crypto-currency elite. What is changed if we replace backroom deals, lobbyists, middleman, and legal monopolies with the tools of the new elite: algorithms, early investor rewards, proof-of-dominating-stake, and smart contracts? Replacing the analog world and breading digital-native inequality does not make the world a better place. We are creating a micro-economy based on fairness, trust, equality, and self-governance. By design we banish rent-seeking. Critical infrastructure rarely makes profit. We are trying to build critical infrastructure.

Tor-inspired onion routing

As of December 2014 Tribler has a build-in version of a Tor-like anonymity system. This is completely disconnected from 'The' Tor network. It is still ongoing work. It gives you probably superior protection than a VPN, but no protection against resourceful spying agencies.

We have implemented the main parts of the Tor wire protocol within Tribler. Instead of the TCP protocol that 'the' Tor network uses, we use UDP. The enables us to do NAT puncturing and traversal. We have created our own network using this Tor variant, our code is not compatible with normal Tor. Work started as a small trial in December 2013 with anonymous Bittorrent downloading. Essential part of our work is that everybody who downloads anonymously also becomes a relay. This brings the Bittorrent tit-for-tat idea to darknets. With this ongoing work we aim to offer in 2018 with Tribler V7.0 proxied downloading for any Bittorrent swarm.

December 2013 trial with TOR-like onion routing for anonymous swarm downloading

Lengthy documentation in the form of two master thesis documents is available. First is a general documentation of the tunnel and relay mechanism, Anonymous HD video streaming, .pdf 68 pages. Second is focused on encryption part, called Anonymous Internet: Anonymizing peer-to-peer traffic using applied cryptography, .pdf 85 pages. In addition, there are the specifications for the protocols for anonymous downloading and hidden seeding on this wiki.

overlay protocol for synchronization

The current foundation of Tribler is the Dispersy overlay. Dispersy functionality includes: making connections, sending messages, puncturing NAT boxes, and distributed database synchronization. Every 5 seconds Dispersy sends out a message to establish a new connection or re-connect to a known peer. Note that we are transitioning to a new overlay for the durations of 2018.

Overlay communication, peer discovery and content discovery (keyword search) are essential building blocks of a peer-to-peer system. Tribler preserves the content and peers it discovered in the past. Every Tribler client runs a full SQL database engine. Several times per second each Tribler peer sends and receives updates for this database. Our protocol for distributed database synchronization is called Dispersy. See a simple messaging client written with just a few lines of code as a simple tutorial example; outdated broken tutorial.

The detailed wire protocol specification: introduction-request-1

Dispersy is a fully decentralized system for synchronization (.pdf), capable of running in challenged network environments. Key features of Dispersy are stateless synchronization using Bloomfilters, decentralized NAT traversal, and data bundle selection algorithms that allow the system to scale over 100,000 bundles in the presence of high churn and high-load scenario's.

Dispersy uses a simple database schema, with the sync table containing the data bundles to synchronise across peers in the packet field. dispersy1

Android port of LibTribler

Android porting teams are working on the downloading and Tor-like protocol part of Tribler and the overlay, channels and search portions. As of June 2014 there is initial running code. The focus is on stability and creating a mature build environment using Jenkins. See below two actual screenshot of current running code. Download the alpha .APK here: https://jenkins.tribler.org/job/Build-Tribler_Android-Python/lastBuild/

LibTribler running as an CML-RPC service on Android

TOR-like tunnels on Android with Bittorrent downloading

Stealth app for Android

The following work is ongoing. We have an operational Android app that can spread itself via NFC. The app can spread viral via friends, even if it is blocked from a central app store.

Original student assignment: The aim is to create an Open Source Android smartphone app to help bypass restrictions by non-democratic governments. The Arab Spring showed the importance of video recording of mass protests. However, possession of a video recording on your phone of human rights violations and mass uprisings brings grave danger. The idea is to make this app “check-point-proof”, meaning that a somewhat knowledgeable person will not detect the presence of the app and will not discover any video content. The app itself should be hidden, you can make a “stealth” app by somehow removing the app icon from your app list (sadly it simply still shows up in the uninstall app list). The app is activated simply by “dialing” a secret telephone number or other method your deem secure. Starting point for your work can be found here: http://stackoverflow.com/questions/5921071/how-to-create-a-stealth-like-android-app. Your Stealth app need to be able to virally spread and be able to bypass an government restrictions on the official app store. Include the feature for NFC and direct-wifi transfer of the .apk with an easy on-screen manual and steps. Thus users can pass your app along to their friends.

NAT Traversal: 80% success rate

Peer-to-Peer (P2P) networks work on the presumption that all nodes in the network are connectable. However, NAT boxes and firewalls prevent connections to many nodes on the Internet. We created a method to puncture NATs which does not require a server. Our method is therefore a simple no-server-needed alternative to the complex STUN, TURN and ICE approaches. We conducted one of the largest measurements of NAT/Firewall behavior and puncture efficiency in the wild. Our method is a UDP hole-punching technique. We measured the success rate using volunteers running Tribler. Number of users in our trials are 907 and 1531 people. Our results show that UDP hole punching is an effective method to increase the connectability of peers on the Internet: approximately 64% of all peers are behind a NAT box or firewall. More than 80% of hole punching attempts between these peers succeed.

Brief description of our UDP puncture method in IETF draft

Jenkins test of NAT puncturing in Dispersy: http://jenkins.tribler.org/job/Experiment_NAT_1H/2/

Lengthy thesis work on UDP puncturing from 2005

Roadmap 2030: a proven alternative model for capitalism

As Tribler scientists and engineer we are actively trying to make a better world. Our micro-economy is our living lab for experimenting with alternative models for capitalism. We aim to re-invent money by creating the first sustainable economy without any moral hazards from bankers, politicians, and megacorporation. Citizens and only the citizens are in control with self-governance.

Our grand vision in a 1+ hour lecture given at Stanford University, via their Youtube channel. We want to do more then be a Youtube alternative. Our grand vision is liberating both media and money. See the talk Abstract and slides (.pdf 78 pages). Keywords: transform money, “Bank-of-Bits”, global financial meltdown isolation. Use cooperation&stability, not volatility&greed. Alter the essence of capitalism (rich get richer) by abolishing compound interest rate and facilitation of safe zero-cost money transfers & lending. We aim for a direct assault on the essence of capitalism, aiming even further then the Bitcoin accomplishment (bypassing the central bank).

Further reading:

Tribler history

  • 2018: Release of Tribler 7
  • 2017: First live tests with decentral marketplace
  • 2016: New blockchain deployment testing
  • 2014: Test network goes live for anonymous Tor-like downloading (not connected in any with with 'the' Tor project)
  • 2013: Anonymous Tor-like download trial
  • 2012: Tribler Mobile live streaming from a phone camera
  • 2011: Libswift accepted as an upcoming IETF Internet Standard
  • 2010: Wikipedia.org uses our technology for live trial
  • 2009: Large HD streaming trial with BBC
  • 2008: Social network without servers and "easy" invites
  • 2007: Our distributed ledger launched in the wild
  • 2006: Tribler 1st release
  • 2005: First Tribler code = social Bittorrent
  • 2004: Slashdot for first time with largest Bittorrent study
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.