Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Google Summer of Code 2016
Table of Contents
This page lists project ideas for Google Summer of Code 2016.
The tasks are split into categories, sorted from high level (web interface, plugins, applications) to low level (crypto, frameworks, transport layer).
- Improve workflows in the web interface
- There is much to do to make it more user friendly. There many small annoyances linked from this bug report. These hurt usability, but the list is not exhaustive: you will find more by simply using the interface and fixing what irks you. A recent but not very detailed, professional mockup focusing on the homepage is here: https://emu.freenetproject.org/freenet-ui-mockup-pupok-1.pdf.
- Get the web-pushing mode working well
- A previous Summer of Code student built the web-pushing ajax/comet/GWT system, which does things like showing progress on pages with lots of images to speed things up when browser connections are limited, and live updating of the downloads page etc. This is rather buggy especially on slow browsers, and tends to lock up, so fixing this, improving it and then extending it to other areas (e.g. Freetalk) would be interesting. Obviously rewriting the whole web interface with a different toolkit would need a different Comet solution, but we'd like to have something like this.
operhiem1 addressed the infrastructure parts of this project in Google Summer of Code 2013 by extending the Infocalypse plugin. He simplified sharing code anonymously to: `hg clone freenet:ID/reponame; cd repo; (hack); hg push freenet:MyID/reponame; hg fn-pull-request; hg fn-check-notifications`
But discovery of repositories inside freenet is pretty hard. This is the part which is still missing to create a decentralized, censorship resistant, anonymous GitHub.
Thus as followup, this project would be turning the existing infrastructure into a polished workflow which makes hacking freenet in freenet enjoyable.
- Add discovery of repositories and connections between repositories by browsing the Web of Trust: “find forks of my repo in freenet”. Offer the option `hg paths --add-forks` (or similar), which shows all public forks in freenet.
- Create freesites with information about the repositories. See the hg site extension for an example.
- Enhance the DVCS web interface plugin to provide nice repository browsing capabilities.
- Add automatic tests for selected collaboration workflows.
- Add communication features, discussion of pull-requests and optionally other parts needed for a full fledged community hacking platform.
- Optionally integrate Infocalypse with the keepalive plugin to improve the longevity of repositories.
The current site is working reasonably well, but it is worlds apart from sites like Spacemacs, MediaGoblin or GNU Guile. Experience from the GNU Guile shows the large impact such a change can have. A core requirement of this project is that at the end of the GSoC, the new site must be deployed and enjoyable.
Translation support can be improved - currently it's not usable with Pontoon, and the download page is currently pretty awful to look at.
- Improve Sone
- Sone is a social-networking style system over Freenet. One way to improve it is to make it automatically insert bundles of posts periodically, to avoid having to download all the posts every time. Other problems relate to the question of scalability: When you can't follow everyone, what does that mean for the user interface? How do you do keyword searches? Etc. There have been ideas on this. Talk to Bombe...
- A good filesharing/file search system
- This should tie in with the Web of Trust, allowing users to publish indexes and search those of their anonymous friends, rate others' indexes, merge them into their own, set up long-term file searches, preload indexes for faster searches, and so on. It might also integrate with Freetalk to help with discussions on labelling or rating. The problems of spam/deliberately corrupt content are very similar on Freenet to on traditional p2p, although the solutions may be different, especially as it isn't possible to trace spammers; trusted community maintained indexes have developed as a working means of solving these problems on web-based filesharing. Note that we already have a scalable forkable on-freenet btree search system to use as a backend, but it is not yet used for anything, and it is not distributed or WoT-compatible.
- Integrate filesharing with other networks, using a distributed, WoT-based way to download data by conventional hashes rather than CHKs; this is also related to the wierd stuff (backups) at the bottom.
- Secure reinsert-on-demand filesharing, to improve the volume of content that is available. This is a lot harder than it sounds, but in any case we need searching first. (Reinserts using the same keys are seriously insecure, although getting the downloaders to do random inserts may help significantly)
- Keepalive plugin improvements
- Some very small changes at the node layer will allow probing for whether a key is available from a random node. Users could then maintain a list of content that they care about; the plugin would download it all as a binary blob, regularly probe for reachability, and reinsert it when needed (possibly single random not fetchable blocks). For this to work really well we might need selective reinsert support in the client layer, but that's not necessary for a basic implementation. What is important is supporting both files and sites, and letting users publish their lists and subscribe to others. This could even evolve into a full blown distributed backup system, with configurable encryption for files and the ability to recognise which files are popular and avoid inserting them (might need fixing GetCHKOnly).
- A microblogging and/or real-time chat system: Both of these things would actually be implemented in a fairly similar way. Evan has done a fair amount of work on how to efficiently implement microblogging over Freenet. Sone does something like this but is fairly greedy with network resources. Flip does IRC. The fritter-proposal would be a good starting point.
- Easy-to-use tools
- We need good, simple tools for inserting freesites (freenet-hosted web sites) and files, and maintaining mirrors of web content.
- Better freesite searching
- Lots of work has been done on this, but more could be done: Using the new library format, rewriting the indexes on the fly after gathering a few hours' data rather than writing it from the database over a week, support for long-term searches, web of trust integration, better support for stop-words (maybe aggregating them with common before/after words), tokenisation for tricky languages (Chinese, Japanese), distributing spidering across multiple users (as scaling is getting to be a serious problem now), etc.
- Wiki over Freenet
- A widespread wiki over Freenet would be really awesome. There have been attempts, but it is not easy to implement, as there are several challenges. You can learn more there. There have been many attempts; some are hard to use and based on DSCMs, some are easier to use and not scalable.
- WebOfTrust optimization
- Currently the Web of Trust plugin is quite slow. Optimizations on the database level and on synchronization strategies should offer the chance it much faster. A first step would be profiling to find the real performance bottlenecks. Care will be needed to ensure using scalable algorithms. There is an alternative Web of Trust implementation called LCWoT, but it is incomplete. Note the next item.
- More content filters
- More simulations
- There is lots that can be simulated: Does swapping scale? Does the proposed fix for Pitch Black really work? Do darknet and opennet get on together? In particular, there are a range of attacks on requests that have been proposed, it would be great to quantify them. Load management is another huge area where simulations (and expertise!) would be helpful.
- Tunnels and so on
- Somebody with some expertise in tunnels would be very useful. Freenet needs an optional mixnet layer for better security, especially on inserts (where it wouldn't cost us much performance because we go lots of hops anyway). We have some ideas (something based on Pisces perhaps?)... This is a more research-oriented project but we would like to have some sort of basic tunneling option available by the end of it.
- Much more transport layer stuff
- The current transport layer has been improved significantly but leaves much room for improvement. Ideally we'd like to detect available bandwidth automatically. We have nothing remotely like Path MTU detection; we should automatically adapt to find the right packet size for the connection, both finding what will work at all, and what gives the best bandwidth/loss tradeoff. We tend to get transfer failures on slow connections (not low bandwidth limit, low bandwidth available on that specific connection). We probably should use something like cumulative acks, currently all packets are acked once, it should be possible to ack the same packet twice with 0 bandwidth cost in many cases using ranges. We may want to divide up blocks differently depending on how fast the connection is. We may want to make tradeoffs between fairness to all peers (the current policy) and allowing individual peers more bandwidth for a short period (e.g. because they have requested a bunch of fproxy pages), or have "idle priority" traffic which is only sent when *no* peer wants to send anything (e.g. bloom filter sharing), which may also impact on packet size. And so on. Generally, the transport layer needs to be more robust, especially on slow connections, and it needs to feed information into the load management layer more quickly so that we only accept requests that we can complete in a reasonable time, given the current state of the connection. There are various bugs about this on the bug tracker. Running as well as possible on fairly slow connections is particularly useful in some of the places where Freenet may be needed most.
- Chetan's work in 2012 means we will soon have a general layer for packet-type transport plugins, but we won't have any actually working yet. We would like support for stream-based transport plugins, and a bunch of working transport plugins, for both making Freenet work behind nasty firewalls, and for steganography. Some, such as something that looks like Skype traffic, will impose particular transport-layer problems due to e.g. packet sizes.
- Different bandwidth limits at different times of the day/week would help many users. We also need support for monthly transfer limits (separate from the existing peak per second limits), and autodetection of the connection's capacity, possibly with options for very latency sensitive behaviour for e.g. gamers (like some Bittorrent clients do). All this must be really easy to use.
- Better darknet
- Make it easier to invite people to Freenet, provide software bundles, provide a simple password-and-IP-address connect protocol, connect to friend-of-a-friends, make it easy to connect to mutual friends, etc. Lots of ideas about this on the bug tracker. Some of it has been implemented in a branch (foaf-connections).
- More F2F functionality
For some projects in this section, the best solution would simply use an external library to do most/much of the work, and simply integrate it into the current freenet code. So you should do some research into existing solutions first. This principle applies to all projects, but the ones where suitable libraries are most likely to actually exist, are tagged prefer-external.
SSKs currently use our own implementation of DSA. This is old, and the pubkeys are huge. We can switch to ECC, using the Java Cryptography Architecture.
- Make SSKs use ECC for signatures, via JCA (see below).
- Get NSS working semi-automatically. For ECC this is a big performance gain.
- Take advantage of the smaller size of ECC-based SSKs' pubkeys: optimise SSK requests and inserts (fewer packets etc).
- Take advantage of the smaller size of ECC-based SSKs' pubkeys: optimise datastore usage (single store for SSKs and pubkeys), and allow keys of ~ 800 bytes (fast, for e.g. IRC over Freenet) and ~ 2KB (slower, for e.g. forums). Automatic, safe migration, even if there are disk space issues, is one of the bigger problems here.
- Implement 32KB SSKs as well, if we're looking at the datastore.
- Possibly implement Programmable Subspace Keys. The basic principle is to allow to specify a set of rules (in a simple language) that determine whether or not a block is valid for a given key. The simplest application of this is a way to allow multiple posters to use a single SSK (greatly increasing the efficiency of a spam-proof freenet-based chat system), while still requiring them to sign their own posts, allowing the "moderator" to change the key to get rid of a spammer.
This project is about changing how we do cryptography in freenet. At the moment, we use our own-implementation of standards... This project is about switching to JCA and BouncyCastle.
JCA has cryptographic strength restrictions, and even for US users resolving this is nontrivial (the java auto-updater wipes out the unlimited strength policy files every time you update!), so we will need to implement some internal API that can wrap either our built-in crypto, Bouncycastle's lightweight API, or an accelerated JCA implementation. This is probably a good idea anyway, to make the code clearer and limit the possibilities for accidental abuse of cryptographic primitives. In particular, crypto setup in connection setup should be factored out into a separate class, and we should consider using third party libraries for parsing the connection setup packets (and possibly for parsing messages too, the same library could do both tasks).
Another problem is that native implementations are often slower, because of the overhead of calling C code through JNI. This may not be true on all hardware (e.g. ATOM, ARM), so we need to benchmark it on startup (this is partly implemented already). For security reasons it is preferable to use JCA, or at least Bouncycastle, where possible, if there is no performance issue.
The major goals of this project are:
- Review and question the usage of cryptography thorough the whole codebase
- Build an internal, high level crypto API, which wraps either Bouncycastle lightweight API or JCA.
- Use this instead of our own code, reducing the amount of crypto code we need to keep in fred.
- Benchmark or profile hot codepaths and propose enhancements
- Enable users to use crypto-accelerators where possible (for x86 systems this probably means asymmetric i.e. ECC; symmetric crypto appears to be slower even with AESNI because of the JNI overhead).
- Write unit tests to ensure some form of backward compatibility with the existing code (there are some already, more would be good)
There is an official update channel, and while it's possible to set up alternative update channels, it's not particularly flexible. It could be good to have support for subscribing to multiple channels: a testing update channel, a stable one, and feature-branch channels. See the (incomplete) update channel RFC. This would include both continuing to refine the design, and implementing initial stages of it, so things like:
- Supporting multiple update channels both in the backend and through the settings UI.
- Build script support for channels - making it easy for an official testing channel or for developers to post custom builds.
- Evaluate or prompt for which build to use given multiple available updates.
freenet uses a custom-written framework for running callbacks and scheduled tasks, which is non-standard and lacks a lot of advanced functionality. Converting everything to use java.util.concurrent and/or com.google.common.util.concurrent would help this a lot. Ideally, the framework would be separated from the rest of freenet, possibly even to the point of being an external library, and provide a clean API for freenet and its plugins to use. This would also allow plugins to do much more advanced tasks (e.g. most of the things in the "plugins and applications" section further below) without having to implement their own framework.
The events framework should be able to, and provide a good API for plugins to:
- run tasks in the background, or according to a particular schedule
- define task+thread matching policies e.g. all database tasks must run in the "database" thread
- cancel running tasks, if the task defines that it's cancellable
- handle dependencies between tasks, so that e.g. if A depends on B and we cancel/complete B, A is automatically cancelled/completed.
- attach callbacks to events such as task completion/cancellation (e.g. using com.google.common.util.concurrent.ListenableFuture)
- group tasks into related categories so one group doesn't affect the other. (e.g. tasks for the group "Library/b-tree-write/index-" won't starve the resources of group "WoT/background-trust-calculation") (this is possibly logically equivalent to the "matching policies" point above but I haven't thought too much about it)
Currently, config management code is very untidy. Code for read/writing the config in embedded inside the classes they control, which is spread out over many files not related to config management. This creates lots of implicit dependencies, which is not directly apparent in the syntax/structure of the code nor the semantics of the language. (In other words, spaghetti code.)
Config code should be separated from the actual application. It would also be nice if there was an API to expose this to plugins.
The config system should be able to:
- support everything the current config system supports
- use "includes" to separate different sections into different files - e.g. keep vital system-specific settings in a separate read-only file
- read and write values, preserving comments in the file.
- With regards to mutability, there are three types of values:
- mutable (takes effect after a restart)
- mutable (effects the running daemon immediately)
- Note that the actual code for doing the setting should be implemented outside of the config system.
- With regards to mutability, there are three types of values:
You can sign up between March 10th and March 21st on the Google Summer of Code page.