Plugin Development Tutorial
NOTICE: An more recent guide is available at freenet-plugin-bare-guide
So you want to write a Freenet app...
In this tutorial you will learn how to write a Freenet application that can do the following things:
- Integrate your application in a Web ui
- Add data to the freenet network
- Request data using cryptographic keys from the network
- Communicate with other Freenet software
- Sufficient knowledge of Java
- A working brain
- fred - the freenet daemon (network communication, crypto, io)
- fproxy - the freenet web interface
- insert - add content to the network
- fetch/download - retrieve content from the network
- node - a specific instance of the freenet software that runs on a computer
- store - the locally stored encrypted data on a specific node
- network - This is used to refer to BOTH the total storage capacity of the network AND the nodes that make up the network
- CHK - a URL/URI to data that is automatically derived from the hash of the data
- SSK/USK - a stable public-private-keypair that points to data
- edition - a version number of the data that a SSK/USK points to
- freesite - a static site inside Freenet
The main class is the entry point in your Freenet application. Think of it as the class that contains your 'public static void main(String[] args)'-method. In this tutorial I will assume that you want to create the class 'MyApplication' in the package 'hello.world'.
Freenet provides several APIs that you can use to write your application. The first one is the FredPlugin Java interface and requires you to implement a runPlugin and terminate method. Let's give you some code to start with:
package hello.world
public class MyApplication extends FredPlugin {
private final static Logger LOGGER = Logger.getLogger(MyApplication.class.getName());
//the entry point to your application, the first method that is called when loading your application in Freenet
public void runPlugin(PluginRespirator pr)
{
}
//called when your plugin is shutdown. This is called when reloading/unloading your plugin or stopping Freenet on your computer
public void terminate()
{
}
}
As you can see, the PluginRespirator object is passed to the runPlugin method. The PluginRespirator object allows your application to access the Freenet internals, such as cryptographic functions and the ability request information from the datastore.
Let's add some code that adds your application to the Freenet web interface menu:
package hello.world
public class MyApplication extends FredPlugin {
private final static Logger LOGGER = Logger.getLogger(MyApplication.class.getName());
//the entry point to your application, the first method that is called when loading your application in Freenet
public void runPlugin(PluginRespirator pr)
{
setupWebInterface();
}
//method called when your plugin is shutdown
public void terminate()
{
}
private void setupWebInterface()
{
// TODO: remove
PluginContext pluginContext = new PluginContext(pr);
this.webInterface = new WebInterface(pluginContext);
pr.getPageMaker().addNavigationCategory(basePath + "/","WebOfTrust.menuName.name", "WebOfTrust.menuName.tooltip", this);
ToadletContainer tc = pr.getToadletContainer();
// pages
Overview oc = new Overview(this, pr.getHLSimpleClient(), basePath, db);
// create fproxy menu items
tc.register(oc, "WebOfTrust.menuName.name", basePath + "/", true, "WebOfTrust.mainPage", "WebOfTrust.mainPage.tooltip", WebOfTrust.allowFullAccessOnly, oc);
tc.register(oc, null, basePath + "/", true, WebOfTrust.allowFullAccessOnly);
// register other toadlets without link in menu but as first item to check
// so it also works for paths which are included in the above menu links.
// full access only will be checked inside the specific toadlet
for(Toadlet curToad : newToadlets) {
tc.register(curToad, null, curToad.path(), true, false);
}
// finally add toadlets which have been registered within the menu to our list
newToadlets.add(oc);
}
}
public abstract class FileReaderToadlet extends Toadlet implements LinkEnabledCallback {
protected String path;
protected String filePath;
public FileReaderToadlet(HighLevelSimpleClient client, String filepath, String URLPath) {
this.path = URLPath;
this.filePath = filepath;
this.db = db;
}
//this handles an HTTP GET to the path exposed in the method path()
public void handleMethodGET(URI uri, HTTPRequest request, ToadletContext ctx) throws ToadletContextClosedException, IOException {
writeHTMLReply(ctx, 200, "pageContent", "Hello world!");
}
//the relative URL that this page is at (i.e. /my_app/mypage)
@Override
public String path() {
return path;
}
}
The code above adds a new navigation category to the main menu and adds the page controlled by the class Overview to that navigation category. It will look something like this:
TODO: include screenshot of the end result
In order to transform your code into a working piece of Freenet software one final thing is required. You need to include 1 line of metadata in your jar. You can/should do this automatically using Maven or Ant, but for the purposes of this tutorial, we'll do it manually. Create a folder META-INF in the root of your project, create a file called XXXX and add the following line to the manifest: *Plugin-Main-Class: hello.world.MyApplication* This tells Freenet where the runPlugin() method is located. Compile your code and create a jar of the resulting class files.
In order to load your plugn into Freenet. Go to fproxy (the Freenet webinterface), click on plugins in the menu, move to the bottom of the page. In the textbox, enter the full path to the jar that you have created and press `load'. In a matter of seconds you'll see your plugin in the webinterface and you can click on it. Hooray!
In order to update your plugin after code modifications: 1) Overwrite the jar on disk, 2) Press the reload button next to your plugin in the plugin overview list, 3) make sure to tick the checkbox such that Freenet disregards the cached copy of your plugin jar.
Now that we have the bare bones of a plugin, let's actually insert content that Freenet will automatically encrypt and distribute throughout the network. We will continue to use the existing Java code that you have created previously.
Let's say that you have an important text file that you want to insert into Freenet *programmatically*. NOTE: inserting files is easily done using the Freenet web ui, you don't need to write a program to do that. However, for the purposes of this tutorial; let's keep it simple.
The text file that you want to insert needs to be written to a Bucket. A Bucket is just a temporary holding place for your data while Freenet is inserting it or fetching it from the network. You can access a bucket as a regular OutputStream or InputStream or in order to write or read data. Let's look at some code:
final String content = "Free speech is important! Let's do something to protect it!"
final Bucket bucket = pr.getNode().clientCore.persistentTempBucketFactory.makeBucket(content.length());
bucket.getOutputStream().write(content);
bucket.setReadOnly();
FreenetURI nextInsertURI = new FreenetURI("SSK@cuohbhuobho4dyg4y8,g5p,");
//wrap the data and where you want to put it together
final InsertBlock ib = new InsertBlock(bucket, null, nextInsertURI);
//the object that will allow you to insert metadata about the insert that is not in the data itself.
//for example: a MIME/type
final InsertContext ictx = hl.getInsertContext(true);
ClientPutter pu = hl.insert(ib, false, null, false, ictx, this, RequestStarter.IMMEDIATE_SPLITFILE_PRIORITY_CLASS);
In the code above we've inserted a key to a CHK (the key to the data is based on a hash to the content). That means that if the content changes, the key you use to access it changes. If you want to have the same key for different versions or updates of your content use an SSK (a public-private-keypair, where the public key is the identifier to your content and is stable accross updates). Last, but not least: the USK. It is a special type of SSK, but with the added functionality that it is versioned with numbers, i.e. 1,2,3 ... every time you update the content. Freenet has the ability to automatically find the latest edition of a USK.
NOTE: the 6th argument passed to hl.insert() is a callback object that implements the ClientPutCallback interface. Freenet will asynchronously call methods on that object to inform your application on the progress of the insertion process.
Let's look at a very similar insert example with a newly generated USK:
//get the key management stuff to generate the SSK keypair
//upgrade the SSK to a USK (a USK is an SSK with the added advantage that Freenet can automatically find the latest version of it)
FreenetURI nextInsertURI = new FreenetURI("SSK@cuohbhuobho4dyg4y8,g5p,");
//wrap the data and where you want to put it together
final InsertBlock ib = new InsertBlock(bucket, null, nextInsertURI);
//the object that will allow you to insert metadata about the insert that is not in the data itself.
//for example: a MIME/type
final InsertContext ictx = hl.getInsertContext(true);
ClientPutter pu = hl.insert(ib, false, null, false, ictx, this, RequestStarter.IMMEDIATE_SPLITFILE_PRIORITY_CLASS);
NOTE: Programming in Freenet is very often asynchronous, so although the hl.insert() method returns immediately. Some time passes before Freenet can tell you whether the insert was a success or a failure. In order to receive this information the ClientGetCallback needs to be implemented on the object passed to hl.insert().
package hello.world.MyApplication
public class InsertCallbackStatus implements ClientPutCallback
{
@Override
public void onSuccess(BaseClientPutter cp, ObjectContainer oc) {
try {
System.out.println("We inserted the content! You can get it at: " + cp.getURI().toASCIIString());
}
finally
{
Closer.close(((ClientPutter) cp).getData());
}
}
@Override
public void onFailure(InsertException ie, BaseClientPutter cp, ObjectContainer oc) {
try
{
System.out.println("Failed to insert content into Freenet!");
}
finally
{
Closer.close(((ClientPutter) cp).getData());
}
}
}
TIP: You can fetch the content by copy-pasting the URI reported by your application into fproxy like so: http : / /localhost:8888/CHK@9hu.ocuhotub.oh . Try recompiling your application and test that you can actually insert content into Freenet and fetch it again using fproxy.
NOTE: Depending on the load of the network, actually inserting a small amount of data can sometimes take more time than expected. Please be patient, Bug us if it's a ridiculuous amount of time though.
Fetching content is conceptually similar to inserting content. However, instead of creating a bucket, filling it with data and then inserting it to a key the process is reversed. Here is a code snippet that shows you how easy it is to fetch data in Freenet programmatically:
private void startFetch()
{
int MAX_FILE_SIZE_IN_BYTES = 1000;
RequestClient rc = ...
ClientGetCallBack cc = ...
FetchContext fc = ...
hl.fetch(key, MAX_FILE_SIZE, rc, cc, fc)
}
The hl.fetch() method call also returns immediately and progress on the fetch is reported to the callback object you have passed (cc). The callback object needs to implement the ClientGetCallback interface in order to have the required methods.
public class FetchStatusCallback implements ClientGetCallback {
@Override
public void onMajorProgress(ObjectContainer oc) {
System.out.println("Something happened!");
}
@Override
public void onFailure(FetchException fe, ClientGetter cg, ObjectContainer oc) {
System.out.println("")
if (fe.mode == FetchException.PERMANENT_REDIRECT)
{
System.out.println("The key no longer points to the content, but it tells me I should look for it at this other key: " + fe.newURI);
}
}
@Override
public synchronized void onSuccess(FetchResult fr, ClientGetter cg, ObjectContainer oc) {
{
System.out.println("Fetch succesful!");
System.out.println("This is the content that I fetched for that key: ");
//TODO: improve
new ByteArrayInputStream(fr.asByteArray())
}
}
NOTE: This tutorial only covers fetching simple keys that consists of a single file. Freesites can also be fetched using similar methods, but requires parsing of their meta-data, because a single Freesite can contain *multiple* files. This functionality isn't part of this tutorial.
Congratulations! You are now able to write Freenet software that inserts and fetches content from the network.
Freenet plugins can also talk to eachother using the FCP-protocol. In this tutorial we will look at communicating with an important Freenet plugin: the WebOfTrust. The WebOfTrust takes take of managing anonymous online identities and calculates trust based on links between identities to prevent SPAM and other attacks. If you haven't installed the WebOfTrust plugin yet, please do so now before reading any further.
Plugins talk to eachother via the exchange of so called FCP-messages. As with everything in Freenet; it is an asynchronous operation. Messages are sent and plugins reply when they feel like it. In order to send an FCP-message to a plugin you need two things. 1) The full package name and class that is able to receive your FCP message. 2) A map with the required key-value pairs of the specfic message that you are sending.
WARNING: Freenet uses a custom implementation of a HashMap that is (deceptively) called a SimpleFieldSet. It does a lot more than a simple HashMap, but for all intents and purposes you can just consider it to be a Java HashMap.
Let's start setting up the machinery to send an FCP message to the WebOfTrust-plugin:
PluginTalker talker;
try {
talker = pr.getPluginTalker(this, "plugins.freenet.WebOfTrust", "WoT");
} catch (PluginNotFoundException e) {
}
//get a list of sone ids
SimpleFieldSet sfs = new SimpleFieldSet(true);
sfs.putOverwrite("Message", "GetLocalSones");
sfs.putOverwrite("Identifier", "startup");
talker.send(sfs, null);
That's it. Now, in order to receive the plugin's reply you need to create a class that implements the Java interface FredPluginTalker and add a method in order to receive the message that the WebOfTrust sends you with the data you requested in a SimpleFieldSet.
class MyApplication implements FredPluginTalker {
@Override
public void onReply(String longid, String shortid, SimpleFieldSet sfs, Bucket binaryData) {
try
{
if (sfs.get("Message").equals("Error"))
{
LOGGER.severe(sfs.get("ErrorMessage"));
}
if (sfs.get("Message").equals("ListLocalSones"))
{
for(int i=0; i < sfs.getInt("LocalSones.Count"); i++)
{
//The name of the nth identity is BLA
}
}
}
catch(FSParseException ex)
{
ex.printStackTrace();
}
}
}
That's it! You are now an elite Freenet-certified free speech crypto god! Congrats!
The Web of Trust plugin is suitable for basing many client applications on. It provides the ability for users to generate pseudonymous "identities". These identities can assign ratings to each others with the purpose of spam filtering. If someone posts spam, the other users can assign a negative rating to his identity. The linked wiki page explains this in depth: Web of Trust.
WOT allows other plugins to obtain identities from it. This can give your plugin a trusted environment of users who don't spam. There is a page which tells you how to do WOT-based development: Web Of Trust development.