Freenet Utility Scripts
freenetrc-sample for script configuration.
Anything which is useful for the development of freenet, which isn't an app, a plugin, or part of Freenet itself.
Obviously the actual private keys etc won't be included...
Note that some of these scripts are supposed to run as different users:
- Most stuff involved in releasing a build is on one user.
- Some of it is on a different user.
- grabbackup is on a third user (which is just used for backups).
You will also need ssh access to osprey.
Verifying Fred builds
Currently the official builds are made using Debian Wheezy, and verify-build verifies only freenet.jar. It does not verify freenet-ext, libraries, source archives, or installers.
lib-pyFreenet is used for
fcpget to fetch and verify the freenet.jar inserted into Freenet.
# apt-get install git-core python openjdk-7-jdk ant unzip $ git clone git://github.com/freenet/scripts.git $ git clone git://github.com/freenet/fred.git $ git clone git://github.com/freenet/lib-pyFreenet.git $ cp scripts/freenetrc-sample ~/.freenetrc $ scripts/set-freenetrc-base $ mkdir -p FreenetReleased/dependencies $ wget https://downloads.freenetproject.org/alpha/freenet-ext.jar -O FreenetReleased/freenet-ext.jar $ wget https://www.bouncycastle.org/download/bcprov-jdk15on-154.jar -O FreenetReleased/dependencies/bcprov-jdk15on-154.jar $ wget https://freenetproject.org/assets/keyring.gpg -O freenetkeys.gpg $ gpg --import freenetkeys.gpg $ cd lib-pyFreenet # python setup.py install
Now everything should be installed and sufficiently configured. Make sure a Freenet node is accessible by FCP.
$ cd ../scripts $ ./verify-build
Merging localization files
MergeSFS utility is in the Fred source tree:
java -cp freenet.jar freenet.tools.MergeSFS src/freenet/node/l10n/freenet.l10n.ja.properties freenet.l10n.ja.override.properties
This will write the changes in the override file to the source file. To write the merged values to standard output instead, use
--stdout as the third argument.
Ensure that the plugin is at a signed version tag. Run
release-plugin [plugin name]. This requires that there is a repository of that plugin (or a link to one) at
$freenetRoot/plugin-[plugin name]. Update the key in the plugin manager, and test that the plugin loads correctly. Do not increment the mandatory version for connection-essential plugins like UPnP: they will not be loaded and the new version cannot be fetched without a connection.
Sign the commit that upates the key (
--gpg-sign) and mention the tag the
plugin is built from:
Update $pluginName to $tag Built from the tag $tag. ...
Releasing testing Freenet builds
Insert the jar to a USK an edition beyond the build number being upgraded from.
This can mean inserting as an SSK:
Set the auto-update key in Configuration > Auto-update in advanced mode to
USK@.../testing-jar/editionNumber and restart the node to apply the setting.
Releasing stable Freenet builds
To upload things to the FPI webserver requires SSH access and membership (
usermod -aG [group] [user]) in the
To set up a release environment, set up freenetrc, then (read through and) run
setup-release-environment. This will install required packages (under Debian
Wheezy), clone the repositories, and download the IP to country database.
This should be kept up to date - maybe a cronjob would be good. Note
that this should not be at the start of an hour
lest their server be overwhelmed.
Performing a release requires:
push access to
SSH access to the FPI webserver. (Osprey) Take care to set a host
~/.ssh/configentry if you need a different username.
(Encrypted) auto-update insert key at the location set in
freenetrc. This should contain
NEWKEY=SSK@...,...,...(bare SSK - no name or trailing slash. Surrounding the value with double quotes is optional.)
A published gpg keypair.
A copy of the
seedrefsdirectory from another release manager. (If updating seednodes.fref)
Dependencies such as plugins either built or downloaded, (listing of loadedFrom() CHKs) into FreenetReleased. Repositories using these files can be set up with symlinks.
A GitHub OAuth token with
public_repoaccess set in
A jarsigner certificate. This can be a self-signed one, though once (or if) one exists for FPI one should use it. See here. For example:
keytool -genkeypair -keyalg RSA -sigalg SHA256withRSA -keysize 4096 -dname "cn=Robert Releaserson, o=The Freenet Project Inc, c=US" -alias freenet -storepass SomePassphrase -validity 365 -keystore ~/.keystore
jarsignerPasswordto the alias and store passphrase, respectively.
For the Windows installer: see the list in the header of
release-wininstallerassumes locations in
FreenetReleased/dependencies. In the case of configuration problems or prompts it might be useful to also install
release-wininstaller --dry-run --xnestfirst.
The FPI code signing certificate set up as though with
keytool -importkeystore -srckeystore code-signing.p12 -srcstoretype pkcs12 -srcalias "freenet project inc's comodo ca limited id" -destkeystore ~/.gnupg/freenet/code-signing/code-signing.jks -deststoretype jks -destalias freenet -destkeypass "password"
The node that updates are inserted on cannot be set to log heavily, lest the update insert key be leaked into logs. freenetrc defaults to using port 9481 for inserting updates under the assumption that the newly installed testing node is on the port. If it is a node with heavy logging, change fcpUpdatePort. TODO: The more thorough solution to this would be to check that the logging level is sufficiently low over FCP.
Also take care that the unencrypted key is not written to disk - disable
swap (or have it encrypted with a random key) and mount
To test auto-updating, useful while setting things up:
- Set up a node of the previous version. (The new one is the one being deployed currently.)
- Set the FCP port to the expected one.
- Make the update insert to local storage at higher priority for speed: Include
LocalRequestOnly=trueand the line before
PCLASSis used set it to 0.
- Generate a new SSK keypair.
- Set up a configuration file with the insert URI.
- Set the node's autoupdate source to
- Restart the node.
- Run the insert-update script.
- The node should find, download, verify, and apply the update.
Ensure the fred, scripts, website, Java installer, and Windows installer repositories are up to
date before starting the release. Ensure the bundled plugins in the
(or equivalent) directory are up to date as well.
The first release with an environment, run these steps individually to get all the configuration right. Once everything works, run
release-build [build number], which automates running these steps:
update-bookmarksupdates the default bookmark editions and commits if any changes have been made.
update-versionupdates the version number, and prompts for and updates the mandatory dates if applicable.
tag-build [build number]tags a build and prompts for a changelog.
release-fredbuilds the Fred jar and source archive, signs them, creates l10n diffs, and uploads all to osprey. If used with
--snapshotinserts the Fred jar and signature into Freenet.
release-installerbuilds the Linux/OS X installer and uploads it to osprey.
release-wininstallerbuilds the Windows installer and uploads it to osprey.
java -jar [location of released jars]/new_installer_offline_[buildnumber].jarruns an installer. The release manager should test installing a node both with the Linux / OS X installer and the Windows one. It should be able to bootstrap successfully, access FProxy, and otherwise have no obvious problems.
upload-assetsuploads the jars and installers to GitHub which serves the majority of downloads.
deploy-website, when run from osprey, updates the website to point to the latest version as defined by the given
fredrepository. The script's
-uswitch updates both
website, so if one wants to avoid pulling in website changes as well it may be preferable to manually update the
fredrepository only, or use the
insert-updateinserts the jars over FCP. This is intended to be the test node which was installed by testing the installer as above. This is so that a development node which may have heavy logging does not leak the keys into the logs.
Once the inserts have completed announce the release: FMS, IRC (including the channel topic), devl, support, website, Wikipedia article.