This tool uses the RRDP data from a (single) "hidden" backend RPKI Publication Server to make a consistent local copy of that data.
This is intended to facilitate a redundant set up where one or more public https and rsync servers are used to make the RPKI repository content available.
This release includes fixes for the following issues:
- Support empty snapshot (#62)
- Increase cleanup-after default to 1 hour (#54)
- Write new notification file only on change (#55)
- Respect --insecure option (#59)
The code has had a massive overhaul since version 0.1.x. It has been simplified and has much better test coverage.
Functionality should be compatible to the known requirements for 0.1.x - but the CLI arguments have also changed. If you are looking to upgrade from 0.1.x to this version, please make sure that you read the updated text below and test things thoroughly. In case you run into any issues, do not hesitate to contact us or make an issue in GitHub.
In this version we added the following:
- Allow a configurable delay before writing the RRDP notification.xml file (#5)
- Use a symlink for the current rsync directory (#2)
- Debian packages (#11)
- Docker file (#9)
We did not make official releases, nor dir we keep any changelogs for earlier versions.
Install using Pre-built Packages
Assuming you have a machine running a recent Debian or Ubuntu distribution, you
can install krill-sync from our software package repository.
To use this repository, add the line below that corresponds to your operating system to
deb [arch=amd64] https://packages.nlnetlabs.nl/linux/debian/ stretch main deb [arch=amd64] https://packages.nlnetlabs.nl/linux/debian/ buster main deb [arch=amd64] https://packages.nlnetlabs.nl/linux/debian/ bullseye main deb [arch=amd64] https://packages.nlnetlabs.nl/linux/ubuntu/ xenial main deb [arch=amd64] https://packages.nlnetlabs.nl/linux/ubuntu/ bionic main deb [arch=amd64] https://packages.nlnetlabs.nl/linux/ubuntu/ focal main
Then run the following commands.
wget -qO- https://packages.nlnetlabs.nl/aptkey.asc | sudo apt-key add - sudo apt update
You can then install krill-sync by running
sudo apt install krill-sync
Build with Cargo
For Ubuntu 20.04 with Rust 1.51.0:
apt update && apt install -y build-essential curl libssl-dev openssl pkg-config curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env cargo install --git https://github.com/NLnetLabs/krill-sync.git --tag v0.2.0-rc1 --locked
This tool is intended to enable public facing RRDP and Rsync servers to serve consistent and fresh content obtained from a "hidden" RRDP source server, e.g. that runs an RPKI Publication Server. In this context "hidden" means that the server is not reachable by the public name of the repository but creates content that points to the public name.
Data Retrieval and Assumptions
The repository data is retrieved by using the RRDP protocol itself, pointed at a hidden
krill-sync (as of 0.2.x) assumes that the back-end publication server
uses the intended public URIs for its RRDP files. But, importantly, we can instruct it to
map the URI of a given public RRDP notification URI to an alternative hidden backend server:
krill-sync --source_uri_base https://hidden.server/rrdp/ https://some.server/rrdp/notification.xml
There is a further assumption that snapshot and delta file URIs will use the same hostname
and base path as the notification file, so that it can be resolved to the back-end using the
--source_uri_base map that is used for the notification file itself.
Using the RRDP protocol as a source ensures that only complete and consistent data is downloaded by following the Notification File to the dependent resources (assuming a publication server such as Krill that ensures that the Notification File is written last during a content update). It also enables the public servers to download only deltas rather than the entire snapshot on update keeping the sync time, bandwidth usage and origin server impact to a minimum.
Other means of synchronizing the content (e.g. rsync, scp, or possibly even file system replication tools like GlusterFS) are not RRDP aware and thus may sync the Notification File before the files it references have all been downloaded which can cause Relying Party clients of the public facing repository servers to encounter errors while attempting to download updates from the service.
krill-sync runs without previous state, it will retrieve and parse the specified
notification file. Then it will retrieve and parse the snapshot file to obtain all current
objects, as well as each delta file mentioned (unless an optional hard limit is set, see below).
If a file cannot be retrieved or parsed then the run will fail. This means that if the back-end system RFC 8182 RRDP XML is somehow not understood by krill-sync the data will not be accepted.
The tool will persist its current state to disk. The default directory for this is
/var/lib/krill-sync but this can be overridden using the
On subsequent runs
krill-sync will check whether delta files can be used to update its
current state. I.e. the session is unchanged and there is a chain of delta files
available. If not, it will re-sync using the current back-end snapshot.
We recommend that you run krill-sync at least every 5 minutes from cron, but every minute should be fine too. Make sure that errors are reported back to you, e.g. by having cron email them. Note that a krill-sync invocation will abort with an error message if it finds that another instance (previous invocation) is still running. In that case you may have to reduce the frequency. That said, we found that on reasonably fast hardware a clean krill-sync takes less than a minute even for huge repositories, and subsequent updates are even faster.
Produced RRDP Data
This tool produces its own RRDP data XML. This means that the formatting, hash values
and even the URI paths of the RRDP files produced by this tool may be different
from the back-end. This is not issue as long as all your public facing RRDP front-ends
The RRDP files are written in a safe order, meaning: new snapshot and delta files are written to disk first. Then a new notification file is written to a temporary file, which is then renamed to avoid avoid race conditions resulting from overwriting this file as it's being served.
Old snapshot and delta files are marked as 'deprecated' as soon they are no longer referenced by the current notification file. Files which have been deprecated for longer than N seconds are removed at the end of each synchronization. The default is 600 seconds, or 10 minutes, but note that this can be configured, and that the clean up only happens when an actual synchronization is done.
The default base directory for these files is
/var/lib/krill-sync/rrdp/, but this
can be overridden using the
--rrdp-dir argument. The notification file will be
notification.xml and will be published directly under this base directory.
Snapshot files are called
snapshot.xml, delta files
delta.xml. Snapshot and
delta files are stored under two further directories based on the session ID, which
is a UUID, and the serial.
Example of a resulting structure:
/var/lib/krill-sync/rrdp/notification.xml /var/lib/krill-sync/rrdp/de3b0c3c-85be-45f8-b89c-5d4a0fa66312/305278/snapshot.xml /var/lib/krill-sync/rrdp/de3b0c3c-85be-45f8-b89c-5d4a0fa66312/305278/delta.xml /var/lib/krill-sync/rrdp/de3b0c3c-85be-45f8-b89c-5d4a0fa66312/305277/delta.xml /var/lib/krill-sync/rrdp/de3b0c3c-85be-45f8-b89c-5d4a0fa66312/305276/delta.xml
Make sure that you map the base directory in your HTTPS front-end server, such as NGINX, to this base directory in a way that the given public RRDP notification URI will map to the file on disk.
Directory content for serving by an Rsync daemon is also created from the data downloaded
via RRDP, no
rsync binary is needed. If you do not need to serve the rsync content on
a specific machine, then you can suppress this by using the
This tool writes complete new rsync directories for each new RRDP session and serial that
is retrieved. On UNIX like systems symlinks are then used to link the
current directory to
the latest content. As it turns out the rsync daemon resolves symlinks whenever a client
connects, so ongoing connections will keep being served the directory which was current
at the time of connection.
The previous directory is marked as deprecated whenever a new current directory is created this way. At the end of each synchronization directories which have been deprecated for more than N seconds (default 10 mins) are removed.
By default the base directory for these rsync directories is
but this can be overridden using the
--rsync-dir argument. Your
should be configured to serve the
current directory, e.g.:
$ cat /etc/rsyncd.conf uid = nobody gid = nogroup max connections = 50 socket options = SO_KEEPALIVE [repo] ## <-- Use your public rsync module name here! path = /var/lib/krill-sync/rsync/current/ comment = RPKI repository read only = yes
This tool is intended to be used as a system "service" invoked periodically e.g. once a minute from cron, it is not meant to be a user facing tool. As such it follows Linux FHS guidelines for storing application state rather than user state.
krill-sync --help krill-sync 0.2.0 A tool to synchronize an RRDP and/or Rsync server with a remote RRDP publication point. USAGE: krill-sync [FLAGS] [OPTIONS] <notification-uri> FLAGS: -h, --help Prints help information --insecure Whether or not localhost connections and self-signed certificates are allowed -q, --quiet Quiet mode (no warnings or informative messages, only errors) --rsync-disable Disable writing the rsync files -V, --version Prints version information -v, --verbose Verbose mode (-v, -vv, -vvv, etc.) OPTIONS: --cleanup-after <seconds> Remove unreferenced files and directories older than X seconds [default: 600] --rrdp-dir <dir> The directory to write RRDP files to [default: /var/lib/krill-sync/rrdp] --rrdp-max-deltas <number> Optional hard upper limit to the number of deltas --rrdp-notify-delay <seconds> Delay seconds before writing the notification.xml file [default: 0] --rsync-dir <dir> The directory to write Rsync files to [default: /var/lib/krill-sync/rsync] --source_uri_base <uri> Base uri for the notify file on the back-end server. Must end with a slash -s, --state-dir <dir> The directory to write state to [default: /var/lib/krill-sync] ARGS: <notification-uri> The public RRDP notification URI
Tip: If connecting directly to a Krill server, and not to e.g. NGINX in front of Krill, you will need to use
--insecure as the Krill RRDP service uses a self-signed TLS certificate.
By default krill-sync will output only warnings and errors. Using
--verbose repeatedly will cause krill-sync to print more and more detailed information about its activity. The first three levels enable info, debug and trace logging for krill-sync itself. The three levels after that enable info, debug and trace logging for 3rd party Rust crates that krill-sync uses.