intake mirrors plugin

Writing an intake parser with mirrors

A mirror list names the same bytes at several addresses, ranked, with the hashes they must match. A plain intake parser proposes one link per file and loses that. This one proposes the best address through parse, as every parser does, and states the whole set through sets — every other source with its priority and location, the whole-file hash and the piece hashes — so that one transfer fetches chunks from several sources at once and checks what it got.

It is not a type of its own: the manifest says plugin_type = "intake", and the world in Cargo.toml, intake-mirrors-plugin, is what adds the mirror-sets export the host looks for.

Scaffold	rdownloader plugin new --type intake-mirrors
plugin_type	intake
World	intake-mirrors-plugin
Exports	claims, parse, normalize and sets
Template	sdk/templates/intake-mirrors
Contract	Interface mirror-sets

What the host promises

• You propose, the application decides. What parse returns goes through the same review, blocklist and routing a pasted link does. A set is kept beside its candidate, and only when the candidate survives.
• A set is matched by its primary address. sets names each file by the address parse proposed for it; a set whose primary-url no candidate carries is dropped.
• Every field is checked. A known scheme, no password in an address, at most 32 sources, hex of the right width, a piece list that covers the stated size exactly. A field that fails is refused, so state only what the document actually said.

Permissions

The scaffold asks for nothing: the list carries every address it needs. Add net_http only if your format has to be fetched to be read.

Its [extension] section has claims empty, because the format is recognised from the text in claims, not from where it came from.

Ask for exactly what you use: the install preview shows every grant, and a component importing an interface its manifest does not grant cannot be instantiated. Every capability is listed under Capabilities.

Step by step

1. Scaffold it

plugin new reads its templates from sdk/templates, so run it from a checkout of the repository:

git clone https://github.com/degoya/rDownloader.git
cd rDownloader
rdownloader plugin new --type intake-mirrors --out ../mymirrors
cd ../mymirrors

Without an installation of rDownloader, the packaging tool rd-pack — cargo build --profile release-test -p rd-pack in the checkout — takes the same arguments: target/release-test/rd-pack plugin new --type intake-mirrors --out ../mymirrors.

The directory name becomes the crate name and the plugin's slug (--name sets another). The scaffold gets its own plugin id, its own copy of the contract in wit/rdownloader.wit and its own Ed25519 key pair: plugin-signing.key, whose public half is already in manifest.toml. Keep that file out of version control.

2. Make it yours

• src/list.rs reads the scaffold's made-up format — one file per line, sources in order of preference — and is what you replace with your format's reader.
• src/guest.rs has parse for the LinkGrabber and sets for the transfer.
• Cargo.toml keeps world = "intake-mirrors-plugin", and manifest.toml keeps plugin_type = "intake".
• locales/en.json: the English text of every failure code you report.

Change description and author under [metadata], and read the comment at the top of src/lib.rs: it states what the host promises this type.

3. Test it

cargo test

The unit tests need no WebAssembly toolchain: only src/guest.rs is wasm32-specific. Give every answer of your provider you read a test of its own.

4. Build the component

cargo build --release --target wasm32-unknown-unknown
wasm-tools component new target/wasm32-unknown-unknown/release/mymirrors.wasm -o target/mymirrors.wasm

This needs the wasm32-unknown-unknown target and wasm-tools 1.261.0, the version the bundled plugins are made with: cargo build makes a core module that carries the world src/guest.rs generates, and wasm-tools component new turns it into the component. The plugin is built without WASI: a component importing anything outside rdownloader:plugin is refused — see Building without WASI.

5. Package and sign it

rdownloader plugin package --manifest manifest.toml \
  --component target/mymirrors.wasm \
  --locales locales --key plugin-signing.key --output mymirrors.rdplug
rdownloader plugin conformance mymirrors.rdplug --json

conformance answers whether this application would run the package; it has to be clean before you publish. For a key that outlives the scaffold, rdownloader plugin keygen --role plugin --output ./keys writes a pair: the Base64 in the .pub file goes into public_key, and key_id gets a name of your own. In CI, RDOWNLOADER_PLUGIN_SIGNING_KEY holding the private key's PEM text replaces --key. Details: Packaging, signing and distribution.

6. Install and trust it

For a first try without signing, package with --development instead of --key and start rdownloader serve --plugin-development-mode, the only mode that accepts an unsigned package.

Install a signed package under Settings → Plugins → Add as an upload, or with rdownloader plugin install mymirrors.rdplug --root data/plugins. Review before installing shows every permission and the signing key; a key this installation has not seen asks to be trusted — compare its fingerprint. The message after installing says whether the plugin runs at once or after the next restart; see Installing and trust.

Paste a mirror list into the LinkGrabber and queue a file: the transfer shows how many sources it fetches from.

7. Publish it

Publish the .rdplug together with your key's fingerprint, or — so that people get your new versions in the Catalogue — in a plugin repository of your own: a signed index built with rdownloader plugin index build and a repository-role key. A repository only delivers: every package in it still needs your plugin key, approved by the person installing it. See A repository index and Running a plugin repository.

The SDK's reusable CI workflow, sdk/ci/plugin.yml, builds, checks the imports, packages and runs conformance on every push and pull request; it signs only on a push to the default branch or a tag, so a pull request never sees the key.

Read next

• Intake parsers in the plugin reference — what the host does with each call, in full
• Interface mirror-sets — every function and type, generated from the contract
• Developing a plugin — the tools, translations, errors and what every type shares
• The intake-mirrors template on GitHub