intake plugin

Writing an intake parser

An intake parser teaches the LinkGrabber a format it does not understand: a link list, a page layout, a container. It is shown pasted text and URLs, proposes the links it finds — grouped into packages where the text says so — and may tidy an address into its canonical form.

A parser that also knows every mirror of a file and its hashes is the intake parser with mirrors.

Scaffold	rdownloader plugin new --type intake
plugin_type	intake
World	intake-plugin
Exports	claims, parse, normalize
Template	sdk/templates/intake
Contract	Interface intake

What the host promises

• You propose; the application decides. Everything you return goes through the LinkGrabber review, the domain blocklist and the routing rules, exactly as a pasted link does, and a package hint is a suggestion, not a package. A parser cannot queue anything.
• You are asked before you are shown anything. claims runs for every paste in the application, so it has to be cheap and claim narrowly.
• A rewrite from normalize is discarded if it changes the host or the scheme. A normaliser tidies an address; it cannot redirect one.

Permissions

The scaffold asks for nothing: a parser reads the text it is handed. Add net_http with the exact domains only if your parser has to fetch something to find the links.

Like every extension type it carries an [extension] section: its slug and claims, the domain patterns it is offered for. Empty means it decides per input in claims.

Ask for exactly what you use: the install preview shows every grant, and a component importing an interface its manifest does not grant cannot be instantiated. Every capability is listed under Capabilities.

Step by step

1. Scaffold it

plugin new reads its templates from sdk/templates, so run it from a checkout of the repository:

git clone https://github.com/degoya/rDownloader.git
cd rDownloader
rdownloader plugin new --type intake --out ../myparser
cd ../myparser

Without an installation of rDownloader, the packaging tool rd-pack — cargo build --profile release-test -p rd-pack in the checkout — takes the same arguments: target/release-test/rd-pack plugin new --type intake --out ../myparser.

The directory name becomes the crate name and the plugin's slug (--name sets another). The scaffold gets its own plugin id, its own copy of the contract in wit/rdownloader.wit and its own Ed25519 key pair: plugin-signing.key, whose public half is already in manifest.toml. Keep that file out of version control.

2. Make it yours

• src/lib.rs claims, reads and normalises. The scaffold finds https://files.example.com/d/<id> links anywhere in the text and groups them under a [Package name] line; replace PREFIX and the reading with your format's.
• src/guest.rs translates the three calls into the contract's types.
• manifest.toml: [extension] slug and claims.
• locales/en.json: the English text of every failure code you report.

Change description and author under [metadata], and read the comment at the top of src/lib.rs: it states what the host promises this type.

3. Test it

cargo test

The unit tests need no WebAssembly toolchain: only src/guest.rs is wasm32-specific. Give every answer of your provider you read a test of its own.

4. Build the component

cargo build --release --target wasm32-unknown-unknown
wasm-tools component new target/wasm32-unknown-unknown/release/myparser.wasm -o target/myparser.wasm

This needs the wasm32-unknown-unknown target and wasm-tools 1.261.0, the version the bundled plugins are made with: cargo build makes a core module that carries the world src/guest.rs generates, and wasm-tools component new turns it into the component. The plugin is built without WASI: a component importing anything outside rdownloader:plugin is refused — see Building without WASI.

5. Package and sign it

rdownloader plugin package --manifest manifest.toml \
  --component target/myparser.wasm \
  --locales locales --key plugin-signing.key --output myparser.rdplug
rdownloader plugin conformance myparser.rdplug --json

conformance answers whether this application would run the package; it has to be clean before you publish. For a key that outlives the scaffold, rdownloader plugin keygen --role plugin --output ./keys writes a pair: the Base64 in the .pub file goes into public_key, and key_id gets a name of your own. In CI, RDOWNLOADER_PLUGIN_SIGNING_KEY holding the private key's PEM text replaces --key. Details: Packaging, signing and distribution.

6. Install and trust it

For a first try without signing, package with --development instead of --key and start rdownloader serve --plugin-development-mode, the only mode that accepts an unsigned package.

Install a signed package under Settings → Plugins → Add as an upload, or with rdownloader plugin install myparser.rdplug --root data/plugins. Review before installing shows every permission and the signing key; a key this installation has not seen asks to be trusted — compare its fingerprint. The message after installing says whether the plugin runs at once or after the next restart; see Installing and trust.

Paste text in your format into the LinkGrabber: your candidates appear for review like any other.

7. Publish it

Publish the .rdplug together with your key's fingerprint, or — so that people get your new versions in the Catalogue — in a plugin repository of your own: a signed index built with rdownloader plugin index build and a repository-role key. A repository only delivers: every package in it still needs your plugin key, approved by the person installing it. See A repository index and Running a plugin repository.

The SDK's reusable CI workflow, sdk/ci/plugin.yml, builds, checks the imports, packages and runs conformance on every push and pull request; it signs only on a push to the default branch or a tag, so a pull request never sees the key.

Read next

• Intake parsers in the plugin reference — what the host does with each call, in full
• Interface intake — every function and type, generated from the contract
• Developing a plugin — the tools, translations, errors and what every type shares
• The intake template on GitHub