stream transform plugin

Writing a stream transform

A few providers encrypt every file on the client and keep the key out of their own reach: it rides in the link's fragment, the storage server only ever sees ciphertext, and a plain download would be a file of the right length full of rubbish. A stream transform is a resolver for them: resolve answers with the address and a description of how its bytes become the file, and the host applies it on its own write path and checks the provider's integrity value before the file is finished.

Scaffold	rdownloader plugin new --type stream-transform
plugin_type	stream-transform
World	stream-transform-plugin
Exports	claims-url, resolve
Template	sdk/templates/stream-transform
Contract	Interface stream-transform

What the host promises

• The plugin describes, the host computes. You name a primitive the host implements (aes-128-ctr today) and its parameters. You never see a byte of the file, and a name the host does not know is refused rather than guessed at.
• The key is a secret from the moment you return it. The host vaults it and keeps it out of every row, log line and answer.
• Nothing survives between calls. A download that resumes asks resolve again and gets a fresh address and a fresh description.

Permissions

The scaffold asks for net_http with the provider's API and the storage hosts the download address lies on, and declares secret_fragment_domains: the hosts whose link fragment is key material. Intake strips the fragment off every other address before a LinkGrabber row exists; without this line the key is gone before the plugin is ever asked. It is a top-level key and has to stand above the first [table] header. A stream transform may also ask for secrets and key_derivation, and it alone among the extension types may carry a [provider] section beside [extension], for a provider whose requests are signed with an account.

Ask for exactly what you use: the install preview shows every grant, and a component importing an interface its manifest does not grant cannot be instantiated. Every capability is listed under Capabilities.

Rules worth knowing first

Your half of the key promise: the key never goes into a request — the provider is the one party that must never learn it. A wrong key otherwise produces a file of exactly the right length under exactly the right name, which is why the integrity check keeps the partial file on a mismatch.

Step by step

1. Scaffold it

plugin new reads its templates from sdk/templates, so run it from a checkout of the repository:

git clone https://github.com/degoya/rDownloader.git
cd rDownloader
rdownloader plugin new --type stream-transform --out ../mystream
cd ../mystream

Without an installation of rDownloader, the packaging tool rd-pack — cargo build --profile release-test -p rd-pack in the checkout — takes the same arguments: target/release-test/rd-pack plugin new --type stream-transform --out ../mystream.

The directory name becomes the crate name and the plugin's slug (--name sets another). The scaffold gets its own plugin id, its own copy of the contract in wit/rdownloader.wit and its own Ed25519 key pair: plugin-signing.key, whose public half is already in manifest.toml. Keep that file out of version control.

2. Make it yours

• src/link.rs reads the address and the key in its fragment.
• src/reply.rs reads the provider's answer.
• src/guest.rs holds claims-url and resolve.
• manifest.toml: secret_fragment_domains, [capabilities.net_http] domains.
• locales/en.json: the English text of every failure code you report.

Change description and author under [metadata], and read the comment at the top of src/lib.rs: it states what the host promises this type.

3. Test it

cargo test

The unit tests need no WebAssembly toolchain: only src/guest.rs is wasm32-specific. Give every answer of your provider you read a test of its own.

4. Build the component

cargo build --release --target wasm32-unknown-unknown
wasm-tools component new target/wasm32-unknown-unknown/release/mystream.wasm -o target/mystream.wasm

This needs the wasm32-unknown-unknown target and wasm-tools 1.261.0, the version the bundled plugins are made with: cargo build makes a core module that carries the world src/guest.rs generates, and wasm-tools component new turns it into the component. The plugin is built without WASI: a component importing anything outside rdownloader:plugin is refused — see Building without WASI.

5. Package and sign it

rdownloader plugin package --manifest manifest.toml \
  --component target/mystream.wasm \
  --locales locales --key plugin-signing.key --output mystream.rdplug
rdownloader plugin conformance mystream.rdplug --json

conformance answers whether this application would run the package; it has to be clean before you publish. For a key that outlives the scaffold, rdownloader plugin keygen --role plugin --output ./keys writes a pair: the Base64 in the .pub file goes into public_key, and key_id gets a name of your own. In CI, RDOWNLOADER_PLUGIN_SIGNING_KEY holding the private key's PEM text replaces --key. Details: Packaging, signing and distribution.

6. Install and trust it

For a first try without signing, package with --development instead of --key and start rdownloader serve --plugin-development-mode, the only mode that accepts an unsigned package.

Install a signed package under Settings → Plugins → Add as an upload, or with rdownloader plugin install mystream.rdplug --root data/plugins. Review before installing shows every permission and the signing key; a key this installation has not seen asks to be trusted — compare its fingerprint. The message after installing says whether the plugin runs at once or after the next restart; see Installing and trust.

Paste a link with its fragment into the LinkGrabber and download it: the file is decrypted as it is written.

7. Publish it

Publish the .rdplug together with your key's fingerprint, or — so that people get your new versions in the Catalogue — in a plugin repository of your own: a signed index built with rdownloader plugin index build and a repository-role key. A repository only delivers: every package in it still needs your plugin key, approved by the person installing it. See A repository index and Running a plugin repository.

The SDK's reusable CI workflow, sdk/ci/plugin.yml, builds, checks the imports, packages and runs conformance on every push and pull request; it signs only on a push to the default branch or a tag, so a pull request never sees the key.

Read next

• Stream transforms in the plugin reference — what the host does with each call, in full
• Interface stream-transform — every function and type, generated from the contract
• Developing a plugin — the tools, translations, errors and what every type shares
• The stream-transform template on GitHub