site rules

Site rules

A release page is a page carrying links to files — a download folder, a release listing, a page that posts one file at several hosters. rDownloader recognises such pages with site rules: a rule is data, not code, and it says which hosts it claims, how to read one of their pages, and where the package name comes from. A paste of an address a switched-on rule claims becomes one package named after the release, holding the links the rule found and none of the page's own; a page that lists several releases becomes one package per release instead.

Open them under Settings → Site rules. The page has the rule form on one side and the rule list on the other.

Rules carry no signature. Every rule here is an ordinary rule of yours — whether it came with rDownloader, from an export somebody gave you, or from your own hand — and you can switch, edit, duplicate, export and remove it alike.

The rules that come with rDownloader

rDownloader brings thirteen rules for sources that publish free software, freely licensed media and public pastes. They are installed at the first start, switched off, so nothing fetches a page you did not ask for, and they are grouped by project:

Group	Rule	What it reads
Linux	Debian installation images	An image folder on cdimage.debian.org: every .iso in it becomes one package, named after the network installer
Linux	Ubuntu release images	A release folder on releases.ubuntu.com (24.04/, noble/ …): its .iso images as one package, named after the page title
Linux	Fedora release images	The image folder of one Fedora release on the kernel.org mirror
Linux	Arch Linux installation image	Arch Linux's ISO folder on its own mirror, latest/ or one dated release
Linux	Linux Mint release images	A version folder on pub.linuxmint.io (22.3/ …): its .iso images as one package
Blender	Blender releases	A release folder on download.blender.org/release/ — every build for every platform; you choose builds by version, platform and format before anything is fetched
Blender	Blender open movies	A film folder under download.blender.org/demo/movies/ (Big Buck Bunny, Tears of Steel, Sintel): you choose the video by resolution and format
Free software	LibreOffice downloads	One platform folder of a stable LibreOffice release (win/x86_64/, deb/x86_64/ …); you choose what to fetch first
Free software	VLC media player downloads	One platform folder of a VLC release on VideoLAN's download server (win64/, macosx/ …); you choose what to fetch first
Internet Archive	Internet Archive items	An item page (archive.org/details/<item>): every uploaded file of the item as one package
Paste services	Pastebin pastes	A public paste on pastebin.com: every address in its text
Paste services	Rentry pages	A public page on rentry.co or rentry.org: every address in its text
Paste services	sourcehut pastes	A public paste on paste.sr.ht: every address in its files

Each of them carries a description, shown below its name in the list, that explains how the rule is built — open one in the form to see the steps. Four of them — the two Blender rules, LibreOffice and VLC — are two-stage rules that let you choose first (see Choose first, then fetch); the others are one-stage rules.

To try one, switch it on, paste an address it claims — its probe address, shown in the form, is a real one — into the LinkGrabber, and switch it off again when you are done.

The project publishes no other rules. Rules for other sites come from an export somebody else wrote, or you write them yourself.

The rule list

Rules are listed by group, each group in a card with its own switch (Switch the whole group) and every rule with its own (Switch this rule). A rule is consulted only when both its switch and its group's are on; The group is switched off, so this rule is not consulted says when the group holds it back. The groups have names in the interface — Linux, Blender, Free software, Internet Archive, Paste services, Boards, Adult, E-books, Graphics — and a group you type yourself is shown as you typed it. Both switches take effect on the next paste, not after a restart.

Each row names the hosts the rule claims and its description. A small icon says where the rule came from; point at it for the details:

Icon	Origin
Light bulb	Bundled — comes with rDownloader
File with arrow	Imported — from a rule export
Pencil	Written here — created or changed in the rule editor
Robot	Through MCP — written or changed by an AI assistant over MCP
Question mark	Origin unknown — stored before the origin was recorded

The form names the origin of the rule you open as well. Switching a rule on or off keeps its origin, and so does saving it unchanged; saving a changed rule makes it Written here (or Through MCP, when an assistant changed it).

A row's buttons: Duplicate makes a switched-off copy under a free identifier (<id>-copy, <id>-copy-2, …) and opens it in the form — the way to try a change to a working rule without losing it. The pencil opens the rule in the form; the red bin removes it after a question, which reminds you to export it first if you want to keep it.

What the check found

A rule shows a badge only once it has been checked; a rule nobody checked has none. The check is rdownloader doctor site-rules (see Diagnostics and maintenance): it tries every switched-on rule against its probe address and stores what it found. Point at a badge for the reason.

Badge	Meaning
Working (quiet outline)	The page was reachable, its layout fits, and the rule found links
Changed	Reachable, but nothing came back — the site's layout moved, or a limit was hit
Blocked	Something answered and refused — a challenge, a captcha nobody answered
Dead	Nothing answered, or the page is gone

A rule found dead is skipped from the next start on, not deleted: it stays switched on and comes back the next time the check finds the service alive. The check runs only when you start it; nothing knocks on a site by itself.

A rule also carries its service's former hosts (Former hosts in the form); an address on one of those is rewritten to the current host, so an old bookmark still works.

Exporting rules

1. Tick the rules you want to give away — the checkbox at the start of each row. Tick none to export all of them.
2. Press Export above the list; with ticks it reads Export N rules. With no rule in the list there is nothing to export, and the button is disabled.
3. A file rdownloader-site-rules-<date>.json is saved. Each rule in it carries the switch it has here, on or off.

The file is all the other side needs: another installation imports it without a key and without rework.

Importing rules

1. Press Import above the list and choose an export file. Nothing is stored yet.

2. The dialog Import rules shows every rule the file brings, with its name, its hosts and whether it arrives Switched on or Switched off, and what an import would do with it:

   Status	Meaning
   New	No rule with this identifier exists here; it is added
   Replaces one you have	A rule with this identifier exists here and differs — in its body or its switch
   Already here	The same rule, with the same switch, is already here; nothing changes
   Refused	The rule is incomplete or broken, or its identifier already appeared earlier in the file; it is left out, the others go on

3. A rule that would replace one of yours does so only when you tick its box (Replace my rule … with the one from the file). Leave it unticked and yours stays as it is.

4. Press Import. A notice counts what happened — Imported: 2 new, 1 replaced, of 4 rules in the file — and names any rule that was refused.

Each imported rule arrives with the switch it had where it was exported: a rule your colleague used switched on is consulted here on the next paste. Read the hosts in the preview before you import a file from somebody you do not know well — a rule fetches the pages it claims, but it can never be pointed at an address in your own network, and every run is bounded (see What a rule can and cannot do).

The file must be an export of this kind; anything else is refused with That file is not a rule export.

Deleting all rules and restoring the bundled ones

Delete all — the red entry in the ⋮ menu (More actions) above the list, beside Export and Import — removes every rule of this installation at once. It asks first: Delete every site rule? names how many rules go and advises to export them before. Confirm with Delete all: the rules go with their check results, and the group switches stay. The deletion is written to the audit log (All site rules deleted).

An empty list says that no release page is recognised by rule and offers Restore bundled rules and Write a rule; an import works as well.

Restore bundled rules — at any time in the same ⋮ menu — writes the rules that come with rDownloader again, switched off. One you still have stays as it is, edited or not; when every bundled rule is already there, the notice says so (Every bundled rule is already here). Bundled rules you deleted one by one come back only this way, never by themselves.

Writing a rule

Write a rule is the form beside the list; the pencil on a row opens an existing rule in it. Every field is a field, not a line of JSON:

• Identifier — lowercase letters, digits and hyphens; it cannot be changed later and never repeats another rule's. Name, Description (shown below the name in the list), Group (one of the existing groups or a new one — a new name is written in lowercase letters, digits and hyphens, so Link Protector becomes link-protector) and Revision (raise it when you change the rule).
• Hosts — one per line; *.example.org covers everything below the host, the first entry must be a concrete host. Path patterns — one regular expression per line; empty claims every path. Former hosts — addresses on them are rewritten to the first host.
• Probe address — a real page the rule claims; the check fetches it.
• Steps — they run in order, and each of the seven kinds shows the fields it needs: Fetch a page, Fetch JSON, Apply a pattern, Decode a value, Submit a form, Follow a redirect and Answer a captcha. fetch writes the page into page; the links are whatever ends up in links. Steps can be moved up and down.
• Package name — taken from the page title, a pattern or a variable.
• The links are copies of one file — for a page that posts one file to several hosters.
• Consult this rule right away — whether the rule is switched on when you save it.

Before saving, run it: Trial run takes a real address (empty means the probe address) and shows the links it found, the package name it read, how many requests it took, and which links would be dropped because they answer with a page rather than a file — for a rule with several packages, one block per package, each link with its mirror number. Nothing is saved by the run.

A page with several releases

Some pages list several releases at once — every build of a program for every platform, every season of a series in every quality. A rule for such a page switches on One package per entry (the groups field of the rule):

1. The rule's own steps leave one entry per release in a variable (Entries from variable).
2. The steps per entry run once for every entry, with the same step kinds, and end with that release's links in links.
3. The package name per entry is read from the entry. A release whose name is not found takes the page's.
4. Mirrors says which links of one release are the same file: Same position at every hoster (the first link at one hoster is a copy of the first at every other, the second of the second, …) or Every link is the same file.

The LinkGrabber then shows one package per release, and each file with its copies at the other hosters as one mirror group. A release that yields no link is left out; a link two releases list stays with the first. Every entry's requests and links count against the same limits as the rest of the rule.

"groups": {
  "from": "releases",
  "steps": [
    { "kind": "regex", "from": "entry", "pattern": "(\"https?:[^\"]+\")", "into": "quoted", "all": true },
    { "kind": "decode", "encoding": "js-string", "from": "quoted", "into": "links" }
  ],
  "package": { "from": "regex", "pattern": "\"title\":\"([^\"]+)\"", "source": "entry" },
  "mirrors": "by-host"
}

A rule with several packages cannot also tick The links are copies of one file: with packages, mirrors are said per package.

Choose first, then fetch

When a page lists many releases and you want only some of them — or each release's links sit behind a captcha — the rule can list the releases and fetch nothing until you choose. In the form that is Choose first, then fetch: one attribute per line as name=pattern, such as season, episode, resolution, language or hoster; the bundled Blender releases rule reads version, platform and format this way. Such a rule may also Send the fields as JSON, name the Page of the captcha and mark the captcha an Invisible widget; its trial run lists the entries and asks no captcha.

"groups": {
  "from": "files",
  "pick": { "attributes": {
    "version": "^blender-([0-9]+\\.[0-9]+\\.[0-9]+)",
    "platform": "-(linux|macos|windows)-",
    "format": "\\.(zip|tar\\.xz|dmg|msi|msix)$"
  } },
  "steps": [ { "kind": "regex", "from": "entry", "pattern": "(.+)", "into": "links" } ],
  "package": { "from": "regex", "pattern": "^(.+?)\\.(?:zip|tar\\.xz|dmg|msi|msix)$", "source": "entry" }
}

The rule's own steps collected the file names of the folder into files; each name is one entry, and the chosen entry is the link itself, read relative to the folder's address.

How choosing works in the LinkGrabber is described in The LinkGrabber.

What a rule can and cannot do

A rule fetches pages, reads them with patterns, follows a redirect without being sent anywhere else, submits a form, decodes a value and, where the page asks for one, has a captcha answered. Base64, hexadecimal, rot13, percent-encoding and concatenated JavaScript string literals are decoded.

A rule can also walk an API that lists ids and answers each one separately. When a fetch step's address names a list of ids, the step asks once per id and keeps every answer in order. Only one list per address: a rule naming two is refused rather than guessing how to pair them.

There is no JavaScript interpreter and no browser. A page that hands out its links by running a program, or that answers a non-browser with a challenge, is refused with a stated reason — the check calls it Blocked.

Every run is bounded — depth, number of requests, number of links, total time and response size — and a run that returns to an address it already fetched ends there. A request per id counts like any other, so a very long list ends at the request limit instead of running on. No rule may be pointed at an address on your own network, whoever wrote it.

A rule itself has a size limit too: at most 32 steps (and 32 more for its packages), and no pattern longer than 1024 characters or too large once compiled. Large Unicode classes count: \w{60} is refused, [A-Za-z0-9_]{60} is not.

Related pages

• The LinkGrabber — where a recognised page arrives, and where you choose
• Subscriptions — watching a listing whose release pages a rule reads
• MCP server — the site-rule tools for an assistant
• Diagnostics and maintenance — the check
• Backup and restore