szurubooru-toolkit

Python package and script collection to manage your szurubooru image board.

Usage: szuru-toolkit [OPTIONS] COMMAND [ARGS]...

  Toolkit to manage your szurubooru image board.

  Defaults can also be set in a config file.

  Visit https://github.com/reluce/szurubooru-toolkit for more information.

Options:
  --url TEXT                      Base URL to your szurubooru instance.
  --username TEXT                 Username which will be used to authenticate with the szurubooru API.
  --api-token TEXT                API token for the user which will be used to authenticate with the
                                  szurubooru API.
  --public                        If your szurubooru instance is reachable from the internet (default:
                                  False).
  --log-enabled                   Create a log file (default: False).
  --log-colorized                 Colorize the log output (default: True).
  --log-file TEXT                 Output file for the log (default: szurubooru_toolkit.log)
  --log-level [DEBUG|INFO|WARNING|ERROR|CRITICAL]
                                  Set the log level (default: INFO).
  --hide-progress                 Hides the progress bar (default: False).
  -h, --help                      Show this message and exit.

Commands:
  auto-tagger          Tag posts automatically
  create-relations     Create relations between character and parody tag categories
  create-tags          Create tags based on a tag file or query
  delete-posts         Delete posts
  find-duplicates      Find visually duplicate posts via perceptual hashing
  fix-relations        Complete post relation sets via transitive closure
  fix-sankaku-sources  Rewrite outdated Sankaku source URLs to their canonical form
  import-from-booru    Download and tag posts from various Boorus
  import-from-url      Download images from URLS or file containing URLs
  preview-tags         Show WD tagger scores near the thresholds without tagging anything
  reset-posts          Remove tags and sources
  tag-posts            Tag posts manually
  upload-media         Upload media files
  webserver            Run the webserver for the browser extensions

In order to run szuru-toolkit, Python 3.11 or newer is required.

This package is available on PyPI and can be installed with pip: pip install szurubooru-toolkit

The WD tagger (local machine learning tagging) and Pixiv support are optional extras since they pull in heavy dependencies:

• pip install "szurubooru-toolkit[wd-tagger]" for WD tagger support (installs ONNX Runtime)
• pip install "szurubooru-toolkit[pixiv]" for Pixiv metadata support
• pip install "szurubooru-toolkit[wd-tagger,pixiv]" for both

Alternatively, you can clone the package from GitHub and set everything up with uv. In the root directory of this repository, execute uv sync (add --extra wd-tagger --extra pixiv for WD tagger and Pixiv support).

If you would like to run the toolkit in a Docker container instead, follow the instructions below.

Several image variants are published on each release, matching the optional extras. The default image is slim; the others bundle heavier dependencies, so pick the smallest one that covers what you enable in config.toml:

The -wd-tagger-cuda image needs an NVIDIA GPU exposed to the container. Install the NVIDIA Container Toolkit on the host (driver 580+ / CUDA 13), use the latest-wd-tagger-cuda image, and uncomment the deploy.resources.reservations.devices block in docker-compose.yml. Set wd_tagger = true and wd_tagger_providers = ["CUDAExecutionProvider"] under [auto_tagger] in config.toml.

The device reservation follows Docker's GPU access example. Alternatively, gpus: all is valid with Docker Compose 2.30.0 or newer; see the Compose reference. Use one GPU configuration, not both. The device reservation does not require an additional runtime: nvidia entry in Docker's documented setup.

Check your Compose version and validate the configuration before starting:

docker compose version
docker compose config --quiet
docker compose run --rm --entrypoint nvidia-smi szurubooru-toolkit

nvidia-smi should list the host GPU inside the container. This confirms device access; it does not prove that model inference uses CUDA. Check the WD tagger's startup logs for the selected provider, since unavailable providers fall back to CPU.

Use the matching tag in your docker-compose.yml — e.g. -wd-tagger if you set wd_tagger = true. Every tag is also published per version, e.g. :2.0.0, :2.0.0-wd-tagger, :2.0.0-pixiv and :2.0.0-all.

If the mounted volumes should not be owned by root (e.g. on NFS mounts or with rootless containers), set the PUID and PGID environment variables (see the commented block in docker-compose.yml): the container then creates a matching user on startup, chowns its working directory and runs the cron jobs as that user instead of root. Cron jobs using the documented >/proc/1/fd/1 2>&1 redirect keep working: the container rewrites it to an internal log file that the non-root user can write and streams it to the container output.

While the script szuru-toolkit can run with just command line options, you can also set your options in a config file.

The script looks for a config.toml file in following locations:

• Your current working directory from which szuru-toolkit is executed
• ~/.config/szurubooru-toolkit/config.toml
• /etc/szurubooru-toolkit/config.toml

• Your current working directory from which szuru-toolkit is executed
• $USERPROFILE/szurubooru-toolkit/config.toml
• $APPDATA/szurubooru-toolkit/config.toml

Options passed to the szuru-toolkit script take priority over the config file.

You can find a sample config file in the GitHub repository of this package.

Note that path names have to be specified with forward slashes (/) if you're using Windows.

Creating a SauceNAO account and an API key is recommended. Please consider supporting the SauceNAO team as well by upgrading your plan. With a free plan, you can request up to 200 posts in 24h.

For local machine learning tagging, posts can be tagged with one of SmilingWolf's WD taggers. Install the wd-tagger extra (pip install "szurubooru-toolkit[wd-tagger]") and set wd_tagger = true in the [auto_tagger] section to use it. The model set in wd_tagger_model (default: SmilingWolf/wd-eva02-large-tagger-v3, ~1.2GB) gets downloaded automatically from Hugging Face on first use and is cached locally afterwards. Any of the WD v3/v2 taggers work, e.g. SmilingWolf/wd-swinv2-tagger-v3 or SmilingWolf/wd-vit-tagger-v3 for smaller and faster models. General tags and character tags use separate confidence thresholds (wd_tagger_threshold and wd_tagger_character_threshold), since character predictions are usually either confident or wrong. Use szuru-toolkit preview-tags <file-or-post-id> to see all scores near the thresholds when tuning them, and szuru-toolkit auto-tagger --dry-run <query> to preview which tags a run would change without updating any post.

With wd_tagger_review = true, posts whose best character score lands between wd_tagger_review_threshold and wd_tagger_character_threshold get tagged needs_review — a szurubooru query for exactly the ambiguous character matches worth curating manually.

Post safety is detected automatically: booru matches carry over their source rating, and the WD tagger predicts one (safe/sketchy/unsafe) for everything else. On top of that, safety_overrides in [auto_tagger] forces a minimum safety whenever certain tags are present, e.g. safety_overrides = { sketchy = ["nude"], unsafe = ["sex"] }. Safety is only ever raised by this, never lowered — a booru-provided unsafe stays unsafe.

Videos are tagged as well if ffmpeg is installed (wd_tagger_videos): frames are sampled across the duration — longer videos get more frames — and their scores averaged, so tags that only appear in a single frame don't stick.

Inference runs on the CPU by default. For hardware acceleration, set wd_tagger_providers in config.toml, e.g. ["CoreMLExecutionProvider"] on Apple Silicon or ["CUDAExecutionProvider"] on NVIDIA GPUs (install the wd-tagger-cuda extra instead of wd-tagger, or use the -wd-tagger-cuda Docker image). Unavailable providers fall back to the CPU.

New in 2.1.0.

Enable [tag_categories] to categorize new tags from auto-tagger, import-from-booru, and import-from-url. For an instance with custom category names:

[tag_categories]
enabled = true
lookup_danbooru = false
category_map = { general = "General", artist = "Artist", character = "Character", copyright = "Copyright", meta = "Metadata" }

Create the target categories in your instance first. Existing tags (including aliases) keep their categories. With the feature disabled, tagging behaves as before. The built-in mapping uses default, artist, character, series, and meta; omitted mapping entries retain these defaults. Category names on the right are case-sensitive.

Danbooru metadata provides source categories; the WD model provides general and character categories without extra network requests. Imports ask gallery-dl for categorized tags on Gelbooru, Konachan, Yandere and Sankaku, which can require additional requests. When multiple tagging sources classify the same tag, the first available category wins (MD5 matches, then SauceNAO, then the WD model).

Set lookup_danbooru = true to look up missing tags without source categories in batches on Danbooru. This is optional: unknown or unmapped tags are left to your instance's default category when the post is saved. New classified tags are created before the post is saved, and reused across workers. An auto-tagger dry run creates no tags and updates no posts. Incorrect target categories or insufficient permissions are reported as errors rather than silently assigning the wrong category.

For compatibility, [auto_tagger] remap_categories and category_map are also accepted and apply to imports as well. Explicit [tag_categories] settings take precedence. This feature applies to newly created tags; it does not migrate existing tags.

The CLI is installed as szuru-toolkit and under the shorter alias szuructl — both are identical.

Following commands are currently available:

• auto-tagger: Tag posts automatically
• create-relations: Create relations between character and parody tag categories
• create-tags: Create tags based on a tag file or query
• delete-posts: Delete posts
• find-duplicates: Find visually duplicate posts via perceptual hashing
• fix-relations: Complete post relation sets so every member of a set references all other members
• fix-sankaku-sources: Rewrite outdated Sankaku source URLs (legacy numeric or md5 based links) to their canonical form
• import-from-booru: Download and tag posts from various Boorus
• import-from-url: Batch importing of URLs based on gallery-dl
• preview-tags: Show WD tagger scores near the thresholds for a file or post without tagging anything
• reset-posts: Remove tags and sources
• tag-posts: Tag posts manually
• upload-media: Upload media files
• webserver: Run the webserver for the browser extensions

Check szuru-toolkit -h or szuru-toolkit COMMAND -h for a detailed description of supported options.

If you cloned the repo from GitHub, prefix the above scripts with uv run, e.g. uv run szuru-toolkit auto-tagger "date:today". Note that your current working directory has to be the root of the GitHub project.

If your query starts with a dash (-), for example to negate a tag, you have to separate the query from the command with two dashes (This doesn't work with uv run):

szuru-toolkit auto-tagger --no-wd-tagger -- "-foo bar"

While most commands are self explanatory, the following require a bit of extra attention:

Examples

• szuru-toolkit create-relations hitori_bocchi
  • Will create the implication bocchi_the_rock for tag hitori_bocchi if other posts are found with query hitori_bocchi containing bocchi_the_rock as the parody (tag has to be of category series or parody)
  • Will also add hitori_bocchi as a suggestion to the parody tag bocchi_the_rock
  • These relations will only get generated if at least X posts are found containing the tags bocchi_the_rock and hitori_bocchi. Control X with threshold under [create_relations] in config.toml.

If no tag_file is specified, the script will download the most recent 100 tags from Danbooru which have been used at least ten times.

You can use tools like Grabber to download a tag list from common boorus.

The tag_file has to be in following format:

<tag_a>,<category_name>
<tag_b>,<category_name>,<implied_tag>,<implied_tag>
<tag_..n>,<category_name>

The category has to be created beforehand manually (e.g. default, artist, parody/series, character and meta).

Any columns after the category are added as implications; implied tags get created (category default) if they don't exist yet. A single tag can also be created directly with implications passed as an argument. With import_implications = true (or --import-implications), tags created by a Danbooru query additionally get their active Danbooru implications, with implied tags created under their Danbooru category.

Examples

• szuru-toolkit create-tags
• szuru-toolkit create-tags --query genshin* --overwrite
• szuru-toolkit create-tags --query genshin* --import-implications
• szuru-toolkit create-tags --tag-file tags.txt
• szuru-toolkit create-tags --category character --implications "slime,monster_girl" slime_girl

This scripts imports posts with their tags from the URL passed to this script. In the background, it simply calls the gallery-dl script and parses its output. Alternatively, an input file with multiple URLs can be specified.

It's recommended to use the --cookies flag for authentication, check https://github.com/mikf/gallery-dl#cookies for details.

Posts imported from e-hentai/exhentai get the gallery URL set as their source and the artist: tag from the gallery applied; the namespaced e-hentai tags themselves are not imported.

Examples

• szuru-toolkit import-from-url "https://danbooru.donmai.us/posts?tags=foo"
• szuru-toolkit import-from-url "https://chan.sankakucomplex.com/?tags=foo"
• szuru-toolkit import-from-url "https://beta.sankakucomplex.com/post/show/<id>"
• szuru-toolkit import-from-url "https://e-hentai.org/g/<gid>/<token>/"
• szuru-toolkit import-from-url --cookies "~/cookies.txt" --range ":100" "https://twitter.com/<USERNAME>/likes"
• szuru-toolkit import-from-url --input-file urls.txt "https://danbooru.donmai.us/posts?tags=foo" "https://beta.sankakucomplex.com/post/show/<id>"

This script uploads media files from a local directory (src_path).

With read_sidecar_tags = true, tags are read from a <file>.txt sidecar file next to each media file, one tag per line — both abc.jpg.txt (as written by gallery-dl --write-tags) and abc.txt are picked up. Files without a sidecar get the configured default tags as before. If a file is already uploaded, update_tags_if_exists = true appends the sidecar tags to the existing post instead. With cleanup = true, consumed sidecar files are removed along with their media files.

Examples

• szuru-toolkit upload-media --cleanup --tags "foo,bar"
• szuru-toolkit upload-media --read-sidecar-tags --update-tags-if-exists

GitHub repo icon: Code icons created by Smashicons - Flaticon