Szurubooru Script Tag Categories

[MrMcMander]

Hello,

Would there be a way to add automatic categorization of tags to the Szurubooru script for artist, character, and copyright? I.E. the artist gets added to an "atrist" category so that it can be colored coded automatically.

Cheers

[Bionus]

Hello

Just to clarify, are you asking about the NodeJS script for commands that can be found here?
https://bionus.github.io/imgbrd-grabber/docs/commands/szurubooru.html

[MrMcMander]

Correct.

I know the "%character%" "%artist%" and "%copyright%" arguments need to be added to the command in the image line and their respective argv[#]. But beyond that, I got nothing as I have little to no experience programming. And without the categories I find myself having to copy the tag, search it, and add it to their respective category per image, which is quite taxing.

On that note, I've also noticed the URLs being pulled for source are all messed up: "https_i.pximg.net_img-original_img". That may be a global setting in Grabber I've changed affecting it, though. Just thought I'd mention it here in case it's not.

[Bionus]

I just pushed an update to the script and the documentation.

Please use this updated script: szurubooru.js
And this updated command:

node szurubooru.js "YOUR_USERNAME" "YOUR_TOKEN" "%all:includenamespace,unsafe%" "%rating%" "%source:unsafe%" "%path%"

And it should fix the two issues you mentioned 👍

[MrMcMander]

Getting "[Error] Error executing command (return code: 1)" with new script and commands.

It looks to be pulling all the tags correctly and I've renamed the categories on my local Szuru to match the categories.

I'm assuming that tags, safety and source need " " around them in the new script. I tried it with and without, no luck. Still code: 1

[Bionus]

You also need to install axios globally, I forgot to mention it in my comment:

npm i -g axios

Updated docs:
https://github.com/Bionus/imgbrd-grabber/blob/develop/docs/_docs/commands/szurubooru.md

If it still doesn't work, can you try running the command manually and see what errors you get?
Grabber should print in the log the command it tries to run. You'll have to open cmd (Windows command line), go to Grabber's install folder (using cd C:/.../Grabber), then run the command starting with node.

[MrMcMander]

That did the trick! Thank you for your work.

[byte-warlock]

@Bionus I'm using the provided solution right now but it doesn't seem to work with posts that don't have a source.

I did some testing and when there is an empty string for the source the string doesn't get sent to the szurubooru script. I copied the command into a terminal then ran it manually and it works.

This is the output of some logging I did in the script (specific image details removed):

0: C:\Program Files\nodejs\node.exe
1: C:\Program Files\Grabber\szurubooru.js
2: myuname
3: mytoken
4: image tags
5: image rating
6: file path

As far as I'm aware, the image source should be in the index between the image rating and the file path.

[Arthur-K-99]

Same, did the API change or something?

[byte-warlock]

I've modified the script to check the number of arguments for now and act appropriately but the empty string not being passed to the program is something that should be looked into when there is time.

Updated script:

// Usage:
// node szurubooru.js "username" "token" "tagType1:tag1 tagType2:tag2" "safe" "http://source" "path/to/file.jpg"

const axios = require("axios");
const fs = require("fs");
const FormData = require("form-data");

async function getTag(name) {
    try {
        const res = await axios.get(`/tag/${name}`);
        return res.data;
    } catch (e) {
        if (e.response.status === 404) {
            return null;
        }
        throw e;
    }
}
async function createTag(name, category) {
    try {
        const data = { names: [name], category };
        await axios.post("/tags", data);
        console.log(`Tag "${name}" created`);
    } catch (e) {
        console.error("Error creating tag: " + e.message);
        console.error(e.response.data);
    }
}
async function updateTag(name, version, category) {
    try {
        const data = { version, category };
        await axios.put(`/tag/${name}`, data);
        console.log(`Tag "${name}" updated`);
    } catch (e) {
        console.error("Error updating tag: " + e.message);
        console.error(e.response.data);
    }
}
async function setTagCategory(name, category) {
    const tag = await getTag(name);
    if (tag === null) {
        createTag(name, category);
    } else if (tag.category !== category) {
        updateTag(name, tag.version, category);
    }
}

(async () => {
    // Szurubooru doesn't use the same ratings as most boorus so we need to map them
    const ratingsMap = {
        "safe": "safe",
        "questionable": "sketchy",
        "explicit": "unsafe",
    };

    // Get parameters
    const argv = process.argv.slice(2);
    const username = argv.shift();
    const token = argv.shift();

    // If 4 arguments are given, assume argv[2] is the source URL.
    var source = undefined;

    if (argv.length == 4)
    {
        source = argv[2];
    }

    // Axios settings
    axios.defaults.baseURL = "http://localhost:8080/api";
    axios.defaults.headers.common["Authorization"] = "Token " + Buffer.from(username + ":" + token).toString("base64");
    axios.defaults.headers.common["Accept"] = "application/json";

    // Parse tags and update categories
    const tags = argv[0].split(" ");
    for (i = 0; i < tags.length; ++i) {
        const parts = tags[i].split(":");
        const category = parts.shift();
        const name = parts.join(":");
        setTagCategory(name, category);
        tags[i] = name;
    }

    // Actual data to send to the server
    var data = {
        tags,
        safety: ratingsMap[argv[1]],
        source: source,
    };

    // Create a multipart form data request
    const form = new FormData();

    // Use correct argument if no source was given
    if (source == undefined)
    {
        form.append("content", fs.createReadStream(argv[2]), "test.tmp");
    }
    else
    {
        form.append("content", fs.createReadStream(argv[3]), "test.tmp");
    }
    
    form.append("metadata", JSON.stringify(data));

    // Create image post
    try {
        const config = {
            headers: form.getHeaders(),
            maxContentLength: 999999999999,
            maxBodyLength: 999999999999,
        }
        await axios.post("/posts/", form, config);
    } catch (e) {
        console.error("Error creating post: " + e.message);
        console.error(e.response.data);
    }
})();

[Bionus]

Yes, this was caused by the way Qt removes empty parameters when splitting command arguments.

I just pushed a fix that should solve this issue in the future, properly passing the source parameter as an empty string to the Szurubooru script.

[Bionus]

If the server replies a 500, it's likely a problem in the back-end itself (so in Szurubooru), either caused by an issue in their code or in the script.

Checking Szurubooru's error logs would give the details required to understand the root cause of the problem.
If you're using Docker it should be quite easy to access the logs with docker-compose logs -f, but if you're running it directly no idea where they're stored.