Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exhentai features #978

Open
ghost opened this issue Sep 5, 2020 · 8 comments
Open

Exhentai features #978

ghost opened this issue Sep 5, 2020 · 8 comments

Comments

@ghost
Copy link

ghost commented Sep 5, 2020

I'd like to see these features in a future update:
Numbered filenames. "001 - original name".
Download as a zip file with normal compression.
One metadata file in the zip file.
The title of the gallery should be the filename and nothing else. The gid or token should not be part of any filename at all.
Download several images at the same time.
A log file for failed downloads containing the URL.
A download should be treated as a failure if it can't download one image in its original resolution.

Pretty please.

@Fukitsu
Copy link

Fukitsu commented Sep 6, 2020

You can customize filenames and directory names and I think you can also download as zip if you configure the post-processor in the configuration file. A log file with failed downloads should already be possible using the config file too. Multi-threaded/concurrent downloads is planned for version 2.0 if I remember correctly, besides exhentai blocks automated downloaders that's why gallery-dl waits a random amount of seconds between each download/request.

@ghost
Copy link
Author

ghost commented Sep 6, 2020

How do you enable a log file for failed downloads and how do you customize the filename and directory name to do that? There's only an option for changing the directory. Exhentai doesn't block automated downloaders, it bans you if you make too many excessive page loads. That absurd delay has no purpose for Exhentai because it's going to get you banned anyways. It should be up to the user to decide the delay time and changing wait-min/max to 0 doesn't reduce the delay time at all.

@mikf
Copy link
Owner

mikf commented Sep 6, 2020

The following config snippet should cover most of what you want, except

  • "One metadata file in the zip file"
  • "Download several images at the same time"
{
    "extractor": {
        "exhentai": {
            "filename": "{num:>04} {filename}.{extension}",
            "directory": ["{title}"],
            "postprocessors": [{"name": "zip"}]
        }
    },
    "output": {
        "log": "/path/to/logfile.txt"
    }
}

it bans you if you make too many excessive page loads

And that's why there is delay between page loads

it's going to get you banned anyways

???

changing wait-min/max to 0 doesn't reduce the delay time at all.

Not true. There is still a 1-2 second delay for each original image, but nothing else.

Maybe you should try your luck with https://github.com/ccloli/E-Hentai-Downloader

@ghost
Copy link
Author

ghost commented Sep 7, 2020

I've been using that userscript but it's a tedious process to do it by hand which is why I'm asking for those features to be added to gallery-dl. The anti-rip ban on e-hentai and exhentai isn't permanent and the ban isn't done to the account (unless you paid for those perks) which means there are obvious workarounds for it. It should be up to the user to have a delay or not. That 1-2 second delay is useless and will get you banned either way. If you don't want to get banned, you need to change it to 35 seconds. Not so long ago I tried to figure out the perfect time for the delay and anything less than 35 seconds got me banned. If you don't believe me, use a 1-2 second delay or anything less than 35 seconds and download several CG galleries at once and make sure each image is 1MB or more and see for yourself.

Not true. There is still a 1-2 second delay for each original image, but nothing else.

That's what I'm saying. I changed the wait-min/max to 0 and 0.1, and the delay was still there.

What happens when gallery-dl can't download the original image due to H@H being wonky as always? Will it ignore the original image and go for the resized image instead?

@ghost
Copy link
Author

ghost commented Sep 7, 2020

Everything worked except for the logfile. I'm running into a problem with "log": "Y:\download\logfile.txt" and "log": "Y:/download/logfile.txt".

[config][warning] Could not parse 'C:\Users\User\gallery-dl\gallery-dl.conf': Expecting ',' delimiter: line 229 column 9 (char 5014)

edit: got it to work with this. Is there a way to write the name of the zip file to the log file whenever it fails to download an image or when it downloads a resized picture?

        "logfile": {
		    "format": "{asctime} {message}",
			"format-date": "%H:%M:%S -",
		    "path": "G:/1/log.txt",
            "mode": "w",
            "level": "info"
		},

@mikf
Copy link
Owner

mikf commented Sep 8, 2020

... and download several CG galleries at once and make sure each image is 1MB or more and see for yourself.

Well, I've only downloaded ~1000 manga/doujinshi galleries over the years and the current default settings worked fine (after my account got temporarily disabled for not using a delay at all)

What happens when gallery-dl can't download the original image due to H@H being wonky as always? Will it ignore the original image and go for the resized image instead?

No, it will report a failed download after retrying a couple of times.
It either downloads the original file or nothing at all.

Is there a way to write the name of the zip file to the log file whenever it fails to download an image

Not the zip file directly, but the (absolute) path to the individual file that failed to download by including {path.…} in the logging format string. For example
"format": "{asctime} {message} {path.realpath}"

@ghost
Copy link
Author

ghost commented Sep 8, 2020

"format": "{asctime} {message} {path.realpath}"
This works for me. Thank you.

I suppose this should be another issue but I'll ask here before making one, sometimes there are galleries with identical names and it doesn't respect that with these settings "postprocessors": [{"name": "zip","extension": "cbz"}]. How would you configure it so that it'll download galleries with the same name as separate zip files with "(1)" at the end?
Here are two galleries with similar names if you can't find any https://exhentai.org/g/1106700/7030db3644/ https://exhentai.org/g/799349/8666d90d0a/.

@mikf
Copy link
Owner

mikf commented Sep 10, 2020

You can't do that. It's not possible to have it recognize two zip file as separate galleries if they have the same name.

mikf added a commit that referenced this issue Sep 11, 2020
depend on 'wait-max', don't use a hard-coded value
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants