Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use image caching proxies to download chapters #897

Closed
1 task done
galpt opened this issue May 26, 2024 · 2 comments
Closed
1 task done

Use image caching proxies to download chapters #897

galpt opened this issue May 26, 2024 · 2 comments

Comments

@galpt
Copy link
Contributor

galpt commented May 26, 2024

Describe your suggested feature

Theoretically, if it's possible to use an image caching proxy to optimize image load times, it's also possible to use the proxy to download the already cached images for users who prefer offline reading.

This could be several times faster compared to using direct ISP connection as the connection between servers (manga servers and image caching proxy servers such as wsrv.nl / 0ms.dev ) is a datacenter connection, some even call it "Premium Network" because the connection is far less congested compared to a home ISP connection where they share the Internet pipe with hundreds of houses.

When users "download" the chapters for offline reading, they only do it once, so they could read chapters multiple times without an Internet connection.

When users read chapters without download-ing them, they are technically downloading them, those images are just saved somewhere temporarily by the app/browser.

So in other word, if it's possible to use the image caching proxies to read the chapters, it's possible to use the proxied image links to download the chapters faster.

Caching proxies are designed for that specific purpose, serving the same files for faster downloads for a single user or multiple users.

The benefits will be:

  1. Users could download a chapter or several chapters faster compared to without using the caching proxy.
  2. Downloading the same chapters multiple times or from multiple users won't burden the original manga servers, since users are downloading from the image proxy.
  3. Downloading images from the image proxy could open the possibility of parallel downloads, with reduced chance of getting rate limited or IP ban from the original manga website. (Related to parallel download support #862)

Related to #771, loading the images one-by-one will feel slower.
The slowness effect of the 0ms.dev proxy can be solved if all links are fetched at the same time to let the proxy download all the images at once.
This has been tested using several manga titles from nhentai.net with each manga could have 100 - 300 images. No problem with fetching all images at once.
Fetching the typical manga consisting of 30-50 images surely won't be a problem.

All the images that have been fetched will make the proxy load the chapters faster, either for users who want to download for offline reading, or users who prefer online reading without downloading the chapters.

Like mentioned above, we could download chapters from the image proxy rather than only using direct ISP connection.
Since the images already cached by the image proxy, downloading will be faster and won't burden the original manga website.

Either we are downloading the images for offline reading or prefer to online read without downloading them, what we do is still essentially "downloading" the images, and caching proxies will have no problem with this type of use case.

I believe for all parsers that work with the image proxies, it should be possible to use the proxies as download servers for offline reading.
This might need some quick testing using several manga sources to verify if it works.

Acknowledgements

  • I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
@galpt
Copy link
Contributor Author

galpt commented May 26, 2024

Another thing to consider is that users' download speed won't be throttled when downloading the images via image proxies such as wsrv.nl & 0ms.dev, while it's very likely to get our download speed throttled when downloading chapters directly from the manga websites.

The worst speed throttling attempts I ever saw were my download speed got throttled to 64 KB/s or 128 KB/s, basically less than 1 MB/s.

The DownloadSlowdown logic is helpful to avoid getting rate limited or getting an IP ban from making so many requests in a short period of time, but won't be able to bypass the speed throttling.

This shouldn't happen when we download from big CDN providers such as Cloudflare, and wsrv.nl & 0ms.dev are tightly integrated with that.

So multiple downloads by multiple users from different countries with unthrottled download speed? Not a problem

Koitharu added a commit that referenced this issue Jul 7, 2024
@galpt
Copy link
Contributor Author

galpt commented Jul 7, 2024

I did a quick test downloading multiple chapters & quick-scrolling chapters to see if I'll get rate limited by the sources, but I didn't, so I believe it's working as expected.

Thanks for implementing the feature! I'm closing this issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant