Compilation of website fixes #3082
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ronny1982
changed the title
Robonau
reviewed
Feb 17, 2021
src/web/mjs/engine/Request.mjs
Outdated
| @@ -478,6 +478,13 @@ export default class Request { | |||
| } | |||
| delete details.requestHeaders['x-sec-fetch-site']; | |||
|
|
|||
| // HACK: Imgur does not support request with accept types containing other mimes then images | |||
| // => overwrite accept header to prevent redirection to HTML notice | |||
| if(/i\.imgur\.com/i.test(uri.hostname)) { | |||
There was a problem hiding this comment.
its not just imgur
many madara sites also need this
There was a problem hiding this comment.
see #2925 (comment)
Aloalivn hosting on Aloalivn responding with a web page and not an image
edit: just tested with this PR, it breaks Aloalivn as well as the other connectors that #2973 fixed
There was a problem hiding this comment.
note: the reason it had to be implemented anyways was a "fix" for mangaku which broke after a couple days anyways
There was a problem hiding this comment.
The previous Accept header was completely wrong. HakuNeko was telling each website for any request that it only accept images and no text, json, html, ...
It is a miracle that it lasted that long until MangaKu decided to respond correctly (in an attempt to block HakuNeko of course).
If there are more hosts that prefer to respond with HTML instead with the image when HakuNeko says "i accept html, text, json, images, ..." then detection for image links needs to be improved, so HakuNeko will say "i only accept images". As an alternative, affected connectors can still set the header in _handleConnectorURI.
There was a problem hiding this comment.
given _handleConnectorURI only deals with images, shouldn't the default behaviour be only accept images?
There was a problem hiding this comment.
the 73efd90 commit seems to have fixed all known cases, but there is still edge cases where it wouldn't work and would require _handleConnectorURI to be altered
the images not having an image extension
but is probably edge enough to not worry about
Sheepux
approved these changes
Feb 21, 2021
ronny1982
added a commit
that referenced
this pull request
Mar 15, 2021
* [fujoshibitches] removed website support * [gekkouscans] changed template, updated domain, referer for images * [lhscan] removed website support * [mundowuxia] camel case file naming * hack to fix imgur accept header globally * do not save the updated manga list if there are no entries * [goldenmangas] updated domain * [mangalib] fixed chapter list extraction * [hikariscan] changed template * [mangasforyou] removed website support * [mangajar] fixed CSS queries, simplified manga ID * [rawsenmanga] fixed CSS queries * improve image URL detection for enforced accept header * [mangasusureborn] update domain * review feedback
ronny1982
added a commit
that referenced
this pull request
Mar 15, 2021
* Update manmanapp.mjs (#3095) * fix _getMangas sukima (#3096) * [fujoshibitches] removed website support (#3082) * [gekkouscans] changed template, updated domain, referer for images (#3082) * [lhscan] removed website support (#3082) * [mundowuxia] camel case file naming (#3082) * hack to fix imgur accept header globally (#3082) * do not save the updated manga list if there are no entries (#3082) * [goldenmangas] updated domain (#3082) * [mangalib] fixed chapter list extraction (#3082) * [hikariscan] changed template (#3082) * [mangasforyou] removed website support (#3082) * [mangajar] fixed CSS queries, simplified manga ID (#3082) * [rawsenmanga] fixed CSS queries (#3082) * improve image URL detection for enforced accept header (#3082) * [mangasusureborn] update domain (#3082) * [shieldmanga] added website support (#3083) * [imperiodosotakus] added website support (#3083) * [geassscan] added website support (#3083) * [animecenterscan] added website support (#3083) * [WordPressMangastream] bypass wp.com proxy for images to download originals (#3083) * [manhwaland] added website support (#3083) * [mangamitsu] added website support (#3083) * [kumascans] added website support (#3083) * [mangasepeti] added website support (#3083) * [flamescans-org] added website support (#3083) * [dragontranslation] added website support (#3083) * [jiangzaitoon] added website support (#3083) * [manga18-us] added website support (#3083) * [manga18-club] added website support (#3083) * [Coreview] decrease resource usage (#3123) * [comicfx] added website support (#3104) * [irisscanlator] added website support (#3104) * [sindriascanlator] added website support (#3104) * [modescanlator] added website support (#3104) * [tonarinoyoungjump] decrease resource usage (#3104) * [animaregia] added website support (#3104) * [olhodalua] added website support (#3104) * [mangaweebs] added website support (#3104) * [manhwaid] added website support (#3104) * [mangareceh] updated domain and name (#3104) * [mangagreat] added website support (#3104) * [tonarinoyoungjump] revert duplicate optimization (#3104) * use mobile site for kuaikanmanhua for larger images. (#3155) Co-authored-by: adelie <adelie@users.noreply.github.com> * [sektekomik] updated protocol (#3143) * [komikmama] updated path (#3143) * [mangakid] updated domain (#3143) * [mangakita] updated path (#3143) * [westmanga] updated path (#3143) * [gurukomik] updated path (#3143) * [masterkomik] updated path (#3143) * [turktoon] update path, remove chapter title bloat (#3143) * [komiksave] added website support (#3143) * [drkmangas] added website support (#3143) * [gloryscans] added website support (#3143) * [mangadex] (#3143) + add retry (3x) for failed API requests - remove user defined servers + introduce multi-image server support (fallback) * [anikiga] updated protocol (#3143) * [wnacg] added website support (#3143) * [naniscans] added website support (#3143) * [manga18fx] added website support (#3143) * [mangasorigines] added website support (#3143) * [wordpressmangastream] increase image detection delay (for ts_reader not initialized yet) (#3143) * [ngomik] fixed captcha protected image CDN server and manga list extraction (#3143) * [manhuaes] changed template to WordPressMadara (#3143) * [mangadex] allow support for multiple md@h seeds (#3143) * [isekairaw] removed dead website (#3143) * [otakusekaiscan] added website support (#3143) * [hunterscan] added website support (#3143) * [manhwasmen] fixed CSS queries (#3143) * [hakaiscan] added website support (#3143) * [mangagenki] added website support (#3143) * [wordpressmadara] bypass proxy and ensure image.src has lowest priority (even when `data:image` url) (#3143) * [manhuaus] fixed image query (#3143) * [wordpressmangastream] prefer first server for images (#3143) * [imhentai] updated domain (#3143) * [mangadex] updated upstream server (#3143) * [mangadex] fix new api server (#3177) * [mangalivre] changed domain (#3177) * [ikifang] added website support (#3177) * [senpaiediciones] added website support (#3177) * [samuraiscan] changed template (#3177) * [toonily] fixed manga list extraction (#3177) * [pmscans] updated protocol (#3177) * [mangaboruto] updated domain and fixed chapter extraction (#3177) * [secretscans] changed domain and name (#3177) * [manhuaus] fixed image extraction (#3177) * [moonloversscan] added website support (#3177) * [littlemonsterscan] added website support (#3177) * [mangadex] improve api URL usage (#3177) * [neoxscan] updated domain (#3177) * [mangalife] fixed image link extraction (#3177) * [unionmangas] changed domain, fixed image link extraction (#3177) * [mangalife] bypass dynamic variable protection (#3177) * Update NewToki.mjs (#3193) * leviatanscans to madara (#3195) resolves #3186 Co-authored-by: Ever4Engel <47645041+Ever4engel@users.noreply.github.com> Co-authored-by: robo <30987265+Robonau@users.noreply.github.com> Co-authored-by: cyn, a small penguin <5573802+adelie@users.noreply.github.com> Co-authored-by: adelie <adelie@users.noreply.github.com> Co-authored-by: Quazzzi <80200413+Quazzzi@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #2769
Resolves #2899
Resolves #2920
Resolves #2923
References #2930
Resolves #3034
Resolves #3050
Resolves #3051
Resolves #3053
Resolves #3068