Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some resource downloads return CloudFlare anti-DDOS challenge pages. #20

Open
PhonicUK opened this issue Jan 16, 2019 · 8 comments
Open

Comments

@PhonicUK
Copy link

For example:

http://api.spiget.org/v2/resources/9089/download

This should be a .zip file as per the resource data, but instead hitting that URL returns a cloudflare anti-ddos challenge page.

@wouterdedroog
Copy link

Yup, and it isn't an issue that can be resolved by InventiveTalent as far as I know. She recommended me to host it myself.

[17:00] inventivetalent: well honestly, in terms of downloads I do recommend hosting stuff yourself - Spigot's cloudflare protection just makes the whole thing unreliable

@InventivetalentDev
Copy link
Member

As MrWouter quoted, if you want to reliably download your resources your best option is accessing your own server to do so.
But I will add an additional check to the stuff that downloads resources to keep it from storing cloudflare pages, so you won't get any more corrupted .jar/.zip files.

@PhonicUK
Copy link
Author

Is it worth perhaps reaching out to the Spigot guys to see if they can give SpiGet direct access as an IP whitelist?

@InventivetalentDev
Copy link
Member

Is it worth perhaps reaching out to the Spigot guys to see if they can give SpiGet direct access as an IP whitelist?

Did that ages ago when I initially made spiget. Not happening.

@PhonicUK
Copy link
Author

Darn, well making sure that things that aren't valid zips aren't saved should go a long way towards improving it :)

@PhonicUK
Copy link
Author

I've been doing a little research and it may be possible to 'pass' the Cloudflare verification by using a headless browser. I've seen some approaches that actually just scrape and execute the javascript that it runs directly. But this does seen a really 'nasty' way to solve this particular problem.

@InventivetalentDev
Copy link
Member

The current way the SpigetClient accesses has proven to be stable enough to not need a different solution.

@PhonicUK
Copy link
Author

PhonicUK commented Apr 9, 2019

It may be worth combing through the existing plugin stores and removing/redownloading any that are either empty 0-length files or are HTML documents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants