Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please turn JavaScript on and reload the page #46

Closed
theRAGEhero opened this issue Apr 30, 2021 · 5 comments
Closed

Please turn JavaScript on and reload the page #46

theRAGEhero opened this issue Apr 30, 2021 · 5 comments

Comments

@theRAGEhero
Copy link

theRAGEhero commented Apr 30, 2021

Hello,
thank you for this awesome tool.

I noticed that it doesn't work on some pages, I suppose it is for Cloudflare ddos protection.

Please` turn JavaScript on and reload the page.
                                               
 DDoS protection by Cloudflare                 
                                               
 Ray ID: xxxxxxxxxxxxxxxx

Do you know if it's possible to bypass this error?
Thank you and have a good day,
Alessandro

@dgtlmoon
Copy link
Owner

Hi, unfortunately it will not support the situation where javascript is required, it is a 'dumb' or 'simple' approach to just fetching the HTML, converting it to text, and checking the difference..

@dgtlmoon
Copy link
Owner

dgtlmoon commented May 1, 2021

I added a note to the readme about this situation..

@dgtlmoon dgtlmoon closed this as completed May 1, 2021
@stefanberndtsson
Copy link

stefanberndtsson commented May 3, 2021

It is sort of possible via commandline using the chrome/chromium headless feature with repl.

echo 'document.getRootNode().documentElement.innerHTML' | chromium-browser --headless --disable-gpu --repl "$URL" 2> /dev/null | sed -e 's/^>>> //'

That will give you a JSON object out with result.value containing the dom html after body.onload (I think :) )

Something similar should be possible to do by talking to the chrome/chromium remote interface. From there it's also possible to take full screenshots of the page that could also be helpful.

@dgtlmoon
Copy link
Owner

dgtlmoon commented May 3, 2021

@stefanberndtsson that's a good idea, but we are only using the html -> text conversion in this project so far...

@dgtlmoon
Copy link
Owner

dgtlmoon commented Jul 4, 2021

@stefanberndtsson @theRAGEhero https://github.com/dgtlmoon/changedetection.io/tree/javascript-browser so there's a new experimental branch, I invite you to check it out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants