Skip to content

Add config option to ignore bad SSL certs when archiving#60

Closed
pirate wants to merge 2 commits intomasterfrom
ssl-check
Closed

Add config option to ignore bad SSL certs when archiving#60
pirate wants to merge 2 commits intomasterfrom
ssl-check

Conversation

@pirate
Copy link
Copy Markdown
Member

@pirate pirate commented Jan 9, 2018

In the modern age of HTTPS everywhere, it's becoming increasingly common for sites to set up SSL badly or forget to renew their certificates (especially older 90's era sites), rendering the sites inaccessible.
When browsing I always want to be alerted of bad certs, but when archiving 90% I want it to attempt archiving cites with bad certs anyway in case the content is important. After all, archiving a certificate error page is useless 100% of the time, so instead of throwing out hands up and skipping the site it's nice to at least attempt getting a snapshot.

TODO:

  • Add option to skip checking SSL certificates for validity
  • Implement fetch_wget cli option to skip cert check
  • Implement fetch_pdf cli option to skip cert check
  • Implement fetch_screenshot cli option to skip cert check

Last two steps just require doing some research on the chrome headless command line interface to find the right flags.

More Info:
- https://bugs.chromium.org/p/chromium/issues/detail?id=721739
- headless --ignore-certificate-errors
- https://groups.google.com/a/chromium.org/forum/#!topic/headless-dev/eiudRsYdc3A

@pirate pirate added this to the v0.0.4 milestone Jan 9, 2018
@ShubhGupta2125
Copy link
Copy Markdown

I am a beginner and would like to work on this issue. Can i take it up and be guided along?

@pirate
Copy link
Copy Markdown
Member Author

pirate commented Jan 20, 2018

Yeah, go for it @ShubhGupta2125 ! The next steps are to figure out how to make chromium-browser ignore SSL errors using a command line flag or some other setting.

@pirate
Copy link
Copy Markdown
Member Author

pirate commented Sep 24, 2018

I implemented this a little while ago, but thanks for your help @ShubhGupta2125

@pirate pirate closed this Sep 24, 2018
@pirate pirate deleted the ssl-check branch December 21, 2018 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants