This repository has been archived by the owner. It is now read-only.

phantomjs 1.9.7 uses SSLv3 per default/fallback #12655

Closed
Wayneoween opened this Issue Oct 16, 2014 · 11 comments

Comments

Projects
None yet
8 participants
@Wayneoween

Wayneoween commented Oct 16, 2014

As stated in issue #12181 (comment) SSLv3 is used as default.

Since POODLE this is discouraged by every server.

Workaround is to call phantomjs with --ssl-protocol=tlsv1 but it should not be the default.

@jrf0110

This comment has been minimized.

jrf0110 commented Oct 16, 2014

God, it took me an hour to figure out what the hell was suddenly going on.

@jcready

This comment has been minimized.

jcready commented Oct 16, 2014

👍

@mrfatboy

This comment has been minimized.

mrfatboy commented Oct 16, 2014

This is probably not the appropriate place to ask this buy my problem led me to here.

My app that scrapes several https website worked perfectly yesterday, . This morning I am getting "SSL handshake failed" in my Phantomjs (injected casperjs) script. With phantomjs reporting this:

[warning] [phantom] Loading resource failed with status=fail: https://www.myurl.com

I have tried the --ignore-ssl-errors=true as well as the --ssl-protocol=tlsv1 flags but getting same error. I'm new to all of this but what the heck changed in the last 24 hours? My code has been working perfectly last 6 months.

@jrf0110

This comment has been minimized.

jrf0110 commented Oct 16, 2014

Maybe try --ssl-protocol=any?

@mrfatboy

This comment has been minimized.

mrfatboy commented Oct 16, 2014

i tried
--ssl-protocol=any
--ssl-protocol=[sslv3|sslv2|tlsv1|any]
--ssl-protocol=[sslv3|sslv2|tlsv1|any'] - I found this in another post but the ' crashes my script. Maybe typo?

with the same result. SSL Handshake error.

@Hainish

This comment has been minimized.

Hainish commented Oct 16, 2014

👍

console.log('Loading a web page');
var page = require('webpage').create();
var url = "https://eff.org";
page.open(url, function (status) {
  console.log(status);
  //Page is loaded!
  phantom.exit();
});
$ phantomjs --debug=yes sslv3_test.js
2014-10-16T15:54:25 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=' to enable persisten cookie storage) 
2014-10-16T15:54:25 [DEBUG] Phantom - execute: Configuration 
2014-10-16T15:54:25 [DEBUG]      0 objectName : "" 
2014-10-16T15:54:25 [DEBUG]      1 cookiesFile : "" 
2014-10-16T15:54:25 [DEBUG]      2 diskCacheEnabled : "false" 
2014-10-16T15:54:25 [DEBUG]      3 maxDiskCacheSize : "-1" 
2014-10-16T15:54:25 [DEBUG]      4 ignoreSslErrors : "false" 
2014-10-16T15:54:25 [DEBUG]      5 localToRemoteUrlAccessEnabled : "false" 
2014-10-16T15:54:25 [DEBUG]      6 outputEncoding : "UTF-8" 
2014-10-16T15:54:25 [DEBUG]      7 proxyType : "http" 
2014-10-16T15:54:25 [DEBUG]      8 proxy : ":1080" 
2014-10-16T15:54:25 [DEBUG]      9 proxyAuth : ":" 
2014-10-16T15:54:25 [DEBUG]      10 scriptEncoding : "UTF-8" 
2014-10-16T15:54:25 [DEBUG]      11 webSecurityEnabled : "true" 
2014-10-16T15:54:25 [DEBUG]      12 offlineStoragePath : "" 
2014-10-16T15:54:25 [DEBUG]      13 offlineStorageDefaultQuota : "-1" 
2014-10-16T15:54:25 [DEBUG]      14 printDebugMessages : "true" 
2014-10-16T15:54:25 [DEBUG]      15 javascriptCanOpenWindows : "true" 
2014-10-16T15:54:25 [DEBUG]      16 javascriptCanCloseWindows : "true" 
2014-10-16T15:54:25 [DEBUG]      17 sslProtocol : "sslv3" 
2014-10-16T15:54:25 [DEBUG]      18 sslCertificatesPath : "" 
2014-10-16T15:54:25 [DEBUG]      19 webdriver : ":" 
2014-10-16T15:54:25 [DEBUG]      20 webdriverLogFile : "" 
2014-10-16T15:54:25 [DEBUG]      21 webdriverLogLevel : "INFO" 
2014-10-16T15:54:25 [DEBUG]      22 webdriverSeleniumGridHub : "" 
2014-10-16T15:54:25 [DEBUG] Phantom - execute: Script & Arguments 
2014-10-16T15:54:25 [DEBUG]      script: "sslv3_test.js" 
2014-10-16T15:54:25 [DEBUG] Phantom - execute: Starting normal mode 
2014-10-16T15:54:25 [DEBUG] WebPage - setupFrame "" 
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/_coffee-script.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/package.json" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/coffee-script.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./lexer.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././rewriter.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././helpers.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./parser.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./helpers.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./nodes.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././scope.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././helpers.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././lexer.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././rewriter.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r") ) )  
Loading a web page
2014-10-16T15:54:25 [DEBUG] WebPage - updateLoadingProgress: 10 
2014-10-16T15:54:25 [DEBUG] Network - Resource request error: 6 ( "SSL handshake failed" ) URL: "https://eff.org/" 
2014-10-16T15:54:25 [DEBUG] WebPage - updateLoadingProgress: 100 
$ phantomjs --ssl-protocol=TLSv1 --debug=true sslv3_test.js 
2014-10-16T15:56:44 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=' to enable persisten cookie storage) 
2014-10-16T15:56:44 [DEBUG] Phantom - execute: Configuration 
2014-10-16T15:56:44 [DEBUG]      0 objectName : "" 
2014-10-16T15:56:44 [DEBUG]      1 cookiesFile : "" 
2014-10-16T15:56:44 [DEBUG]      2 diskCacheEnabled : "false" 
2014-10-16T15:56:44 [DEBUG]      3 maxDiskCacheSize : "-1" 
2014-10-16T15:56:44 [DEBUG]      4 ignoreSslErrors : "false" 
2014-10-16T15:56:44 [DEBUG]      5 localToRemoteUrlAccessEnabled : "false" 
2014-10-16T15:56:44 [DEBUG]      6 outputEncoding : "UTF-8" 
2014-10-16T15:56:44 [DEBUG]      7 proxyType : "http" 
2014-10-16T15:56:44 [DEBUG]      8 proxy : ":1080" 
2014-10-16T15:56:44 [DEBUG]      9 proxyAuth : ":" 
2014-10-16T15:56:44 [DEBUG]      10 scriptEncoding : "UTF-8" 
2014-10-16T15:56:44 [DEBUG]      11 webSecurityEnabled : "true" 
2014-10-16T15:56:44 [DEBUG]      12 offlineStoragePath : "" 
2014-10-16T15:56:44 [DEBUG]      13 offlineStorageDefaultQuota : "-1" 
2014-10-16T15:56:44 [DEBUG]      14 printDebugMessages : "true" 
2014-10-16T15:56:44 [DEBUG]      15 javascriptCanOpenWindows : "true" 
2014-10-16T15:56:44 [DEBUG]      16 javascriptCanCloseWindows : "true" 
2014-10-16T15:56:44 [DEBUG]      17 sslProtocol : "tlsv1" 
2014-10-16T15:56:44 [DEBUG]      18 sslCertificatesPath : "" 
2014-10-16T15:56:44 [DEBUG]      19 webdriver : ":" 
2014-10-16T15:56:44 [DEBUG]      20 webdriverLogFile : "" 
2014-10-16T15:56:44 [DEBUG]      21 webdriverLogLevel : "INFO" 
2014-10-16T15:56:44 [DEBUG]      22 webdriverSeleniumGridHub : "" 
2014-10-16T15:56:44 [DEBUG] Phantom - execute: Script & Arguments 
2014-10-16T15:56:44 [DEBUG]      script: "sslv3_test.js" 
2014-10-16T15:56:44 [DEBUG] Phantom - execute: Starting normal mode 
2014-10-16T15:56:44 [DEBUG] WebPage - setupFrame "" 
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/_coffee-script.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/package.json" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/coffee-script.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./lexer.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././rewriter.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././helpers.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./parser.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./helpers.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./nodes.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././scope.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././helpers.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././lexer.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././rewriter.js" QMap(("mode", QVariant(QString, "r") ) )  
2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r") ) )  
Loading a web page
2014-10-16T15:56:44 [DEBUG] WebPage - updateLoadingProgress: 10 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 15 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 16 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 23 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 27 
2014-10-16T15:56:45 [DEBUG] WebPage - setupFrame "" 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 34 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 41 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 43 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 48 
2014-10-16T15:56:45 [DEBUG] CookieJar - Saved "has_js=1; domain=www.eff.org; path=/" 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 48 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 52 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 55 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 57 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 60 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 62 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 64 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 67 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 70 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 72 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 77 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 80 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 82 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 82 
2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 100 
success
@mrfatboy

This comment has been minimized.

mrfatboy commented Oct 16, 2014

Thanks for gut checking me. I finally got it to work. I had to add two arguments for phantomjs PLUS I had the problem of the arguments where at the of cmd line. The arguments at the end work for casperjs but not for phantomjs. My phantomjs script injects casperjs and I switch around sometimes.

phantomjs --debug=yes --ignore-ssl-errors=true --ssl-protocol=any ssltest.js

@mrfatboy

This comment has been minimized.

mrfatboy commented Oct 16, 2014

Well, I spoke a little too soon. I can scrape most of my websites but I can't scrape British Airways using the the option --ssl-protocol=any. It hangs halfway thru the page load. I have to use --ssl-protocol=tlsv1 for it to work.

The url is https://www.britishairways.com/travel/home/public/en_us

As I have many websites that I scrape, am I now going to have to know exactly what ssl version each of them require and use the correct --ssl-protocol version for each one? Before Poodle I did not have to worry about it. It just worked.

Will --ssl-protocol=all work in the future? Is this just a BA website issue?

Thanks

@micahwedemeyer

This comment has been minimized.

micahwedemeyer commented Oct 17, 2014

+1

1 similar comment
@pasangsherpa

This comment has been minimized.

pasangsherpa commented Oct 17, 2014

+1

@ariya

This comment has been minimized.

Owner

ariya commented Oct 25, 2014

This is now fixed in the latest patch release 1.9.8 (#12670).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.