phantomjs 1.9.7 uses SSLv3 per default/fallback #12655
Comments
God, it took me an hour to figure out what the hell was suddenly going on. |
|
This is probably not the appropriate place to ask this buy my problem led me to here. My app that scrapes several https website worked perfectly yesterday, . This morning I am getting "SSL handshake failed" in my Phantomjs (injected casperjs) script. With phantomjs reporting this: [warning] [phantom] Loading resource failed with status=fail: https://www.myurl.com I have tried the --ignore-ssl-errors=true as well as the --ssl-protocol=tlsv1 flags but getting same error. I'm new to all of this but what the heck changed in the last 24 hours? My code has been working perfectly last 6 months. |
Maybe try |
i tried with the same result. SSL Handshake error. |
console.log('Loading a web page');
var page = require('webpage').create();
var url = "https://eff.org";
page.open(url, function (status) {
console.log(status);
//Page is loaded!
phantom.exit();
}); $ phantomjs --debug=yes sslv3_test.js 2014-10-16T15:54:25 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=' to enable persisten cookie storage) 2014-10-16T15:54:25 [DEBUG] Phantom - execute: Configuration 2014-10-16T15:54:25 [DEBUG] 0 objectName : "" 2014-10-16T15:54:25 [DEBUG] 1 cookiesFile : "" 2014-10-16T15:54:25 [DEBUG] 2 diskCacheEnabled : "false" 2014-10-16T15:54:25 [DEBUG] 3 maxDiskCacheSize : "-1" 2014-10-16T15:54:25 [DEBUG] 4 ignoreSslErrors : "false" 2014-10-16T15:54:25 [DEBUG] 5 localToRemoteUrlAccessEnabled : "false" 2014-10-16T15:54:25 [DEBUG] 6 outputEncoding : "UTF-8" 2014-10-16T15:54:25 [DEBUG] 7 proxyType : "http" 2014-10-16T15:54:25 [DEBUG] 8 proxy : ":1080" 2014-10-16T15:54:25 [DEBUG] 9 proxyAuth : ":" 2014-10-16T15:54:25 [DEBUG] 10 scriptEncoding : "UTF-8" 2014-10-16T15:54:25 [DEBUG] 11 webSecurityEnabled : "true" 2014-10-16T15:54:25 [DEBUG] 12 offlineStoragePath : "" 2014-10-16T15:54:25 [DEBUG] 13 offlineStorageDefaultQuota : "-1" 2014-10-16T15:54:25 [DEBUG] 14 printDebugMessages : "true" 2014-10-16T15:54:25 [DEBUG] 15 javascriptCanOpenWindows : "true" 2014-10-16T15:54:25 [DEBUG] 16 javascriptCanCloseWindows : "true" 2014-10-16T15:54:25 [DEBUG] 17 sslProtocol : "sslv3" 2014-10-16T15:54:25 [DEBUG] 18 sslCertificatesPath : "" 2014-10-16T15:54:25 [DEBUG] 19 webdriver : ":" 2014-10-16T15:54:25 [DEBUG] 20 webdriverLogFile : "" 2014-10-16T15:54:25 [DEBUG] 21 webdriverLogLevel : "INFO" 2014-10-16T15:54:25 [DEBUG] 22 webdriverSeleniumGridHub : "" 2014-10-16T15:54:25 [DEBUG] Phantom - execute: Script & Arguments 2014-10-16T15:54:25 [DEBUG] script: "sslv3_test.js" 2014-10-16T15:54:25 [DEBUG] Phantom - execute: Starting normal mode 2014-10-16T15:54:25 [DEBUG] WebPage - setupFrame "" 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/_coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/package.json" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./lexer.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././rewriter.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./parser.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./nodes.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././scope.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././lexer.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././rewriter.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:54:25 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r") ) ) Loading a web page 2014-10-16T15:54:25 [DEBUG] WebPage - updateLoadingProgress: 10 2014-10-16T15:54:25 [DEBUG] Network - Resource request error: 6 ( "SSL handshake failed" ) URL: "https://eff.org/" 2014-10-16T15:54:25 [DEBUG] WebPage - updateLoadingProgress: 100 $ phantomjs --ssl-protocol=TLSv1 --debug=true sslv3_test.js 2014-10-16T15:56:44 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=' to enable persisten cookie storage) 2014-10-16T15:56:44 [DEBUG] Phantom - execute: Configuration 2014-10-16T15:56:44 [DEBUG] 0 objectName : "" 2014-10-16T15:56:44 [DEBUG] 1 cookiesFile : "" 2014-10-16T15:56:44 [DEBUG] 2 diskCacheEnabled : "false" 2014-10-16T15:56:44 [DEBUG] 3 maxDiskCacheSize : "-1" 2014-10-16T15:56:44 [DEBUG] 4 ignoreSslErrors : "false" 2014-10-16T15:56:44 [DEBUG] 5 localToRemoteUrlAccessEnabled : "false" 2014-10-16T15:56:44 [DEBUG] 6 outputEncoding : "UTF-8" 2014-10-16T15:56:44 [DEBUG] 7 proxyType : "http" 2014-10-16T15:56:44 [DEBUG] 8 proxy : ":1080" 2014-10-16T15:56:44 [DEBUG] 9 proxyAuth : ":" 2014-10-16T15:56:44 [DEBUG] 10 scriptEncoding : "UTF-8" 2014-10-16T15:56:44 [DEBUG] 11 webSecurityEnabled : "true" 2014-10-16T15:56:44 [DEBUG] 12 offlineStoragePath : "" 2014-10-16T15:56:44 [DEBUG] 13 offlineStorageDefaultQuota : "-1" 2014-10-16T15:56:44 [DEBUG] 14 printDebugMessages : "true" 2014-10-16T15:56:44 [DEBUG] 15 javascriptCanOpenWindows : "true" 2014-10-16T15:56:44 [DEBUG] 16 javascriptCanCloseWindows : "true" 2014-10-16T15:56:44 [DEBUG] 17 sslProtocol : "tlsv1" 2014-10-16T15:56:44 [DEBUG] 18 sslCertificatesPath : "" 2014-10-16T15:56:44 [DEBUG] 19 webdriver : ":" 2014-10-16T15:56:44 [DEBUG] 20 webdriverLogFile : "" 2014-10-16T15:56:44 [DEBUG] 21 webdriverLogLevel : "INFO" 2014-10-16T15:56:44 [DEBUG] 22 webdriverSeleniumGridHub : "" 2014-10-16T15:56:44 [DEBUG] Phantom - execute: Script & Arguments 2014-10-16T15:56:44 [DEBUG] script: "sslv3_test.js" 2014-10-16T15:56:44 [DEBUG] Phantom - execute: Starting normal mode 2014-10-16T15:56:44 [DEBUG] WebPage - setupFrame "" 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/_coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/package.json" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./lexer.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././rewriter.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./parser.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./nodes.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././scope.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././lexer.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././rewriter.js" QMap(("mode", QVariant(QString, "r") ) ) 2014-10-16T15:56:44 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r") ) ) Loading a web page 2014-10-16T15:56:44 [DEBUG] WebPage - updateLoadingProgress: 10 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 15 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 16 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 23 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 27 2014-10-16T15:56:45 [DEBUG] WebPage - setupFrame "" 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 34 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 41 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 43 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 48 2014-10-16T15:56:45 [DEBUG] CookieJar - Saved "has_js=1; domain=www.eff.org; path=/" 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 48 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 52 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 55 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 57 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 60 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 62 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 64 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 67 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 70 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 72 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 77 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 80 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 82 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 82 2014-10-16T15:56:45 [DEBUG] WebPage - updateLoadingProgress: 100 success |
Thanks for gut checking me. I finally got it to work. I had to add two arguments for phantomjs PLUS I had the problem of the arguments where at the of cmd line. The arguments at the end work for casperjs but not for phantomjs. My phantomjs script injects casperjs and I switch around sometimes. phantomjs --debug=yes --ignore-ssl-errors=true --ssl-protocol=any ssltest.js |
Well, I spoke a little too soon. I can scrape most of my websites but I can't scrape British Airways using the the option --ssl-protocol=any. It hangs halfway thru the page load. I have to use --ssl-protocol=tlsv1 for it to work. The url is https://www.britishairways.com/travel/home/public/en_us As I have many websites that I scrape, am I now going to have to know exactly what ssl version each of them require and use the correct --ssl-protocol version for each one? Before Poodle I did not have to worry about it. It just worked. Will --ssl-protocol=all work in the future? Is this just a BA website issue? Thanks |
+1 |
1 similar comment
+1 |
This is now fixed in the latest patch release 1.9.8 (#12670). |
As stated in issue #12181 (comment) SSLv3 is used as default.
Since POODLE this is discouraged by every server.
Workaround is to call phantomjs with
--ssl-protocol=tlsv1
but it should not be the default.The text was updated successfully, but these errors were encountered: