Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL issue #8

Closed
sennierer opened this issue Mar 6, 2014 · 14 comments
Closed

SSL issue #8

sennierer opened this issue Mar 6, 2014 · 14 comments
Labels

Comments

@sennierer
Copy link

Hi,

we had your great too already running, but since a month or so it won't start anymore.
In the track.error.log I found an error message regarding SSL:
'error' => 'error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure',
'errno' => 35,

Does anybody have an idea why this is occurring and how we can solve the problem?

Thanks very much!

@ErikBorra ErikBorra added bug and removed bug labels Mar 6, 2014
@ErikBorra
Copy link
Member

Update

The problem occurred because sennierer's openssel did not accept the cipher used by Twitter. To fix this problem do the following:

  • open the file capture/common/tmhOAuth/tmhOAuth.php
  • At around line 783 you see a call to curl_setopt_array() and the following lines set the curl parameters.
  • Add the following line there: CURLOPT_SSL_CIPHER_LIST => 'ALL',

Original comment

Hi Sennierer,

this seems to be an error with CURL. According to http://curl.haxx.se/libcurl/c/libcurl-errors.html error 35 means that a "problem occurred somewhere in the SSL/TLS handshake. You really want the error buffer and read the message there as it pinpoints the problem slightly more. Could be certificates (file formats, paths, permissions), passwords, and others."

Would you mind sending me your config.php and the full track.error.log via email? Also, please do the following cd /var/www/dmi-tcat/capture/stream; php track.php and let me know whether any errors are returned.

Best,

Erik

@supersambo
Copy link

hi Erik,
I'm on the same machine as sennierer.
php track.php does not return any error (apart from the deprected mysql_connect() warning)
I'm not sure how to enable the error buffer you pointed us to.
My naive guess was to insert
CURLOPT_ERRORBUFFER => $this->config['errorBuffer'],
on line 784 of capture/common/tmhOAuth/tmhOAuth.php
but this didn't work.

Best,
stephan

@ErikBorra
Copy link
Member

Hi Stephan,

maybe you could try the following:

  • look at capture/common/tmhOAuth/cacert.pam and see whether it is readable for the user with which you run dmi-tcat. You can get the latest cacert.pem from http://curl.haxx.se/ca/cacert.pem
  • verify whether curl functions properly: go to https://apps.twitter.com/ > click the app you created for dmi-tcat > click 'test OAuth' > fill in https://stream.twitter.com/1.1/statuses/filter.json for 'Request URI' and track=test for 'Request Query' and click 'See OAuth signature for this request'. Copy-paste the resulting curl command in your console, hit enter and inspect the results.
  • send me the full message of track.error.log with errno 35

Best,

Erik

@supersambo
Copy link

Hi Erik,
thank you for your help.

  • capture/common/tmhOAuth/cacert.pam is read and writeable so I did not download it again
  • curl returns the following
 curl --get 'https://stream.twitter.com/1.1/statuses/filter.json' --data 'test=track' --header 'Authorization: OAuth oauth_consumer_key="XXX", oauth_nonce="XXX", oauth_signature="XXX", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1394528407", oauth_token="XXX", oauth_version="1.0"' --verbose
 Hostname was NOT found in DNS cache
*  Trying 199.16.156.20...
* Connected to stream.twitter.com (199.16.156.20) port 443 (#0)
* successfully set certificate verify locations:
   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Server hello (2):
* error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
* Closing connection 0
curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
  • and here is the full error message
2014-03-11 10:24:01 connecting to API socket
2014-03-11 10:24:01 connecting - query array (
  'track' => 'minimum alcohol pricing,minimum pricing alcohol,harmful drinking,minimum unit pricing,global warming,globalwarming,climate,climatechange',
)
2014-03-11 10:24:01 stream stopped - error tmhOAuth::__set_state(array(
   'response' => 
  array (
    'raw' => '',
    'content-length' => 0,
    'code' => 0,
    'response' => false,
    'info' => 
    array (
      'url' => 'https://stream.twitter.com/1.1/statuses/filter.json',
      'content_type' => NULL,
      'http_code' => 0,
      'header_size' => 0,
      'request_size' => 0,
      'filetime' => -1,
      'ssl_verify_result' => 0,
      'redirect_count' => 0,
      'total_time' => 0.270964,
      'namelookup_time' => 0.028419,
      'connect_time' => 0.143359,
      'pretransfer_time' => 0,
      'size_upload' => 0,
      'size_download' => 0,
      'speed_download' => 0,
      'speed_upload' => 0,
      'download_content_length' => -1,
      'upload_content_length' => -1,
      'starttransfer_time' => 0,
      'redirect_time' => 0,
      'redirect_url' => '',
      'primary_ip' => '199.16.156.110',
      'certinfo' => 
      array (
      ),
      'primary_port' => 443,
      'local_ip' => '158.255.212.46',
      'local_port' => 52148,
    ),
    'error' => 'error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure',
    'errno' => 35,
  ),
   'buffer' => NULL,
   'config' => 
  array (
    'user_agent' => 'tmhOAuth 0.8.3+SSL - //github.com/themattharris/tmhOAuth',
    'host' => 'stream.twitter.com',
    'consumer_key' => 'XXX',
    'consumer_secret' => 'XXX',
    'token' => 'XXX',
    'secret' => '',
    'bearer' => '',
    'oauth_version' => '1.0',
    'oauth_signature_method' => 'HMAC-SHA1',
    'curl_http_version' => 2,
    'curl_connecttimeout' => 30,
    'curl_timeout' => 10,
    'curl_ssl_verifyhost' => 2,
    'curl_ssl_verifypeer' => true,
    'use_ssl' => true,
    'curl_cainfo' => '/home/supersambo/www/dmi-tcat/capture/common/tmhOAuth/cacert.pem',
    'curl_capath' => '/home/supersambo/www/dmi-tcat/capture/common/tmhOAuth',
    'curl_followlocation' => false,
    'curl_proxy' => false,
    'curl_proxyuserpwd' => false,
    'curl_encoding' => '',
    'is_streaming' => true,
    'streaming_eol' => '
',
    'streaming_metrics_interval' => 10,
    'as_header' => true,
    'force_nonce' => false,
    'force_timestamp' => false,
    'streaming_callback' => 'streamCallback',
  ),
   'request_settings' => 
  array (
    'params' => 
    array (
      'track' => 'minimum alcohol pricing,minimum pricing alcohol,harmful drinking,minimum unit pricing,global warming,globalwarming,climate,climatechange',
    ),
    'headers' => 
    array (
      'Host' => 'stream.twitter.com',
      'Authorization' => 'OAuth oauth_consumer_key="XXX", oauth_nonce="XXX", oauth_signature="XXX", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1394533441", oauth_token="XXX", oauth_version="1.0"',
    ),
    'with_user' => true,
    'multipart' => false,
    'method' => 'POST',
    'without_bearer' => false,
    'url' => 'https://stream.twitter.com/1.1/statuses/filter.json',
    'oauth1_params' => 
    array (
      'oauth_consumer_key' => 'XXX',
      'oauth_nonce' => 'XXX',
      'oauth_signature' => 'XXX%2BwrApk%3D',
      'oauth_signature_method' => 'HMAC-SHA1',
      'oauth_timestamp' => '1394533441',
      'oauth_token' => 'XXX',
      'oauth_version' => '1.0',
    ),
    'prepared_params' => 
    array (
      'track' => 'minimum%20alcohol%20pricing%2Cminimum%20pricing%20alcohol%2Charmful%20drinking%2Cminimum%20unit%20pricing%2Cglobal%20warming%2Cglobalwarming%2Cclimate%2Cclimatechange',
    ),
    'basestring_params' => 'oauth_consumer_key=XXX&oauth_nonce=2c9e492e6fdddb9d2bf022acb04d30b7&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1394533441&oauth_token=XXX&oauth_version=1.0&track=minimum%20alcohol%20pricing%2Cminimum%20pricing%20alcohol%2Charmful%20drinking%2Cminimum%20unit%20pricing%2Cglobal%20warming%2Cglobalwarming%2Cclimate%2Cclimatechange',
    'postfields' => 'track=minimum%20alcohol%20pricing%2Cminimum%20pricing%20alcohol%2Charmful%20drinking%2Cminimum%20unit%20pricing%2Cglobal%20warming%2Cglobalwarming%2Cclimate%2Cclimatechange',
    'basestring' => 'POST&https%3A%2F%2Fstream.twitter.com%2F1.1%2Fstatuses%2Ffilter.json&oauth_consumer_key%3DXXXoauth_nonce%XXX%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1394533441%26oauth_token%3DXXXoauth_version%3D1.0%26track%3Dminimum%2520alcohol%2520pricing%252Cminimum%2520pricing%2520alcohol%252Charmful%2520drinking%252Cminimum%2520unit%2520pricing%252Cglobal%2520warming%252Cglobalwarming%252Cclimate%252Cclimatechange',
    'signing_key' => 'XXX',
  ),
   'metrics' => 
  array (
    'start' => 1394533441,
    'interval_start' => 1394533441,
    'messages' => 0,
    'last_messages' => 0,
    'bytes' => 0,
    'last_bytes' => 0,
  ),
))
2014-03-11 10:24:01 processing buffer before exit
2014-03-11 10:24:01 automatically restarting ...

Best,
stephan

@dentoir
Copy link
Contributor

dentoir commented Mar 11, 2014

Hi Stephan,

It seams the curl command is failing early in the SSL connection, even before receiving the server certificate. That is strange. But at least we know now it is not php/oauth related.

Can you do a test on the command line (to reproduce outside php, and enforce TLSv1) with this command:
curl --get 'https://stream.twitter.com/1.1/statuses/filter.json' -1 --verbose

And produce the output (hopefully this will get further, to where you see: * SSL certificate verify ok.)

Also, can you send the output of the following commands from the console:
curl --version
(this will give you the libraries curl is compiled against)
and
uname -a
(your operating system version)

This might give more clues. Do you run other (ssl related) webapps on the machine? Does the following command on the console get you passed the handshake?

curl --get 'https://tools.digitalmethods.net/' -1 --verbose

Cheers,

Emile

@supersambo
Copy link

Hi Emile,
thank you very much for helping. here are the outputs of the commands

~$ curl --get 'https://stream.twitter.com/1.1/statuses/filter.json' -1 --verbose
* Hostname was NOT found in DNS cache
*   Trying 199.16.156.20...
* Connected to stream.twitter.com (199.16.156.20) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Server hello (2):
* error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
* Closing connection 0
curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

~$ curl --version
curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 libssh2/1.4.3 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smtp smtps telnet tftp 
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP 
supersambo@sensamb:~$ uname -a
Linux sensamb.schloegl.net 2.6.32-5-amd64 #1 SMP Fri May 10 08:43:19 UTC 2013 x86_64 GNU/Linux

~$ curl --get 'https://tools.digitalmethods.net/' -1 --verbose
* Hostname was NOT found in DNS cache
*   Trying 82.94.190.199...
* Connected to tools.digitalmethods.net (82.94.190.199) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*    subject: OU=Domain Control Validated; OU=EssentialSSL Wildcard; CN=*.digitalmethods.net
*    start date: 2013-02-26 00:00:00 GMT
*    expire date: 2016-02-26 23:59:59 GMT
*    subjectAltName: tools.digitalmethods.net matched
*    issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=EssentialSSL CA
*    SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: tools.digitalmethods.net
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Date: Tue, 11 Mar 2014 13:14:48 GMT
* Server Apache/2.2.22 (FreeBSD) PHP/5.4.6 mod_ssl/2.2.22 OpenSSL/0.9.8x DAV/2 is not blacklisted
< Server: Apache/2.2.22 (FreeBSD) PHP/5.4.6 mod_ssl/2.2.22 OpenSSL/0.9.8x DAV/2
< Location: https://wiki.digitalmethods.net/Dmi/ToolDatabase
< Vary: Accept-Encoding
< Content-Length: 256
< Content-Type: text/html; charset=iso-8859-1
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://wiki.digitalmethods.net/Dmi/ToolDatabase">here</a>.</p>
</body></html>
* Connection #0 to host tools.digitalmethods.net left intact

Best regards,
stephan

@dentoir
Copy link
Contributor

dentoir commented Mar 11, 2014

At this point I am guessing that the cipher the Twitter server uses is not accepted by your client due to updates on either side. But the curl output doesn't show this. The following console command tests openssl connection to Twitter specifically. It might reveal more information.

openssl s_client -state -nbio -connect stream.twitter.com:443

Can you post the output of this command? You have an up-to-date openssl library, but it might have been compiled to not allow the specific cipher (RC4-SHA) which Twitter currently likes.
The following command lists all the ciphers your client supports.

openssl ciphers -v

RC4-SHA should be among them. Let's also try a lenient curl connect by adding a flag.

curl --ciphers ALL --get 'https://stream.twitter.com/1.1/statuses/filter.json' -1 --verbose

You can you give the output of these commands?
It might be worth opening a ticket on the dev.twitter.com developers forum at this time and post the output of the curl and openssl connect commands there.

@supersambo
Copy link

Hi Emile,
these are the outputs:

~$ openssl s_client -state -nbio -connect stream.twitter.com:443
CONNECTED(00000003)
turning on non blocking io
SSL_connect:before/connect initialization
SSL_connect:unknown state
SSL_connect:error in unknown state
write R BLOCK
SSL_connect:SSLv3 read server hello A
depth=1 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = Terms of use at https://www.verisign.com/rpa (c)10, CN = VeriSign Class 3 Secure Server CA - G3
verify error:num=20:unable to get local issuer certificate
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:error in SSLv3 read finished A
SSL_connect:error in SSLv3 read finished A
read R BLOCK
SSL_connect:SSLv3 read finished A
read R BLOCK
---
Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=Twitter, Inc./OU=Twitter Security/CN=stream.twitter.com
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
 1 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFhzCCBG+gAwIBAgIQIZ2nY6JKVZJ0ancgDflZdzANBgkqhkiG9w0BAQUFADCB
tTELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2Ug
YXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMm
VmVyaVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwHhcNMTMxMDA5
MDAwMDAwWhcNMTYxMjMwMjM1OTU5WjCBijELMAkGA1UEBhMCVVMxEzARBgNVBAgT
CkNhbGlmb3JuaWExFjAUBgNVBAcUDVNhbiBGcmFuY2lzY28xFjAUBgNVBAoUDVR3
aXR0ZXIsIEluYy4xGTAXBgNVBAsUEFR3aXR0ZXIgU2VjdXJpdHkxGzAZBgNVBAMU
EnN0cmVhbS50d2l0dGVyLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAL+5NMskUpdZukJAp2BICavTUladFyEDnzqUxF+XNAumgmccYmKbxn6tr5uP
zDudcPNPN+BLYFwT5qdq38G13BVU2Ljcd+auEH9p74DJkgn49uyg6WeMJiQwD59x
ua4DUjzFnrnNEduiB8gEuOAQn/NbD1hotPymeCYs6qE/4pxt6ZiTMhv6TwWR8jTB
eWREp4T/IqJKaZBqobCJ7QU3YXhESXIN0dCnWCPBdLKS/clS+IyV7m93s8tegj2n
Za+c5qrDTj4q6R6t/7uNCanb6bqK9FPK+a0PpP66UfxUQe5Sd20BOS/6JDaPtOmd
d9YV5aVIwyxxPXF2XK/TtqJ1Pj0CAwEAAaOCAbowggG2MFUGA1UdEQROMEyCGnBh
cnRuZXJzdHJlYW0xLnR3aXR0ZXIuY29tghpwYXJ0bmVyc3RyZWFtMi50d2l0dGVy
LmNvbYISc3RyZWFtLnR3aXR0ZXIuY29tMAkGA1UdEwQCMAAwDgYDVR0PAQH/BAQD
AgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjBDBgNVHSAEPDA6MDgG
CmCGSAGG+EUBBzYwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24u
Y29tL2NwczAfBgNVHSMEGDAWgBQNRFwWU0TBgn4dIKsl9AFj2L55pTBFBgNVHR8E
PjA8MDqgOKA2hjRodHRwOi8vU1ZSU2VjdXJlLUczLWNybC52ZXJpc2lnbi5jb20v
U1ZSU2VjdXJlRzMuY3JsMHYGCCsGAQUFBwEBBGowaDAkBggrBgEFBQcwAYYYaHR0
cDovL29jc3AudmVyaXNpZ24uY29tMEAGCCsGAQUFBzAChjRodHRwOi8vU1ZSU2Vj
dXJlLUczLWFpYS52ZXJpc2lnbi5jb20vU1ZSU2VjdXJlRzMuY2VyMA0GCSqGSIb3
DQEBBQUAA4IBAQBPiaEVJXaDoc1lEv/FkN20ShHAjrlw3zy4jbxXJl81/xtqD3Y2
ZD4QHsjS6BTwIQ5E4rbBkWKi0kNrmiEGzf659qkoW+U/j/Hj87+Tan/PdFXbrXqz
Faia4xCv/WvsmzgQDMpulsyvUyRBGdzLnJfLULm7nC3y/Pqe9/zxy6afLjMqSsnN
+hpGFymJpnwFmWU5EKPIOQbJUnnbRVf4Jrhdw3VEMEWsIsOjxLi8U+YU7kkMP9Kp
dBVnI53OjVvwTYapZS4YysWJs/Ig42WqK478PaXQB4PuQAKSXGnHh/2DhO/iqBJn
Ewt+rvz+6CwYby3UPhqKsDCLo31bYaJiUJEb
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=San Francisco/O=Twitter, Inc./OU=Twitter Security/CN=stream.twitter.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
---
No client certificate CA names sent
---
SSL handshake has read 3089 bytes and written 631 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : RC4-SHA
    Session-ID: 531F2354ED19A79A950830BDB1B823A03066EC7C0E0837CC2FFA0DB011CE16C8
    Session-ID-ctx: 
    Master-Key: AB160A21E2D7EF9C0DF9E5B5B1ECD2B35C546F82A4E59C83F11EBE11F13B28B1C52BB48AB10D78EE05E9A8D6827F8A9C
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1394549588
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---
SSL3 alert read:warning:close notify
closed
SSL3 alert write:warning:close notify

~$ openssl ciphers -v
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA    SSLv3 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA1
ECDHE-ECDSA-AES256-SHA  SSLv3 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA1
SRP-DSS-AES-256-CBC-SHA SSLv3 Kx=SRP      Au=DSS  Enc=AES(256)  Mac=SHA1
SRP-RSA-AES-256-CBC-SHA SSLv3 Kx=SRP      Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-DSS-AES256-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES256-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA1
DHE-RSA-CAMELLIA256-SHA SSLv3 Kx=DH       Au=RSA  Enc=Camellia(256) Mac=SHA1
DHE-DSS-CAMELLIA256-SHA SSLv3 Kx=DH       Au=DSS  Enc=Camellia(256) Mac=SHA1
ECDH-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AESGCM(256) Mac=AEAD
ECDH-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AESGCM(256) Mac=AEAD
ECDH-RSA-AES256-SHA384  TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AES(256)  Mac=SHA384
ECDH-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AES(256)  Mac=SHA384
ECDH-RSA-AES256-SHA     SSLv3 Kx=ECDH/RSA Au=ECDH Enc=AES(256)  Mac=SHA1
ECDH-ECDSA-AES256-SHA   SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=AES(256)  Mac=SHA1
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
CAMELLIA256-SHA         SSLv3 Kx=RSA      Au=RSA  Enc=Camellia(256) Mac=SHA1
PSK-AES256-CBC-SHA      SSLv3 Kx=PSK      Au=PSK  Enc=AES(256)  Mac=SHA1
ECDHE-RSA-DES-CBC3-SHA  SSLv3 Kx=ECDH     Au=RSA  Enc=3DES(168) Mac=SHA1
ECDHE-ECDSA-DES-CBC3-SHA SSLv3 Kx=ECDH     Au=ECDSA Enc=3DES(168) Mac=SHA1
SRP-DSS-3DES-EDE-CBC-SHA SSLv3 Kx=SRP      Au=DSS  Enc=3DES(168) Mac=SHA1
SRP-RSA-3DES-EDE-CBC-SHA SSLv3 Kx=SRP      Au=RSA  Enc=3DES(168) Mac=SHA1
EDH-RSA-DES-CBC3-SHA    SSLv3 Kx=DH       Au=RSA  Enc=3DES(168) Mac=SHA1
EDH-DSS-DES-CBC3-SHA    SSLv3 Kx=DH       Au=DSS  Enc=3DES(168) Mac=SHA1
ECDH-RSA-DES-CBC3-SHA   SSLv3 Kx=ECDH/RSA Au=ECDH Enc=3DES(168) Mac=SHA1
ECDH-ECDSA-DES-CBC3-SHA SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=3DES(168) Mac=SHA1
DES-CBC3-SHA            SSLv3 Kx=RSA      Au=RSA  Enc=3DES(168) Mac=SHA1
PSK-3DES-EDE-CBC-SHA    SSLv3 Kx=PSK      Au=PSK  Enc=3DES(168) Mac=SHA1
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA    SSLv3 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA1
ECDHE-ECDSA-AES128-SHA  SSLv3 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA1
SRP-DSS-AES-128-CBC-SHA SSLv3 Kx=SRP      Au=DSS  Enc=AES(128)  Mac=SHA1
SRP-RSA-AES-128-CBC-SHA SSLv3 Kx=SRP      Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-DSS-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-DSS-AES128-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-DSS-AES128-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA1
DHE-RSA-SEED-SHA        SSLv3 Kx=DH       Au=RSA  Enc=SEED(128) Mac=SHA1
DHE-DSS-SEED-SHA        SSLv3 Kx=DH       Au=DSS  Enc=SEED(128) Mac=SHA1
DHE-RSA-CAMELLIA128-SHA SSLv3 Kx=DH       Au=RSA  Enc=Camellia(128) Mac=SHA1
DHE-DSS-CAMELLIA128-SHA SSLv3 Kx=DH       Au=DSS  Enc=Camellia(128) Mac=SHA1
ECDH-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AESGCM(128) Mac=AEAD
ECDH-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AESGCM(128) Mac=AEAD
ECDH-RSA-AES128-SHA256  TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AES(128)  Mac=SHA256
ECDH-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AES(128)  Mac=SHA256
ECDH-RSA-AES128-SHA     SSLv3 Kx=ECDH/RSA Au=ECDH Enc=AES(128)  Mac=SHA1
ECDH-ECDSA-AES128-SHA   SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=AES(128)  Mac=SHA1
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1
SEED-SHA                SSLv3 Kx=RSA      Au=RSA  Enc=SEED(128) Mac=SHA1
CAMELLIA128-SHA         SSLv3 Kx=RSA      Au=RSA  Enc=Camellia(128) Mac=SHA1
PSK-AES128-CBC-SHA      SSLv3 Kx=PSK      Au=PSK  Enc=AES(128)  Mac=SHA1
ECDHE-RSA-RC4-SHA       SSLv3 Kx=ECDH     Au=RSA  Enc=RC4(128)  Mac=SHA1
ECDHE-ECDSA-RC4-SHA     SSLv3 Kx=ECDH     Au=ECDSA Enc=RC4(128)  Mac=SHA1
ECDH-RSA-RC4-SHA        SSLv3 Kx=ECDH/RSA Au=ECDH Enc=RC4(128)  Mac=SHA1
ECDH-ECDSA-RC4-SHA      SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=RC4(128)  Mac=SHA1
RC4-SHA                 SSLv3 Kx=RSA      Au=RSA  Enc=RC4(128)  Mac=SHA1
RC4-MD5                 SSLv3 Kx=RSA      Au=RSA  Enc=RC4(128)  Mac=MD5 
PSK-RC4-SHA             SSLv3 Kx=PSK      Au=PSK  Enc=RC4(128)  Mac=SHA1
EDH-RSA-DES-CBC-SHA     SSLv3 Kx=DH       Au=RSA  Enc=DES(56)   Mac=SHA1
EDH-DSS-DES-CBC-SHA     SSLv3 Kx=DH       Au=DSS  Enc=DES(56)   Mac=SHA1
DES-CBC-SHA             SSLv3 Kx=RSA      Au=RSA  Enc=DES(56)   Mac=SHA1
EXP-EDH-RSA-DES-CBC-SHA SSLv3 Kx=DH(512)  Au=RSA  Enc=DES(40)   Mac=SHA1 export
EXP-EDH-DSS-DES-CBC-SHA SSLv3 Kx=DH(512)  Au=DSS  Enc=DES(40)   Mac=SHA1 export
EXP-DES-CBC-SHA         SSLv3 Kx=RSA(512) Au=RSA  Enc=DES(40)   Mac=SHA1 export
EXP-RC2-CBC-MD5         SSLv3 Kx=RSA(512) Au=RSA  Enc=RC2(40)   Mac=MD5  export
EXP-RC4-MD5             SSLv3 Kx=RSA(512) Au=RSA  Enc=RC4(40)   Mac=MD5  export

~$ curl --ciphers ALL --get 'https://stream.twitter.com/1.1/statuses/filter.json' -1 --verbose
* Hostname was NOT found in DNS cache
*   Trying 199.16.156.20...
* Connected to stream.twitter.com (199.16.156.20) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-SHA
* Server certificate:
*    subject: C=US; ST=California; L=San Francisco; O=Twitter, Inc.; OU=Twitter Security; CN=stream.twitter.com
*    start date: 2013-10-09 00:00:00 GMT
*    expire date: 2016-12-30 23:59:59 GMT
*    subjectAltName: stream.twitter.com matched
*    issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of use at https://www.verisign.com/rpa (c)10; CN=VeriSign Class 3 Secure Server CA - G3
*    SSL certificate verify ok.
> GET /1.1/statuses/filter.json HTTP/1.1
> User-Agent: curl/7.35.0
> Host: stream.twitter.com
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm="Firehose"
< Content-Type: text/html
< Cache-Control: must-revalidate,no-cache,no-store
< Transfer-Encoding: chunked
< 
<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>\n<title>Error 401 Unauthorized</title>
</head>
<body>
<h2>HTTP ERROR: 401</h2>
<p>Problem accessing '/1.1/statuses/filter.json'. Reason:
<pre>    Unauthorized</pre>
</body>
</html>
* Connection #0 to host stream.twitter.com left intact

Concerning the ticket on dev.twitter.com: I'm afraid this issue is getting beyond my understanding. Honestly I do not know how to describe the problem properly and what exactly to post there.

Best,
stephan

@dentoir
Copy link
Contributor

dentoir commented Mar 11, 2014

Hi Stephan,

It is indeed a cipher issue! That new flag for curl makes your connection work. I hope I have a fix for your application now ;-)

Can you edit the file capture/common/tmhOAuth/tmhOAuth.php
At around line 783 you see a call to curl_setopt_array() and the following lines set the curl parameters. Add the following line there.

CURLOPT_SSL_CIPHER_LIST => 'ALL',

I haven't tested this myself yet. I hope this resumes your capture, let me know if you still get something in the error log.

Cheers,

Emile

@supersambo
Copy link

Hi Emile,
thank you very much, this did actually solve the problem and tcat started collecting tweets yesterday night. Nevertheless there I was receiving the "DMI-TCAT controller killed a process" mail every 10 minutes (idletime=600). In the morning I realized that there were about 20 track.php processes running simultaneously. The error logs were about these duplicate stream attempts.

Unfortunatley I noticed afterwards that my dmi-tcat version was not up-to-date. I pulled today and and newly ran into problems (not sure if I should open a new issue for this?). There seems to be no mysql table for the query manager. The error message is

PHP Fatal error:  Uncaught exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table 'twittercapture.tcat_query_phrases' doesn't exist' in /home/supersambo/www/dmi-tcat/capture/common/functions.php:380
Stack trace:
#0 /home/supersambo/www/dmi-tcat/capture/common/functions.php(380): PDOStatement->execute()
#1 /home/supersambo/www/dmi-tcat/capture/stream/track.php(46): getActivePhrases()
#2 /home/supersambo/www/dmi-tcat/capture/stream/track.php(30): stream()
#3 {main}
  thrown in /home/supersambo/www/dmi-tcat/capture/common/functions.php on line 380

Can/should I create the mising table manually?

best,
stephan

@ErikBorra
Copy link
Member

Hi Stephan,

we have integrated a query manager in the newest source code. It assumes that there are a couple of database tables containing the query bin definitions - instead of querybins.php and followbins.php. I am in the process of writing a migration guide (which should be finished tonight) for alpha users like you. I will close the ssl issue and have opened a new issue for the query tables here. I will keep you posted.

Best,

Erik

@ErikBorra
Copy link
Member

Hi Stephan,

unfortunately I won't be able to finish the migration guide before Friday. Since you did not track any tweets last month, you might consider doing a clean install following the install guide. This should not take more than 15 minutes. Make sure to pull the latest code, as I fixed the unfortunate problem you mentioned of multiple track scripts running simultaneously.

Best,

Erik

@supersambo
Copy link

Hi Erik,
thanks for your message. That's no problem. I'm going to wait, it's not that urgent.
Also because I'm not quite clear how pulling now and editing config.php is different from a clean install and how this would solve the problem of the missing table.

Best,
stephan

@ErikBorra
Copy link
Member

Hi Stephan,

I have implemented a migration script. (Pardon the delay, I have been sick). Please pull the latest source and run the following: cd dmi-tcat/helpers; php migrate.php. It will guide you through the migration process. Let me know if you run into problems.

Best,

Erik

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants