Run test on remote Server: enableCrypto operation on CakeSocket

[jaegeral]

Trying to set up connection to a new server.

I have two MISP instances that verified the issue.

MISP1 is productive and already connected to a remote MISP (so connection with proxy works)
MISP2 is a VM (the test VM) with updated OS and latest version Currently installed version..... v2.4.47

RMISP (the remote MISP) is Currently installed version..... v2.4.26

Added the RMISP as remote connection, added the auth key, uploaded the pem file.

Now running a test it gives:

Error: Connection test failed. Reason: "Unable to perform enableCrypto operation on CakeSocket"      

Doing a curl
curl -k https://RMISP/events/xml/download.json
-->

{"name":"Authentication failed. Please make sure you pass the API key of an API enabled user along in the Authorization header.","message":"Authentication failed. Please make sure you pass the API key of an API enabled user along in the Authorization header.","url":"\/events\/xml\/download.json"}

I saw #740
But I updated MISP according to the update guide already, so I assume my cakephp is up2date as well.

I see that traffic is going to my proxy, but no data is on the receiver site. However if I browse the MISP instance, it works and curl also pops up in RMISP logs.

Don't use Proxy authentication.

Any ideas how to fix that?

Work environment

Questions	Answers
Type of issue	Bug
OS version (server)	ubuntu
OS version (client)
PHP version	5.4, 5.5, 5.6, 7.0, 7.1...
MISP version / git hash	2.4.47, hash of the commit
Browser	If applicable

Expected behavior

Actual behavior

"
Error: Connection test failed. Reason: "Unable to perform enableCrypto operation on CakeSocket"
"

Steps to reproduce the behavior

Logs, screenshots, configuration dump, ...

[RichieB2B]

When troubleshooting don't assume anything. ;-) Check your CakePHP version with:

git submodule status

When this is ok (running 2.8.x) check what MISP is actually sending to your proxy using tcpdump and/or wireshark.

[Rafiot]

Assuming it's fixed.

[jaegeral]

Nope, still an issue:

root@misp:/var/www/MISP/app# git submodule status e7b22b70e0389c39b3ae1e863121d3859a061b62 ../PyMISP (v2.4.53-22-ge7b22b7) 73dcbf2aa33b79512373baa63b39adf3fb838608 Lib/cakephp (2.8.9) 088c04e2f261c33bed6ca5245491cfca69195ccf Lib/random_compat (v2.0.2) 0ce745a12fa46c36ee8e74c397e84e3659161e04 files/taxonomies (remotes/origin/travis-135-g0ce745a) 6850a349c0985c7212e4e9d38c77877181be0e1b files/warninglists (remotes/origin/HEAD)

Error: Connection test failed. Reason: "Unable to perform enableCrypto operation on CakeSocket"

Enabled debug I found the stacktrace:

Unable to perform enableCrypto operation on CakeSocket
Error: An Internal Error Has Occurred.
Stack Trace
APP/Lib/cakephp/lib/Cake/Network/CakeSocket.php line 208 → CakeSocket->enableCrypto(string, string)
APP/Lib/cakephp/lib/Cake/Network/CakeSocket.php line 347 → CakeSocket->connect()
APP/Lib/cakephp/lib/Cake/Network/Http/HttpSocket.php line 377 → CakeSocket->write(string)
APP/Lib/cakephp/lib/Cake/Network/Http/HttpSocket.php line 466 → HttpSocket->request(array)
APP/Model/Server.php line 3027 → HttpSocket->get(string, string, array)
APP/Controller/ServersController.php line 83 → Server->previewIndex(string, array, array)
[internal function] → ServersController->previewIndex(string)
APP/Lib/cakephp/lib/Cake/Controller/Controller.php line 491 → ReflectionMethod->invokeArgs(ServersController, array)
APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php line 193 → Controller->invokeAction(CakeRequest)
APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php line 167 → Dispatcher->_invoke(ServersController, CakeRequest)
APP/webroot/index.php line 92 → Dispatcher->dispatch(CakeRequest, CakeResponse)

The system has no issue to pull data from a well known misp instance.

[jaegeral]

@Rafiot can you re-open that issue please?

[jaegeral]

thx

@RichieB2B as you asked:

/var/www/MISP# git submodule status
 8cec9377c1c4e2ea756282e9bfee7701200c94b6 PyMISP (v2.4.54-29-g8cec937)
 73dcbf2aa33b79512373baa63b39adf3fb838608 app/Lib/cakephp (2.8.9)
 088c04e2f261c33bed6ca5245491cfca69195ccf app/Lib/random_compat (v2.0.2)
 ff17ac998e213ad205436c0c230959375bb70b09 app/files/misp-galaxy (heads/master)
 1d957da2245203f9fa9f4d01b71d6f62c11b4442 app/files/taxonomies (remotes/origin/travis-148-g1d957da)
 338241e4999fa618909241e7e413c9fab5debca5 app/files/warninglists (heads/master-26-g338241e)

Is that maybe an issue with ssl certs that are issued by an internal CA?

The MISP Version both servers running:

MISP 2.4.59

[RichieB2B]

Those version numbers look ok. The error message you are getting suggests CakePHP fails to set up the ssl tunnel correctly. The CakePHP error handling is not very good and the error message not descriptive enough to determine the cause. Check with tcpdump/wireshark what MISP is actually sending to your proxy and how the proxy responds.

[jaegeral]

fwiw I am still trying to find the root cause.

[RichieB2B]

Did you check with tcpdump what MISP is actually sending to your proxy?

[jaegeral]

Ok found the reason.

Checked the TCPDUMP and I get Unknown CA. The CA is however added to the trusted CAs of the OS (and verified with wget).

So the solution was to add the CRT of the CA as pem to Server certificate and it worked.

(Btw I discovered another issue as it would be handy to get feedback from the UI if the server certificate is not with the right file extension: *.pem) it will simply do nothing, this is somehow not really intuitive but I will create a new issue for that.

Anyway, I would have expected that if I set "self signed" that the complete check is disabled and it would trust whatever cert is provided from System B.

[jaegeral]

another thing I observerd that it looks like the server SSL cert does not work, it is always the signing CA that needs to be imported (tested with two instances)

That is a little odd, but not in our hands I guess but maybe we can increase the visibility for people that are trying to add a server to tell WHY a connection failed e.g. the CakePHP error message

[iglocska]

Basically for servers with a full chain - signing CA, (intermediary,) server cert you need to create a merged pem file and it should work fine. I know of some communities that use their own full signing chain that is signed by their own root CA that is not a "known" root CA by the mozilla CA store. For them it works fine with the full chain exported as a single merged pem.

[RichieB2B]

Adding the full chain (including intermediate CA's) is not actually needed. You need to add the top CA, the server needs to provide the intermediate CA's. Adding the intermediate CA's in the file will not fix the issue of the server only sending the server certificate and not the intermediate CA's.

[jaegeral]

So we just had a chat session on gitter, and it if you have a root ca, a sub ca and the server cert.
YOu need to provide all three in a pem file to MISP.

If it is missing the root ca (CakePHP is ignoring the OS CA) it will fail to use the cert.

[sim0nx]

I am recently seeing exactly the same issue with one sync partner.

First some details about the install:

• Debian 9
• MISP 2.4.73
• submodules:
  619cb104039836bf5a18daa9cdc600264b72d943 PyMISP (v2.4.71-36-g619cb10)
  8d0e1fadf77dcde9d48262f8501b7c9c2b6f8aac app/Lib/cakephp (2.9.6-6-g8d0e1fadf7)
  088c04e2f261c33bed6ca5245491cfca69195ccf app/Lib/random_compat (v2.0.2)
  def85a5dbbc5083d569106e919ede697cda1f073 app/files/misp-galaxy (remotes/origin/HEAD)
  0122eff56ba6b500dbba3fd1985c5e8665a7f9e9 app/files/taxonomies (remotes/origin/HEAD)
  bc05ddcdd030188279aa709cec4cc6702591071e app/files/warninglists (remotes/origin/HEAD)

In my case curl/browsing also works for the remote partner, only MISP does not work.
In a network trace I can see that MISP only tries TLS1.0. Manually forcing curl to do anything else than TLS1.2 also makes it fail.
So my guess is that at least in my case this might be related to #2080

2017-05-10 17:17:57 Error: [SocketException] Unable to perform enableCrypto operation on CakeSocket Request URL: /servers/previewIndex/3 Stack Trace: #0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Network/CakeSocket.php(208): CakeSocket->enableCrypto('tls', 'client') #1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Network/CakeSocket.php(347): CakeSocket->connect() #2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Network/Http/HttpSocket.php(377): CakeSocket->write('GET /events/ind...')

[sim0nx]

My issue was indeed related to #2080 which is fixed by 468834b