MISP42 Getting Proxy Error with Splunk 9.0

[brwnskndgirl]

I can't seem to figure out the following error with MISP42 once we upgraded to Splunk 9.0.0. It appears the Python library changed also.

07-27-2022 22:15:26.024 ERROR script [31825 phase_1] - SearchMessage orig_component=script sid=1658960004.369_AEB75738-63D0-4F91-879F-AD62D0A7A6EC message_key=EXTERN:SCRIPT_NONZERO_RETURN__%s_%d_%s message=External search command 'mispgetioc' returned error code 1. Script output = "error_message=ProxyError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 512 : HTTPSConnectionPool(host='xxxxx.me.com', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))\r\n\r\n".

[timo92700]

Hello,
After upgrading to splunk 9.0, I have the same kind of error :

07-28-2022 12:35:28.821 ERROR script [1564718 phase_1] - SearchMessage orig_component=script sid=1659004527.1855_96CB9E0B-21F7-47EB-8C09-391F40BE0E16 message_key=EXTERN:SCRIPT_NONZERO_RETURN_%s_%d_%s message=External search command 'mispgetioc' returned error code 1. Script output = "error_message=SSLError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 514 : HTTPSConnectionPool(host='xxxxxx', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by SSLError(SSLError(1, '[SSL: UNKNOWN_PROTOCOL] unknown protocol (ssl.c:1106)')))\r\n\r\n".

Notice that when running the query on the same search head using the curl command in the terminal, as the below command, it works :

curl -k -X POST "https://xxxxxxxx/attributes/restSearch" -H "Content-type:application/json" -H "Authorization:<TOKEN_HERE>" -H "Accept:application/json" -x http://xxxxx:8080 --connect-timeout 300

[firm-0ne]

Same as above having issues with MISP42 command mispgetioc after upgrading to Splunk v9.0 (was v8.2.4).

07-29-2022 15:13:31.050 INFO PhaseNodeGenerationVisitor [15036 searchOrchestrator] - FallBackReason: Fallback to 2-phase mode because of empty split key of cmd: mispgetioc
.
.
.
07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [IO-101] logging level is set to DEBUG
07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [IO-102] PYTHON VERSION: 3.7.11 (default, May 25 2022, 12:23:55)
07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [GCC 9.1.0]
07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': GET request to https://127.0.0.1:8089/servicesNS/nobody/misp42splunk/misp42splunk_instances (body: {})
07-29-2022 15:13:31.967 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Operation took 0:00:00.618513
07-29-2022 15:13:31.967 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D01] response.status=200
07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D02] instance_count=2
07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D04] instance item={'title': 'CSO_misp_test', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'updated': '1969-12-31T16:00:00-08:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'remove'}], 'author': {'name': 'REDACTED@acme.com'}, 'content': {'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'REDACTED@acme.com', 'perms': {'read': [''], 'write': ['admin']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '', 'misp_url': 'https://OUR.SERVER.REDACTED.acme.com', 'misp_use_proxy': '1', 'misp_verifycert': '0', 'type': 'text/xml'}}
07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D04] instance item={'title': 'CSO_misp_test2', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'updated': '1969-12-31T16:00:00-08:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'remove'}], 'author': {'name': 'REDACTED@acme.com'}, 'content': {'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'REDACTED@acme.com', 'perms': {'read': [''], 'write': ['admin']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '**', 'misp_url': 'https://OUR.SERVER.REDACTED.acme.com', 'misp_use_proxy': '1', 'misp_verifycert': '0', 'type': 'text/xml'}}
07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': GET request to https://127.0.0.1:8089/servicesNS/nobody/misp42splunk/storage/passwords/ (body: {'count': -1, 'offset': 0})
07-29-2022 15:13:31.976 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Operation took 0:00:00.008125
07-29-2022 15:13:31.978 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Option "last" set with 2d
07-29-2022 15:13:31.980 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Starting new HTTPS connection (1):OUR.SERVER.REDACTED.acme.com:443
07-29-2022 15:13:32.213 INFO ReducePhaseExecutor [15050 StatusEnforcerThread] - ReducePhaseExecutor=1 action=PREVIEW
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': ProxyError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 510 : HTTPSConnectionPool(host='OUR.SERVER.REDACTED.acme.com', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Traceback:
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/lib/splunklib/searchcommands/search_command.py", line 619, in process_protocol_v1
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': self.execute(ifile, None)
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/lib/splunklib/searchcommands/generating_command.py", line 211, in execute
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': self.record_writer.write_records(self.generate())
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/lib/splunklib/searchcommands/internals.py", line 576, in write_records
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': records = list(records)
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py", line 450, in generate
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': proxies=my_args['proxies'])
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/api.py", line 119, in post
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': return request('post', url, data=data, json=json, **kwargs)
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/api.py", line 61, in request
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': return session.request(method=method, url=url, **kwargs)
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': resp = self.send(prep, **send_kwargs)
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/sessions.py", line 655, in send
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': r = adapter.send(request, **kwargs)
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 510, in send
07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': raise ProxyError(e, request=request)
07-29-2022 15:15:34.887 ERROR script [15057 phase_1] - SearchMessage orig_component=script sid=1659107610.2342_3AEC3D0E-E3C5-47A2-945F-AD3A6E99B633 message_key=EXTERN:SCRIPT_NONZERO_RETURN%s%d%s message=External search command 'mispgetioc' returned error code 1. Script output = "error_message=ProxyError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 510 : HTTPSConnectionPool(host='OUR.SERVER.REDACTED.acme.com', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))\r\n\r\n".

[brwnskndgirl]

I fixed this issue by adding the following line to mispgetioc.py. The error coming from adapters.py will confuse you. When editing the adapters.py I found the error to actually be coming from the mispgetioc.py script. This is the second python script where I had to hard-code the proxy for Splunk. It appears with the latest python urlib3 code disallows proxy defined in MISP Splunk not being acknowledged and passed downstream.

proxies = {http: 'proxy.com:xxxx', https: 'proxy.com:xxxx'}

[remg427]

I cannot fix that issue. @brwnskndgirl I tried your patch but then there is an error message and the request fails.
researching on Internet links this bug to a version of lib request

[timo92700]

Hello,
Well this does not appears to come from python request module.
I developed this application that permit to create curl request using the search bar : https://splunkbase.splunk.com/app/5667/

Once installed ( on splunk9 ), by providing the correct parameters, for example, tried with this :

| curl url="https://<YOUR_MISP_URL>/events/1" method=post headers="{'Authorization':'<YOUR_TOKEN>','Content-type': 'application/json','Accept':'application/json'}" proxies="http://,<YOUR_HTTPS_PROXY" timeout=10 output=json
| spath input=Event

It works, this is only a workaround, but at least it is usable.
I did not investigated the code of the misp application, but as it is using the requests module ( maybe the code uses requests module part that became deprecated / deleted after the splunk9/python upgrade ? ), it should work in some way.

Feel free to modify the endpoint / parameters of the request to match your needs, then add a | collect and schedule the search, until the official app is upgraded.
Regards,

[remg427]

Hi,
it is still related as switch to urllib3 helped to fix the requests using proxy and self-signed certificates.
version 4.2.0 works on my side.

[timo92700]

Hi,
Just tested 4.2.0, also works on my side now.
Well done !

[remg427]

Hi Thank you so much for feedback I will publish soon on splunkbase Remi Le 3 octobre 2022 11:40:43 GMT+02:00, timo92700 ***@***.***> a écrit :

…

Hi, Just tested 4.2.0, also works on my side now. Well done ! -- Reply to this email directly or view it on GitHub: #216 (comment) You are receiving this because you were assigned. Message ID: ***@***.***>

-- Sent with K-9 Mail.