fix for Splunk Support Case: 204977

[whisperstream]

This is a fix for the support Case 204977 which I opened a while back: Without this fix, whenever I tried to execute a oneshot query, I would get back the following exception (this pull request fixes this issue):

File "/usr/local/lib/python2.7/dist-packages/splunklib/client.py", line 305, in connect
return Service(**kwargs).login()
File "/usr/local/lib/python2.7/dist-packages/splunklib/binding.py", line 764, in login
session = XML(body).findtext("./sessionKey")
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
return parser.close()
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err

[itay]

@whisperstream can you post a small script that shows your problem without your fix? We have many tests and usage of oneshot queries which work fine, so I want to verify.

[whisperstream]

import splunklib.client as SplunkSDKClient
print str(SplunkSDKClient.connect({
'host': ‘_’,
'port': 443,
'scheme': 'https',
'username': ‘_’,
'password': ‘******’,
}).jobs.oneshot('search * | head 10', *{'count': 0}))

------------ results without fix ----------------------
ubuntu@spark:$ python test.py
Traceback (most recent call last):
File "test.py", line 7, in
'password': '*********',
File "/usr/local/lib/python2.7/dist-packages/splunklib/client.py", line 305, in connect
return Service(*kwargs).login()
File "/usr/local/lib/python2.7/dist-packages/splunklib/binding.py", line 764, in login
session = XML(body).findtext("./sessionKey")
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
return parser.close()
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: no element found: line 1, column 0
ubuntu@spark:$

------------- results with fix ---------------------
ubuntu@spark:~$ python test.py

_bkt _cd _indextime _raw _serial _si _sourcetype _time host index linecount source sourcetype splunk_server main~94~3956FB27-9471-427A-A1AA-06F876745CE1 etc....

[itay]

Just tried this locally with the SDK, no issues:

import splunklib.client as client

print str(client.connect(**{
    'host': 'localhost',
    'port': 8089,
    'scheme': 'https',
    'username': 'admin',
    'password': 'changeme',
}).jobs.oneshot('search * | head 10', **{'count': 0}))

Output is:

$ PYTHONPATH=~/work/splunk-sdk-python:$PYTHONPATH python test2.py      
<?xml version='1.0' encoding='UTF-8'?>

<results preview='0'/>

The issue I have with the fix is that the connection.close() doesn't actually do it seems like. If you look in the Python code, that socket is actually reference counted, and will remain in effect until "response" object holding is closed. If this weren't true, then no request whatsoever would ever work with the SDK, which is clearly the not the case at the moment.

It's very odd that you're seeing the issue. Can you share a bit about your environment (OS, Python version, etc)?

[whisperstream]

Sure, I installed the sdk using:
pip install splunk-sdk

On both Mac OSX (10.9.5) using python 2.7.5 and Ubuntu (14.04) using
Python 2.7.6 and get the same issue on both.

-Axel

On Fri, Jul 10, 2015 at 5:05 PM, Itay Neeman notifications@github.com
wrote:

Just tried this locally with the SDK, no issues:

import splunklib.client as client

print str(client.connect(*{
'host': 'localhost',
'port': 8089,
'scheme': 'https',
'username': 'admin',
'password': 'changeme',
}).jobs.oneshot('search * | head 10', *{'count': 0}))

Output is:

$ PYTHONPATH=~/work/splunk-sdk-python:$PYTHONPATH python test2.py

The issue I have with the fix is that the connection.close() doesn't
actually do it seems like. If you look in the Python code, that socket is
actually reference counted, and will remain in effect until "response"
object holding is closed. If this weren't true, then no request whatsoever
would ever work with the SDK, which is clearly the not the case at the
moment.

It's very odd that you're seeing the issue. Can you share a bit about your
environment (OS, Python version, etc)?

—
Reply to this email directly or view it on GitHub
#111 (comment)
.

[itay]

@whisperstream we've tried now on multiple machines, including clean VMs, and a couple of Python versions, and simply cannot reproduce this. It's also a section of code that hasn't changed since we launched the SDKs.

I think we're likely going to have to decline the PR, because the solution of not calling connection.close will cause us to leak the connection. That said, we do realize you're hitting this issue, so we want to try and help you there.

I just want to doublecheck: when you set it to port 443 and some host, this is where the splunk management port is running (normally 8089), yes? i.e. not the UI ports.

[whisperstream]

yeah normally 8089, but ops team is running it on 443. Very strange that
you're not seeing it on your end, but understand the data leakage issue (a
concern of mine too....but pragmatism won out). I''ll just continue to
patch until I figure it out.....but thanks for your consideration. Will
post back here if I figure out the root cause.

-Axel

On Wed, Jul 15, 2015 at 2:27 PM, Itay Neeman notifications@github.com
wrote:

@whisperstream https://github.com/whisperstream we've tried now on
multiple machines, including clean VMs, and a couple of Python versions,
and simply cannot reproduce this. It's also a section of code that hasn't
changed since we launched the SDKs.

I think we're likely going to have to decline the PR, because the solution
of not calling connection.close will cause us to leak the connection.
That said, we do realize you're hitting this issue, so we want to try and
help you there.

I just want to doublecheck: when you set it to port 443 and some host,
this is where the splunk management port is running (normally 8089), yes?
i.e. not the UI ports.

—
Reply to this email directly or view it on GitHub
#111 (comment)
.