Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Python 3 #91

Closed
tylereaves opened this issue Aug 5, 2014 · 49 comments
Labels

Comments

@tylereaves
Copy link

@tylereaves tylereaves commented Aug 5, 2014

Currently splunklib only suppports Python 2. This is quite a bit annoying for those of us that have made the jump to Python 3, especially as the spunk api is not easily handleable via standard Python XML tools.

@glennblock

This comment has been minimized.

Copy link
Contributor

@glennblock glennblock commented Oct 16, 2014

Hi @tylereaves

Thanks for reporting this. Splunk ships with the SDK in the box and currently it does not support Python 3. For this reason and due to the fact that Python 3 introduces significant / non-backward compat changes, we are holding off on Python 3 for now.

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Dec 3, 2014

Back-porting of python functionality to versions 2.x ceased with the introduction of Python 3.2 in Feb 2011, almost four years ago. All scientific/numeric packages have long since made the transition, making 2.x untenable. Please correct this major deficiency.

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Dec 3, 2014

I have begun work to port the code to python 3. Basing functionality (connecting, accessing indexes, query retrieval) work: https://github.com/martindurant/splunk-sdk-python

@glennblock

This comment has been minimized.

Copy link
Contributor

@glennblock glennblock commented Dec 3, 2014

Hi @martindurant

First, thanks for your effort here, this is awesome. Second, are you planning to have backward compat API wise, or will it introduce breaking changes that would not be compat with Python 2.7?

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Dec 4, 2014

As you see from my comments, I did not start this with a particular plan, and certainly I must have broken py2 compatability; although __future__ imports will fix some, I changed some module imports and bytes/string conversions.

@itay

This comment has been minimized.

Copy link
Contributor

@itay itay commented Dec 9, 2014

All - thank you for the comments on this ticket, and the request. Let me try and explain a bit where @glennblock and I are coming from, so that it makes sense.

As you know, Splunk ships with Python 2.7. That is unlikely to change in the near-term, given the platform support matrix we have as well as some libraries still not being available on Python 3. Given that, we need to keep working on Python 2.x.

That said, we're not against having Python 3 support added, as long as it is:

  1. Backwards compatible with Python 2.x
  2. Does not break any features/behavior in the current SDK
  3. Does not preclude us from implementing things in the future

I think (3) is not a big risk, so as long as (1) and (2) are met, I think we'd be happy to take a pull request that achieved Python 3 compatibility.

I hope this helps explain the situation a little bit.

@filmor

This comment has been minimized.

Copy link

@filmor filmor commented Nov 12, 2015

Would you be open to adding six as a dependency?

@davidbono

This comment has been minimized.

Copy link

@davidbono davidbono commented Nov 19, 2015

+1

@rhaarm

This comment has been minimized.

Copy link
Contributor

@rhaarm rhaarm commented Nov 19, 2015

I started writing a Python 2&3 compatible version with Six and future, but after making changes the Test Cases were not working and many revisions were coming in from the Splunk developers. I might revisit the Python 3 compatibility when time allots again.

@itay

This comment has been minimized.

Copy link
Contributor

@itay itay commented Nov 20, 2015

@filmor we've been talking about it internally. The main issue we have with requiring six (or any other module) is that it would break the current pattern of not having any external dependencies. This is doubly important for us because of how some people use the SDK in their apps - they simply copy the splunklib folder into their app and don't need to worry about any dependencies.

If there was a way for us to not break that (i.e. not require pip/easy_install, have splunklib be entirely self contained), we'd be open to it.

@clrogers2

This comment has been minimized.

Copy link

@clrogers2 clrogers2 commented Feb 2, 2016

From the Six documentation on https://pythonhosted.org/six/

Six provides simple utilities for wrapping over differences between Python 2 and Python 3. It is intended to support codebases that work on both Python 2 and 3 without modification. six consists of only one Python file, so it is painless to copy into a project. (emphasis mine)

Is it possible just to keep your SDK updated with the latest version of six prepackaged?

@dmikov

This comment has been minimized.

Copy link

@dmikov dmikov commented Oct 12, 2016

+1 it's time guys

@shakeelmohamed

This comment has been minimized.

Copy link
Member

@shakeelmohamed shakeelmohamed commented Oct 14, 2016

Is it possible just to keep your SDK updated with the latest version of six prepackaged?

Since six is MIT licensed, I think this is reasonable especially since the license and version number are embedded in six.py.

Submitting a PR should be much easier now that we have support for Travis CI, so you don't have to run tests locally.

@wjo1212

This comment has been minimized.

Copy link

@wjo1212 wjo1212 commented Oct 14, 2016

cannot agree more, considering pack six as a part of Public SDK (in its sub-folder like package/six.py), refer to request way

@tltx

This comment has been minimized.

Copy link
Contributor

@tltx tltx commented Oct 20, 2016

Pack six with the code and get it done. This is one of the last blockers for a large project I'am working on to move over to Python 3. It's time!

@swedishmike

This comment has been minimized.

Copy link

@swedishmike swedishmike commented Nov 8, 2016

+1 from me as well. I'm working on a specific project where I really don't want to have to drop down to 2.7 just to get the data into Splunk using the SDK.

@buysse

This comment has been minimized.

Copy link

@buysse buysse commented Dec 7, 2016

My organization has a large amount of Python 3 code we're integrating with Splunk, so we're doing it the hard way right now. Would it help if I submitted an enhancement request as a support ticket?

@shakeelmohamed

This comment has been minimized.

Copy link
Member

@shakeelmohamed shakeelmohamed commented Dec 16, 2016

@buysse please do

@raghav130593

This comment has been minimized.

Copy link

@raghav130593 raghav130593 commented Jan 26, 2017

It's high time. Please port the SDK to python 3.x

@mbrenig

This comment has been minimized.

Copy link

@mbrenig mbrenig commented Mar 7, 2017

Python 2.7 will retire in 3 years, 1 month.... when can we expect this to be ported?

@boxed

This comment has been minimized.

Copy link

@boxed boxed commented Mar 13, 2017

I've looked at using modernize to move splunklib to a six world, but I can't get all tests to run with dockerized splunk. This makes it pretty frustrating to work with.

@GhostofGoes

This comment has been minimized.

Copy link

@GhostofGoes GhostofGoes commented Mar 13, 2017

What tests are failing?

@boxed

This comment has been minimized.

Copy link

@boxed boxed commented Mar 14, 2017

Had to recreate the docker image.. One thing the docs don't seem to mention, at least on mac, is that the admin port isn't bound by default so is unavailable. This bit me last time too and took a while to figure out. Anyway, some errors:

test_upload (test_examples.ExamplesTestCase) ... Traceback (most recent call last):
  File "upload.py", line 81, in <module>
    main(sys.argv[1:])
  File "upload.py", line 78, in main
    index.upload(fullpath, **kwargs_submit)
  File "../splunklib/client.py", line 2094, in upload
    self.service.post(path, name=filename, **kwargs)
  File "../splunklib/binding.py", line 287, in wrapper
    return request_fun(self, *args, **kwargs)
  File "../splunklib/binding.py", line 69, in new_f
    val = f(*args, **kwargs)
  File "../splunklib/binding.py", line 738, in post
    response = self.http.post(path, all_headers, **query)
  File "../splunklib/binding.py", line 1201, in post
    return self.request(url, message)
  File "../splunklib/binding.py", line 1221, in request
    raise HTTPError(response)
splunklib.binding.HTTPError: HTTP 400 Bad Request -- unable to open file: path='/Users/andersh/Projects/splunk-sdk-python-clean/examples/upload.py' error='No such file or directory'
FAIL

This is weird because that file does in fact exist at that path. It it trying that path inside docker maybe?

test_countmatches_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp) ... FAIL
test_generatehello_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp) ... FAIL
test_pypygeneratetext_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp) ... FAIL
test_sum_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp) ... FAIL
...
test_analytics (test_examples.ExamplesTestCase) ... ERROR
test_async (test_examples.ExamplesTestCase) ... Traceback (most recent call last):
  File "async/async.py", line 205, in <module>
    main(sys.argv[1:])
  File "async/async.py", line 143, in main
    do_search(query)
  File "async/async.py", line 94, in do_search
    results = job.results()
  File "async/../../splunklib/client.py", line 2757, in results
    return self.get("results", **query_params).body
  File "async/../../splunklib/client.py", line 989, in get
    return super(Entity, self).get(path_segment, owner=owner, app=app, sharing=sharing, **query)
  File "async/../../splunklib/client.py", line 746, in get
    **query)
  File "async/../../splunklib/binding.py", line 287, in wrapper
    return request_fun(self, *args, **kwargs)
  File "async/../../splunklib/binding.py", line 69, in new_f
    val = f(*args, **kwargs)
  File "async/../../splunklib/binding.py", line 665, in get
    response = self.http.get(path, self._auth_headers, **query)
  File "async/../../splunklib/binding.py", line 1160, in get
    return self.request(url, { 'method': "GET", 'headers': headers })
  File "async/../../splunklib/binding.py", line 1221, in request
    raise HTTPError(response)
splunklib.binding.HTTPError: HTTP 404 Not Found -- Unknown sid.
FAIL
...
test_build_dir_exists (test_examples.ExamplesTestCase) ... FAIL
...
test_upload [(like above) ...]
...
test_default (test_index.IndexTest) ... ERROR
...
test_disable_enable (test_index.IndexTest) ... ERROR
test_integrity (test_index.IndexTest) ... ERROR
test_prefresh (test_index.IndexTest) ... ERROR
test_submit (test_index.IndexTest) ... ERROR
test_submit_namespaced (test_index.IndexTest) ... ERROR
test_submit_via_attach (test_index.IndexTest) ... ERROR
test_submit_via_attach_using_token_header (test_index.IndexTest) ... ERROR
test_submit_via_attach_with_cookie_header (test_index.IndexTest) ... ERROR
test_submit_via_attach_with_multiple_cookie_headers (test_index.IndexTest) ... ERROR
test_submit_via_attached_socket (test_index.IndexTest) ... ERROR
test_upload (test_index.IndexTest) ... Test requires sdk-app-collection. Skipping.
ERROR
...
test_lists_modular_inputs (test_input.TestInput) ... Test requires sdk-app-collection. Skipping.
...
test_inputs_list_on_one_kind_with_search (test_input.TestRead) ... FAIL
...
test_enable_preview (test_job.TestJobWithDelayedDone) ... Test requires sdk-app-collection. Skipping.
ok
test_setpriority (test_job.TestJobWithDelayedDone) ... Test requires sdk-app-collection. Skipping.
...

What is "sdk-app-collection"? No mention of it on https://github.com/splunk/splunk-sdk-python README.

test_lists_modular_inputs (test_modular_input.ModularInputKindTestCase) ... Test requires sdk-app-collection. Skipping.
test_list_arguments (test_modular_input_kinds.ModularInputKindTestCase) ... Test requires sdk-app-collection. Skipping.
test_list_modular_inputs (test_modular_input_kinds.ModularInputKindTestCase) ... Test requires sdk-app-collection. Skipping.
test_update_raises_exception (test_modular_input_kinds.ModularInputKindTestCase) ... Test requires sdk-app-collection. Skipping.

The error log at the end:

======================================================================
ERROR: test_analytics (test_examples.ExamplesTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_examples.py", line 322, in test_analytics
    index.clean()
  File "/Users/andersh/Projects/splunk-sdk-python-clean/splunklib/client.py", line 2032, in clean
    (self.name, timeout)
OperationError: Cleaning index sdk-test took longer than 60 seconds; timing out.

======================================================================
ERROR: test_default (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_disable_enable (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_integrity (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_prefresh (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_submit (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_submit_namespaced (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_submit_via_attach (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_submit_via_attach_using_token_header (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_submit_via_attach_with_cookie_header (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_submit_via_attach_with_multiple_cookie_headers (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_submit_via_attached_socket (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
ERROR: test_upload (test_index.IndexTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_index.py", line 43, in tearDown
    self.assertEventuallyTrue(lambda: self.index_name not in self.service.indexes)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/testlib.py", line 92, in assertEventuallyTrue
    self.fail(timeout_message)
AssertionError: Operation timed out.

======================================================================
FAIL: test_countmatches_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/searchcommands/test_searchcommands_app.py", line 166, in test_countmatches_as_unit
    self.assertEqual('', errors)
AssertionError: u'' != 'Traceback (most recent call last):\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 76, in emit\n    if self.shouldRollover(record):\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 154, in shouldRollover\n    self.stream = self._open()\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 945, in _open\n    stream = codecs.open(self.baseFilename, self.mode, self.encoding)\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 896, in open\n    file = __builtin__.open(filename, mode, buffering)\nIOError: [Errno 2] No such file or directory: \'/Users/andersh/Projects/splunk-sdk-python-clean/tests/var/log/splunk/searchcommands_app.log\'\nLogged from file search_command.py, line 278\n'

======================================================================
FAIL: test_generatehello_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/searchcommands/test_searchcommands_app.py", line 185, in test_generatehello_as_unit
    self.assertEqual('', errors)
AssertionError: u'' != 'Traceback (most recent call last):\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 76, in emit\n    if self.shouldRollover(record):\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 154, in shouldRollover\n    self.stream = self._open()\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 945, in _open\n    stream = codecs.open(self.baseFilename, self.mode, self.encoding)\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 896, in open\n    file = __builtin__.open(filename, mode, buffering)\nIOError: [Errno 2] No such file or directory: \'/Users/andersh/Projects/splunk-sdk-python-clean/tests/var/log/splunk/searchcommands_app.log\'\nLogged from file search_command.py, line 278\n'

======================================================================
FAIL: test_pypygeneratetext_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/searchcommands/test_searchcommands_app.py", line 205, in test_pypygeneratetext_as_unit
    self.assertEqual('', errors)
AssertionError: u'' != 'Traceback (most recent call last):\n  File "/usr/local/Cellar/pypy/5.1.0/libexec/lib-python/2.7/logging/handlers.py", line 76, in emit\n    if self.shouldRollover(record):\n  File "/usr/local/Cellar/pypy/5.1.0/libexec/lib-python/2.7/logging/handlers.py", line 154, in shouldRollover\n    self.stream = self._open()\n  File "/usr/local/Cellar/pypy/5.1.0/libexec/lib-python/2.7/logging/__init__.py", line 956, in _open\n    stream = codecs.open(self.baseFilename, self.mode, self.encoding)\n  File "/usr/local/Cellar/pypy/5.1.0/libexec/lib-python/2.7/codecs.py", line 884, in open\n    file = __builtin__.open(filename, mode, buffering)\nIOError: [Errno 2] No such file or directory: \'/Users/andersh/Projects/splunk-sdk-python-clean/tests/var/log/splunk/searchcommands_app.log\'\nLogged from file search_command.py, line 278\n'

======================================================================
FAIL: test_sum_as_unit (searchcommands.test_searchcommands_app.TestSearchCommandsApp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/searchcommands/test_searchcommands_app.py", line 229, in test_sum_as_unit
    self.assertEqual('', errors)
AssertionError: u'' != 'Traceback (most recent call last):\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 76, in emit\n    if self.shouldRollover(record):\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 154, in shouldRollover\n    self.stream = self._open()\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 945, in _open\n    stream = codecs.open(self.baseFilename, self.mode, self.encoding)\n  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 896, in open\n    file = __builtin__.open(filename, mode, buffering)\nIOError: [Errno 2] No such file or directory: \'/Users/andersh/Projects/splunk-sdk-python-clean/tests/var/log/splunk/searchcommands_app.log\'\nLogged from file search_command.py, line 278\n'

======================================================================
FAIL: test_async (test_examples.ExamplesTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_examples.py", line 73, in test_async
    self.assertEquals(result, 0)
AssertionError: 1 != 0

======================================================================
FAIL: test_build_dir_exists (test_examples.ExamplesTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_examples.py", line 85, in test_build_dir_exists
    self.assertTrue(os.path.exists("../build"), 'Run setup.py build, then setup.py dist')
AssertionError: Run setup.py build, then setup.py dist

======================================================================
FAIL: test_upload (test_examples.ExamplesTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_examples.py", line 244, in test_upload
    "upload.py --index=sdk-tests %s" % file_to_upload)
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_examples.py", line 62, in check_commands
    self.assertEquals(result, 0, '"{0}" run failed with result code {1}'.format(arg, result))
AssertionError: "upload.py --index=sdk-tests ./upload.py" run failed with result code 1

======================================================================
FAIL: test_inputs_list_on_one_kind_with_search (test_input.TestRead)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/andersh/Projects/splunk-sdk-python-clean/tests/test_input.py", line 158, in test_inputs_list_on_one_kind_with_search
    self.assertEqual(expected, found)
AssertionError: Lists differ: ['$SPLUNK_HOME/etc/splunk.vers... != ['$SPLUNK_HOME/etc/splunk.vers...

Second list contains 6 additional elements.
First extra element 6:
'/host/containers/*/[a-f0-9]+-json.log$'

  ['$SPLUNK_HOME/etc/splunk.version',
   '$SPLUNK_HOME/var/log/introspection',
   '$SPLUNK_HOME/var/log/splunk',
   '$SPLUNK_HOME/var/log/splunk/license_usage_summary.log',
   '$SPLUNK_HOME/var/spool/splunk',
-  '$SPLUNK_HOME/var/spool/splunk/...stash_new']
?                                              ^

+  '$SPLUNK_HOME/var/spool/splunk/...stash_new',
?                                              ^

+  '/host/containers/*/[a-f0-9]+-json.log$',
+  '/host/containers/*/config.v2.json$',
+  '/host/containers/*/hostconfig.json$',
+  '/host/containers/*/hostname$',
+  '/host/containers/*/hosts$',
+  '/host/containers/*/resolv.conf$']

----------------------------------------------------------------------
Ran 340 tests in 442.851s

FAILED (failures=8, errors=13)
@boxed

This comment has been minimized.

Copy link

@boxed boxed commented Mar 14, 2017

Oh, and having to manually run splunk-sdk-python-clean/examples/searchcommands_app/setup.py build and not having that mentioned in the README for how to run the tests is also annoying.

@GhostofGoes

This comment has been minimized.

Copy link

@GhostofGoes GhostofGoes commented Mar 15, 2017

Did any of this happen when you ran the tests using Python 2.7, or is that what you were trying originally?

@boxed

This comment has been minimized.

Copy link

@boxed boxed commented Mar 15, 2017

@cobaltjacket

This comment has been minimized.

Copy link

@cobaltjacket cobaltjacket commented Mar 17, 2017

How close are we? Some of us only write for Python3.x.

@stainedhat

This comment has been minimized.

Copy link

@stainedhat stainedhat commented Mar 21, 2017

How is there still not Python 3 support in 2017? I can't remember the last thing I wrote in 2.x. Is there any official word on this from the Splunk team? Was just gearing up to build some internal tooling and the thought of having to develop and maintain a 2.x Splunk app is pretty disappointing.

@jcavanagh

This comment has been minimized.

Copy link

@jcavanagh jcavanagh commented May 23, 2017

Adding my +1.

The rest of the world seems able to move forward to Python 3, and I'm sure a lot of them had big, internal Python 2.x libraries as well.

@boxed

This comment has been minimized.

Copy link

@boxed boxed commented Jun 27, 2017

So... is anything happening? Some feedback on how to even run the test suite under python 2 would be appreciated.

@llamahunter

This comment has been minimized.

Copy link

@llamahunter llamahunter commented Jun 28, 2017

I would like to echo the disbelief that splunk does not have python3 support when python 2 is nearing the end of its maintenance phase, active development on it long since ended.

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Jun 28, 2017

After posting a partial port on Dec 3, 2014 (see above), allow me to make a pitch for dask, a pure-python parallel data processing and analytic computation engine. Where splunk is failing some people, dask may be the solution; although the focus is really quite different, dask will work excellently for ETL problems and aggregations that people wish to express in python, and real-time, batch processing and queues are rapidly advancing. You would use that computation engine in combination with ordinary python ecosystem tools for visualisation (e.g., jupyter with bokeh).
Just a thought!

@tltx

This comment has been minimized.

Copy link
Contributor

@tltx tltx commented Jul 2, 2017

I made an attempt to add Py3 support on a fork https://github.com/tltx/splunk-sdk-python/. There are still unicode handling problems, some tests fail and I have not tested it besides running the unit tests. I'm going to put it aside for now but if someone want to continue the work or test it, go ahead.

@tltx

This comment has been minimized.

Copy link
Contributor

@tltx tltx commented Jul 3, 2017

I fixed most of the issues I found so I think it's ready to be tested now.

@MZamanMughal

This comment has been minimized.

Copy link

@MZamanMughal MZamanMughal commented Jul 19, 2017

Please upgrade quickly. Python 3.x is the future.

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Jul 19, 2017

@MZamanMughal : python 3 is perhaps the present, certainly not the future - by the time this is done, we'll probably have python 4!

@cobaltjacket

This comment has been minimized.

Copy link

@cobaltjacket cobaltjacket commented Aug 5, 2017

We're at three months since the last official response. Please advise. Anyone who has commented on this or the similar issues is a paying customer who would like to have this resolved.

@cudneys

This comment has been minimized.

Copy link

@cudneys cudneys commented Sep 7, 2017

I'm a paying customer and would really like to have this resolved ASAP. It's ridiculous that this has been ignored for several years. The Python community has made is quite easy to bridge the gap between 2.x and 3.x.

Elasticsearch offers excellent Python support for far less money.

@boxed

This comment has been minimized.

Copy link

@boxed boxed commented Sep 7, 2017

@cudneys If you can try @tltx 's branch it would be nice. We are also a paying customer but we just can't wait for this as we're going to py3 very soon. His branch is what we'll use in production when we switch over and it handles our cases at least.

We really should deduct this development effort from their next invoice :P

@stainedhat

This comment has been minimized.

Copy link

@stainedhat stainedhat commented Sep 7, 2017

We're actually gearing up to drastically reduce our Splunk use in production and moving the majority of our logging to ELK. A good portion of the reason is that we can't develop against it without significant effort or using unsupported SDK's. It's really a shame because it's a good product. Oh well, c'est la vie.

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Sep 7, 2017

To the people complaining on this issue,
at Anaconda, inc., we may be interested in supporting bulk data access to Splunk, including parallelised/clustered access. We would do this probably by sidestepping the client here, and accessing the REST interface directly. The client is rather comprehensive... if the aim is access to data, we need not implement more than auth, post a query, poll a query and fetch results - to what extent does that cover the everyone's use cases?

@cobaltjacket

This comment has been minimized.

Copy link

@cobaltjacket cobaltjacket commented Sep 8, 2017

I'm just going to leave this here:

Python 2 Countdown Clock

@boxed

This comment has been minimized.

Copy link

@boxed boxed commented Sep 8, 2017

@martindurant I think we could probably just fix this ourselves. I'm sure @tltx will gladly add commit rights to his fork if you've found bugs. Just try that branch and fix the bugs you find!

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Sep 8, 2017

@boxed , you misunderstand me. I already wrote a py3 port that satisfied my needs at the time. I am asking whether there is community interest in parallel and distributed data access to Splunk via python.

@dstromberg

This comment has been minimized.

Copy link

@dstromberg dstromberg commented Jan 9, 2018

3.x is a must.
I recommend that people simply not use code that is still 2.x-only.

@martindurant

This comment has been minimized.

Copy link

@martindurant martindurant commented Jan 9, 2018

I am interested, does anyone have a use for https://github.com/martindurant/splunk_connector - parallel downloading of splunk results with dask.

@splunk splunk locked as off topic and limited conversation to collaborators Jan 9, 2018
@shakeelmohamed

This comment has been minimized.

Copy link
Member

@shakeelmohamed shakeelmohamed commented Feb 26, 2018

Python 3 support will be in the next release, thank you all for your patience. Feel free to grab the develop branch if you need it sooner.

@shakeelmohamed

This comment has been minimized.

Copy link
Member

@shakeelmohamed shakeelmohamed commented Feb 26, 2018

Enormous thank you to @tltx for doing all the heavy lifting!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
You can’t perform that action at this time.