Master branch doesn't work with Elasticsearch 1.0.1 #56

akkie · 2014-03-03T10:47:29Z

Hi,

I can't run the latest master branch with Elasticsearch version 1.0.1. I've installed the latest version by cloning the master branch and then run the command setup.py install. This was suggested in a comment of this blog post.

This is the output:

python setup.py install
running install
running bdist_egg
running egg_info
writing requirements to elasticsearch_curator.egg-info/requires.txt
writing elasticsearch_curator.egg-info/PKG-INFO
writing top-level names to elasticsearch_curator.egg-info/top_level.txt
writing dependency_links to elasticsearch_curator.egg-info/dependency_links.txt
writing entry points to elasticsearch_curator.egg-info/entry_points.txt
reading manifest file 'elasticsearch_curator.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
writing manifest file 'elasticsearch_curator.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/curator
copying build/lib/curator/curator.py -> build/bdist.linux-x86_64/egg/curator
copying build/lib/curator/__init__.py -> build/bdist.linux-x86_64/egg/curator
byte-compiling build/bdist.linux-x86_64/egg/curator/curator.py to curator.pyc
byte-compiling build/bdist.linux-x86_64/egg/curator/__init__.py to __init__.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying elasticsearch_curator.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying elasticsearch_curator.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying elasticsearch_curator.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying elasticsearch_curator.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying elasticsearch_curator.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying elasticsearch_curator.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/elasticsearch_curator-1.0.0_dev-py2.6.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing elasticsearch_curator-1.0.0_dev-py2.6.egg
creating /usr/lib/python2.6/site-packages/elasticsearch_curator-1.0.0_dev-py2.6.egg
Extracting elasticsearch_curator-1.0.0_dev-py2.6.egg to /usr/lib/python2.6/site-packages
Adding elasticsearch-curator 1.0.0-dev to easy-install.pth file
Installing curator script to /usr/bin

Installed /usr/lib/python2.6/site-packages/elasticsearch_curator-1.0.0_dev-py2.6.egg
Processing dependencies for elasticsearch-curator==1.0.0-dev
Searching for elasticsearch>=1.0.0,<2.0.0
Reading http://pypi.python.org/simple/elasticsearch/
Best match: elasticsearch 1.0.0
Downloading https://pypi.python.org/packages/source/e/elasticsearch/elasticsearch-1.0.0.tar.gz#md5=ac087d3f7a704b2c45079e7e25b56b9f
Processing elasticsearch-1.0.0.tar.gz
Running elasticsearch-1.0.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-T98TOX/elasticsearch-1.0.0/egg-dist-tmp-1CKF3Z
error: /tmp/easy_install-T98TOX/elasticsearch-1.0.0/README.rst: No such file or directory

As you can see, the install quits with an error.

If I try to install the script with pip install . then it seems to work.

Unpacking /root/curator
  Running setup.py (path:/tmp/pip-Zn9sKP-build/setup.py) egg_info for package from file:///root/curator
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
Downloading/unpacking elasticsearch>=1.0.0,<2.0.0 (from elasticsearch-curator==1.0.0-dev)
  Downloading elasticsearch-1.0.0-py2.py3-none-any.whl (47kB): 47kB downloaded
Downloading/unpacking urllib3>=1.5,<2.0 (from elasticsearch>=1.0.0,<2.0.0->elasticsearch-curator==1.0.0-dev)
  Downloading urllib3-1.7.1.tar.gz (67kB): 67kB downloaded
  Running setup.py (path:/tmp/pip_build_root/urllib3/setup.py) egg_info for package urllib3
Installing collected packages: elasticsearch, elasticsearch-curator, urllib3
  Running setup.py install for elasticsearch-curator
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
    Installing curator script to /usr/bin
  Running setup.py install for urllib3
Successfully installed elasticsearch elasticsearch-curator urllib3
Cleaning up...

But now it shows the following error if I execute the curator command.

Traceback (most recent call last):
  File "/usr/bin/curator", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0

Last but not least the installed elasticsearch version:

curl -XGET 'localhost:9200' 
{
  "status" : 200,
  "name" : "Lucky Luke",
  "version" : {
    "number" : "1.0.1",
    "build_hash" : "5c03844e1978e5cc924dab2a423dc63ce881c42b",
    "build_timestamp" : "2014-02-25T15:52:53Z",
    "build_snapshot" : false,
    "lucene_version" : "4.6"
  },
  "tagline" : "You Know, for Search"
}

The text was updated successfully, but these errors were encountered:

honzakral · 2014-03-03T12:25:03Z

Hi,
thanks for the report. This is a weird error with python packaging schema, nothing actually to do with the version of elasticsearch. Curator is failing here to install it's dependency - elasticsearch-py in version >=1.0.0. The script curator then checks whether this is installed so it fails. I have no idea why this happens, I just tried it exactly as you described and it works for me.

Could you please provide more information about your environment: python version, are you using virtualenv?, what OS you are using?

Thanks

akkie · 2014-03-03T14:55:17Z

Hi,

thanks for you answer.

The Python version is 2.6.9 and it runs on Amazon Linux 3.4.82-69.112.amzn1.x86_64 which is based on RHEL. The package virtualenv isn't installed.

If I run pip packages then it lists me the installed package elasticsearch (1.0.0). Is this the package you mentioned? This package gets installed when I run pip install . in the curator directory.

Thanks

honzakral · 2014-03-03T15:09:35Z

hmm, even on 2.6 everything still works for me. Time to get hold of some amazon linux.

As a work-around you should be able to just call the curator.py file in the curator directory in the curator repo directly, that will bypass the version checking and should work for you.

untergeek · 2014-03-03T15:10:43Z

I think this is the second installation issue that has been RHEL-specific. 😦

akkie · 2014-03-03T15:30:17Z

OK, thanks the workaround works.

msolujic · 2014-03-04T12:07:58Z

Exactly same environment here as @akkie reported, just ES is bit older: 1.0.0

I did installation directly from tarball
sudo pip install --upgrade https://github.com/elasticsearch/curator/tarball/master

same errors.
@honzakral Thanks for workaround

kevinava · 2014-03-13T12:46:13Z

Hi,
I have the same issue on Centos 6.5 with Python 2.6.6 and Elasticsearch 1.0.1 installed.
Previous version 0.6.x+ worked fine with ES 0.90.x.

When I install latest curator from master branch it wants to install an old 0.4.4 elasticsearch lib even if its requesting version >=1.0.0. and this gives me a host of errors when running curator, even if I use workaround.

elasticsearch>=1.0.0,<2.0.0 (from elasticsearch-curator==1.0.0-dev), but installing version 0.4.4

If I run with @honzakral workaround I get the following error:

./curator.py --host localhost -c 31 -d 90

2014-03-13T13:42:32.826 INFO                  index_loop:309  DELETE index operations completed.
2014-03-13T13:42:32.826 INFO                        main:364  Closing indices older than 31 days...
2014-03-13T13:42:32.828 INFO         log_request_success:49   GET http://localhost:9200/logstash-*/_settings?expand_wildcards=closed [status:200 request:0.002s]
2014-03-13T13:42:32.836 INFO                  index_loop:298  Attempting to close index logstash-2014.02.04 because it is 6 days, 0:00:00 older than cutoff.
Traceback (most recent call last):
  File "./curator.py", line 383, in <module>
    main()
  File "./curator.py", line 366, in main
    index_loop(client, 'close', expired_indices, arguments.dry_run)
  File "./curator.py", line 302, in index_loop
    skipped = op(client, index_name, **kwargs)
  File "./curator.py", line 250, in _close_index
    if index_closed(client, index_name):
  File "./curator.py", line 245, in index_closed
    metric='metadata',
  File "/usr/lib/python2.6/site-packages/elasticsearch/client/utils.py", line 70, in _wrapped
    return func(*args, params=params, **kwargs)
TypeError: state() got an unexpected keyword argument 'index'

I've tried git clone install, pip install from http etc, it all results in the same.

Example full output from installation:

 pip install . --upgrade
Unpacking /root/curatorupg/curator
  Running setup.py egg_info for package from file:///root/curatorupg/curator
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
Downloading/unpacking elasticsearch>=1.0.0,<2.0.0 (from elasticsearch-curator==1.0.0-dev)
  Running setup.py egg_info for package elasticsearch
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'test_elasticsearch'
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
  Requested elasticsearch>=1.0.0,<2.0.0 (from elasticsearch-curator==1.0.0-dev), but installing version 0.4.4
Downloading/unpacking urllib3>=1.5,<2.0 from https://pypi.python.org/packages/source/u/urllib3/urllib3-1.8.tar.gz#md5=d664781c08f98a83f28df087b6edfdca (from elasticsearch>=1.0.0,<2.0.0->elasticsearch-curator==1.0.0-dev)
  Running setup.py egg_info for package urllib3
Installing collected packages: elasticsearch, elasticsearch-curator, urllib3
  Running setup.py install for elasticsearch
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'test_elasticsearch'
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
  Found existing installation: elasticsearch-curator 1.0.0-dev
    Uninstalling elasticsearch-curator:
      Successfully uninstalled elasticsearch-curator
  Running setup.py install for elasticsearch-curator
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
    Installing curator script to /usr/bin
  Found existing installation: urllib3 1.7.1
    Uninstalling urllib3:
      Successfully uninstalled urllib3
  Running setup.py install for urllib3
Successfully installed elasticsearch elasticsearch-curator urllib3
Cleaning up...

honzakral · 2014-03-13T14:29:41Z

So I installed CentOS, when I try to install curator to virtualenv everything works fine: pip install git+https://github.com/elasticsearch/curator or pip install https://github.com/elasticsearch/curator/tarball/master both work.

If I install it into the system (using sudo, no virtualenv) everything gets installed correctly but I get the message saying elasticsearch isn't installed when running curator, the workaround helps (just use curator.py from the package).

If you see error with incorrect version of elasticsearch you need to upgrade it manually using pip install --upgrade elasticsearch==1.0.0 (@kevinava).

I am still investigating why does RHEL hate python so much and I will keep updating this ticket with new information/fixes.

Thanks for the reports and for your patience.

untergeek · 2014-03-13T14:50:21Z

👍 @honzakral
👎 RHEL's busted python implementation.

kevinava · 2014-03-13T20:32:26Z

Hi, again and thanks for grate support 👍 @honzakral

I've installed a new elasticsearch cluster from scratch with CentOS 6.5, Elasticsearch 1.0.1 and Curator (Master). And I can report that curator installed with pip install https://github.com/elasticsearch/curator/tarball/master works like a charm on a clean CentOS/ES/Logstash/Curator install/setup.
I don't even need to use the workaround.

However curator won't work on our existing ES cluster which had ES 0.90.x and Curator 0.6.x.
Existing ES cluster is working perfectly after upgrade from 0.90.x to 1.0.1, but Curator for some reason can't be reinstalled, installed, upgraded or run without errors.
On our setup it refuses to install elasticsearch-1.0.0 , and only installs 0.4.4 no matter what I do...
Manual upgrade doesn't help either, eg. pip install --upgrade elasticsearch==1.0.0.

``Requested elasticsearch>=1.0.0,<2.0.0 (from elasticsearch-curator==1.0.0-dev), but installing version 0.4.4`

I'll also investigate some more and see if I can figure it out.

Cheers

kevinava · 2014-03-14T10:48:46Z

Hi, I have now found a solution to "our" problem.
It has to do with some old lingering Curator / PIP files that prevent us from upgrading/reinstalling Curator to current version.

So this is what I did to resolve the issue on CentoOS 6.5.

Enter directory where Python's site-packages are installed.
cd /usr/lib/python2.6/site-packages/

Now uninstall previously installed Curator packages using pip and remove lingering files.

pip uninstall urllib3 elasticsearch elasticsearch_curator
rm -rf curator elasticsearch_curator*

Then remove all other possible pip and Curator files on your system.

rm -rf ~/.pip
rm -rf ~/curator
rm -rf /tmp/pip-build-root*

Finally use find /* -name curator to check if there are any lingering files from previously installed Curator and remove them.

Now we can install Curator and run it without the workarond.

pip install https://github.com/elasticsearch/curator/tarball/master

nwood888 · 2014-03-24T16:02:53Z

I think our issue was similar to what @kevinava described. But I didn't need to go to those lengths. I just did:

pip uninstall elasticsearch
pip install elasticsearch

And now everything works.

cleesmith · 2014-03-24T22:19:07Z

I see the following:

curator --host some.com -d 500 -n
2014-03-24T22:14:00.416 INFO                        main:332  Job starting...
2014-03-24T22:14:00.417 INFO                   _new_conn:172  Starting new HTTP connection (1): some.com
2014-03-24T22:14:02.584 INFO         log_request_success:49   GET http://some.com:9200/ [status:200 request:2.167s]
Expected Elasticsearch version range > 0.19.4 < 1.0.0
ERROR: Incompatible with version 1.0.1 of Elasticsearch.  Exiting.

python --version
Python 2.7.3

pip --version
pip 1.0 from /usr/lib/python2.7/dist-packages (python 2.7)

elasticsearch-curator     - Tending your time-series indices in Elasticsearch
  INSTALLED: 0.6.2 (latest)

untergeek · 2014-03-25T16:50:15Z

I will be releasing curator 1.0 later today. It should work with ES 1.0+ (including 1.1, which was released today).

—Aaron

On March 24, 2014 at 3:19:08 PM, chris (notifications@github.com) wrote:

I see the following:

curator --host some.com -d 500 -n
2014-03-24T22:14:00.416 INFO main:332 Job starting...
2014-03-24T22:14:00.417 INFO _new_conn:172 Starting new HTTP connection (1): some.com
2014-03-24T22:14:02.584 INFO log_request_success:49 GET http://some.com:9200/ [status:200 request:2.167s]
Expected Elasticsearch version range > 0.19.4 < 1.0.0
ERROR: Incompatible with version 1.0.1 of Elasticsearch. Exiting.

python --version
Python 2.7.3

pip --version
pip 1.0 from /usr/lib/python2.7/dist-packages (python 2.7)

—
Reply to this email directly or view it on GitHub.

cleesmith · 2014-03-25T16:52:09Z

Thanks, good job, it looks like a useful program ... even if it isn't written in Ruby :-)

untergeek · 2014-03-25T17:42:09Z

To clean up (if you have problems):

pip uninstall elasticsearch-curator
pip uninstall elasticsearch

To install the updated version:

pip install elasticsearch-curator

The release of curator 1.0.0 should resolve this, with these steps. If I get no updates after a week or so, I will mark this resolved.

cleesmith · 2014-03-25T17:44:47Z

Just to be clear, what is meant by:

pip uninstall elasticsearch

... as I installed elasticsearch by downloading the ".zip" file (or the ".deb" file for Ubuntu).
Is it uninstalling elasticsearch, if not, what does it mean ?
Please, pardon my lack of python/pip knowledge.

untergeek · 2014-03-25T18:22:39Z

pip uninstall elasticsearch will uninstall the python elasticsearch module, not your elasticsearch installation. The following pip install elasticsearch-curator will install the newest version of the elasticsearch python module as a dependency, so it will immediately get re-installed.

cleesmith · 2014-03-25T18:25:31Z

I see, the "elasticsearch" module is a dependency for "elasticsearch-curator".
Just curious, but is there a time line for adding snapshot/restore to curator ... I know it's
very new.
Thanks again and I will update if everything works now.

untergeek · 2014-03-25T18:26:29Z

No timeline yet. Working on this to be out as soon as possible, though. Too many other things are pulling me in other directions 😦

cleesmith · 2014-03-25T18:33:44Z

It works:

curator --host some.com -d 500 -n
2014-03-25T18:30:48.765 INFO                        main:333  Job starting...
2014-03-25T18:30:48.766 INFO                   _new_conn:172  Starting new HTTP connection (1): some.com
2014-03-25T18:30:50.878 INFO         log_request_success:49   GET http://some.com:9200/ [status:200 request:2.112s]
2014-03-25T18:30:50.879 INFO                        main:359  Deleting indices older than 500 days...
2014-03-25T18:30:50.884 INFO         log_request_success:49   GET http://some.com:9200/logstash-*/_settings?expand_wildcards=closed [status:200 request:0.004s]
2014-03-25T18:30:50.885 INFO                  index_loop:309  DELETE index operations completed.
2014-03-25T18:30:50.885 INFO                        main:379  Done in 0:00:02.123951.

I would "pitch in" and help with curator, but I'm an old C programmer and mostly use Ruby now with zero python experience.
Thanks again.

untergeek · 2014-03-25T18:41:14Z

Resolved!

Akshaykapoor · 2014-08-07T23:10:36Z

Hi @untergeek @honzakral ,

I have been trying to instal curator on my MAC OSX 10.9 following the pip install method but to no luck, I still get the same message,

File "/usr/local/bin/curator", line 5, in
from pkg_resources import load_entry_point
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2603, in
working_set.require(requires)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 666, in require
needed = self.resolve(parse_requirements(requirements))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0

Can you help me out with this.

Any help would be appreciated!

untergeek · 2014-08-08T16:53:49Z

Please try these steps in order:

1. Try re-installing the `elasticsearch` module

sudo pip uninstall elasticsearch
sudo pip install elasticsearch

See if curator works after this. If not, move on to attempt 2.

2. Complete reinstallation.

sudo pip uninstall elasticsearch-curator
sudo pip uninstall elasticsearch
sudo pip install elasticsearch
sudo pip install elasticsearch-curator

If this still fails, and it's possible it will, something about your python setup is disallowing discovery. Not all Mac users have encountered this, but a few have. I use curator on my Macbook Air, and I have never had this problem. I cannot for the life of me discover why it happens to some and not to others. Perhaps it's because I have Xcode installed too.

At worst, you'll have to run it from /Library/Python/2.7/site-packages/curator/curator.py (which is where pip installs it on mine).

Akshaykapoor · 2014-08-09T01:33:55Z

Hey @untergeek

I did try these steps before and just to be sure, did it again and to no surprise it failed. So, I guess running it from /Library/Python/2.7/site-packages/curator/curator.py is the only option in my case as well.

Anyways, thanks for the prompt reply!

cdenneen · 2014-09-26T15:48:53Z

Same here had to run it from curator.py.

untergeek · 2014-09-26T15:52:42Z

With curator 2.0, you'll have to run curator_script.py as curator.py now contains the library but not the command-line portions.

@cdenneen, What version of python and what OS + release are you seeing this with?

cdenneen · 2014-09-26T16:06:47Z

OS X 10.9.4, curator 0.6.2 for ES 0.99

untergeek · 2014-09-26T16:11:08Z

I'm not sure what is different about my mac environment, but some people have it work, some don't. I apologize. I may bundle a Mac standalone at some point via pyinstaller. Stay tuned.

I advise migrating to a 1.x version soon. The 0.90.x branch is being end-of-lived on 14 Oct 2014. That will also necessitate a newer version of curator as well.

cdenneen · 2014-09-26T16:13:13Z

@untergeek thanks... yeah this is my old ES cluster still running... new one is lastest GA

cdenneen · 2014-09-26T16:16:11Z

@untergeek even seeing on Linux:
[root@ctclnxmgr01 ~]# pip install elasticsearch==0.4.5
Requirement already satisfied (use --upgrade to upgrade): elasticsearch==0.4.5 in /usr/lib/python2.6/site-packages
Cleaning up...
[root@ctclnxmgr01 ~]# curator
Traceback (most recent call last):
File "/usr/bin/curator", line 5, in
from pkg_resources import load_entry_point
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in
working_set.require(requires)
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: elasticsearch>=0.4.4,<1.0.0

[root@ctclnxmgr01 ~]# pip install elasticsearch==0.4.5
Requirement already satisfied (use --upgrade to upgrade): elasticsearch==0.4.5 in /usr/lib/python2.6/site-packages
Cleaning up...

cdenneen · 2014-09-26T16:18:09Z

@untergeek

same with latest versions too:

[root@ctclnxmgr02 ~]# pip install elasticsearch-curator
Downloading/unpacking elasticsearch-curator
Downloading elasticsearch-curator-2.0.0.tar.gz
Running setup.py (path:/tmp/pip_build_root/elasticsearch-curator/setup.py) egg_info for package elasticsearch-curator
warning: no files found matching 'CHANGELOG'
warning: no previously-included files matching 'pycache' found under directory ''
warning: no previously-included files matching '.py[co]' found under directory ''
Downloading/unpacking elasticsearch>=1.0.0,<2.0.0 (from elasticsearch-curator)
Downloading elasticsearch-1.2.0-py2.py3-none-any.whl (51kB): 51kB downloaded
Downloading/unpacking argparse>=1.1.0 (from elasticsearch-curator)
Downloading argparse-1.2.1.tar.gz (69kB): 69kB downloaded
Running setup.py (path:/tmp/pip_build_root/argparse/setup.py) egg_info for package argparse
warning: no previously-included files matching '.pyc' found anywhere in distribution
warning: no previously-included files matching '.pyo' found anywhere in distribution
warning: no previously-included files matching '.orig' found anywhere in distribution
warning: no previously-included files matching '.rej' found anywhere in distribution
no previously-included directories found matching 'doc/build'
no previously-included directories found matching 'env24'
no previously-included directories found matching 'env25'
no previously-included directories found matching 'env26'
no previously-included directories found matching 'env27'
Downloading/unpacking urllib3>=1.8,<2.0 (from elasticsearch>=1.0.0,<2.0.0->elasticsearch-curator)
Downloading urllib3-1.9.1.tar.gz (171kB): 171kB downloaded
Running setup.py (path:/tmp/pip_build_root/urllib3/setup.py) egg_info for package urllib3
warning: no previously-included files matching '' found under directory 'docs/build'
Installing collected packages: elasticsearch-curator, elasticsearch, argparse, urllib3
Running setup.py install for elasticsearch-curator
warning: no files found matching 'CHANGELOG'
warning: no previously-included files matching 'pycache' found under directory ''
warning: no previously-included files matching '.py[co]' found under directory ''
Installing curator script to /usr/bin
Installing es_repo_mgr script to /usr/bin
Running setup.py install for argparse
warning: no previously-included files matching '.pyc' found anywhere in distribution
warning: no previously-included files matching '.pyo' found anywhere in distribution
warning: no previously-included files matching '.orig' found anywhere in distribution
warning: no previously-included files matching '_.rej' found anywhere in distribution
no previously-included directories found matching 'doc/build'
no previously-included directories found matching 'env24'
no previously-included directories found matching 'env25'
no previously-included directories found matching 'env26'
no previously-included directories found matching 'env27'
Running setup.py install for urllib3
warning: no previously-included files matching '' found under directory 'docs/_build'
Successfully installed elasticsearch-curator elasticsearch argparse urllib3
Cleaning up...
[root@ctclnxmgr02 ~]# curator
Traceback (most recent call last):
File "/usr/bin/curator", line 5, in
from pkg_resources import load_entry_point
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in
working_set.require(requires)
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0
[root@ctclnxmgr02 ~]# pip install elasticsearch
Requirement already satisfied (use --upgrade to upgrade): elasticsearch in /usr/lib/python2.6/site-packages
Cleaning up...

untergeek · 2014-09-26T16:23:01Z

Let me guess, RHEL or Centos? It's an old version of python, for one, and I've experienced this problem with entry_points with some distros of linux. For some reason the code in the entry_point does something funny when the required module is loaded.

#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'elasticsearch-curator==2.0.0','console_scripts','curator'
__requires__ = 'elasticsearch-curator==2.0.0'
import sys
from pkg_resources import load_entry_point

sys.exit(
   load_entry_point('elasticsearch-curator==2.0.0', 'console_scripts', 'curator')()
)

That's all that's in /usr/local/bin/curator on my MacBook Air. It could be something with the load_entry_point method, but it's frustrating to not know why.

cdenneen · 2014-09-26T16:23:44Z

CentOS... yes... and correct there as well... if I run the curator.py or curator_script.py on those 2 linux nodes with different version it works... but the bin wrapper doesn't want to work regardless

cdenneen · 2014-09-26T16:27:18Z

@untergeek i can create new issue for this if you'd like:

2014-09-26T12:23:35.223 INFO         log_request_success:57   DELETE http://eslogs-1.domain.loc:9200/logstash-2014.09.07 [status:200 request:10.032s]
2014-09-26T12:23:35.223 INFO                  index_loop:307  logstash-2014.09.07: Successfully deleted.
2014-09-26T12:23:35.224 INFO                  index_loop:297  Attempting to delete index logstash-2014.09.08 because it is 4 days, 0:00:00 older than cutoff.
2014-09-26T12:23:45.259 INFO         log_request_success:57   DELETE http://eslogs-1.domain.loc:9200/logstash-2014.09.08 [status:200 request:10.036s]
2014-09-26T12:23:45.260 INFO                  index_loop:307  logstash-2014.09.08: Successfully deleted.
2014-09-26T12:23:45.260 INFO                  index_loop:297  Attempting to delete index logstash-2014.09.09 because it is 3 days, 0:00:00 older than cutoff.
2014-09-26T12:23:55.296 INFO         log_request_success:57   DELETE http://eslogs-1.domain.loc:9200/logstash-2014.09.09 [status:200 request:10.036s]
2014-09-26T12:23:55.297 INFO                  index_loop:307  logstash-2014.09.09: Successfully deleted.
2014-09-26T12:23:55.297 INFO                  index_loop:297  Attempting to delete index logstash-2014.09.10 because it is 2 days, 0:00:00 older than cutoff.
2014-09-26T12:24:05.332 INFO         log_request_success:57   DELETE http://eslogs-1.domain.loc:9200/logstash-2014.09.10 [status:200 request:10.035s]
2014-09-26T12:24:05.332 INFO                  index_loop:307  logstash-2014.09.10: Successfully deleted.
2014-09-26T12:24:05.333 INFO                  index_loop:297  Attempting to delete index logstash-2014.09.11 because it is 1 day, 0:00:00 older than cutoff.
2014-09-26T12:24:15.365 INFO         log_request_success:57   DELETE http://eslogs-1.domain.loc:9200/logstash-2014.09.11 [status:200 request:10.032s]
2014-09-26T12:24:15.366 INFO                  index_loop:307  logstash-2014.09.11: Successfully deleted.
2014-09-26T12:24:15.366 INFO        find_expired_indices:203  logstash-2014.09.12 is 0:00:00 above the cutoff.
2014-09-26T12:24:15.366 INFO        find_expired_indices:203  logstash-2014.09.13 is 1 day, 0:00:00 above the cutoff.
2014-09-26T12:24:15.366 INFO        find_expired_indices:203  logstash-2014.09.14 is 2 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.366 INFO        find_expired_indices:203  logstash-2014.09.15 is 3 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.367 INFO        find_expired_indices:203  logstash-2014.09.16 is 4 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.367 INFO        find_expired_indices:203  logstash-2014.09.17 is 5 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.367 INFO        find_expired_indices:203  logstash-2014.09.18 is 6 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.367 INFO        find_expired_indices:203  logstash-2014.09.19 is 7 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.367 INFO        find_expired_indices:203  logstash-2014.09.20 is 8 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.367 INFO        find_expired_indices:203  logstash-2014.09.21 is 9 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.368 INFO        find_expired_indices:203  logstash-2014.09.22 is 10 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.368 INFO        find_expired_indices:203  logstash-2014.09.23 is 11 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.368 INFO        find_expired_indices:203  logstash-2014.09.24 is 12 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.368 INFO        find_expired_indices:203  logstash-2014.09.25 is 13 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.368 INFO        find_expired_indices:203  logstash-2014.09.26 is 14 days, 0:00:00 above the cutoff.
2014-09-26T12:24:15.369 INFO                  index_loop:308  DELETE index operations completed.
2014-09-26T12:24:15.369 INFO                        main:363  Closing indices older than 2 days...
2014-09-26T12:24:15.395 INFO         log_request_success:57   GET http://eslogs-1.domain.loc:9200/_settings [status:200 request:0.026s]
2014-09-26T12:24:15.396 INFO                  index_loop:297  Attempting to close index logstash-2014.09.12 because it is 12 days, 0:00:00 older than cutoff.
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/curator/curator.py", line 382, in <module>
    main()
  File "/Library/Python/2.7/site-packages/curator/curator.py", line 365, in main
    index_loop(client, 'close', expired_indices, arguments.dry_run)
  File "/Library/Python/2.7/site-packages/curator/curator.py", line 301, in index_loop
    skipped = op(client, index_name, **kwargs)
  File "/Library/Python/2.7/site-packages/curator/curator.py", line 249, in _close_index
    if index_closed(client, index_name):
  File "/Library/Python/2.7/site-packages/curator/curator.py", line 244, in index_closed
    filter_routing_table=True,
  File "/Library/Python/2.7/site-packages/elasticsearch/client/utils.py", line 68, in _wrapped
    return func(*args, params=params, **kwargs)
TypeError: state() got an unexpected keyword argument 'filter_blocks'

untergeek · 2014-09-26T16:30:13Z

Indeed, please raise a new issue with this filter_blocks thing. I've not seen that and would like to know what exactly triggered it. I may need a new unit test to catch this as my current ones all pass.

RJ3000 · 2015-03-09T12:21:39Z

Tried everything on this thread and had no luck with getting Curator to work, posting incase it helps anyone else

CentOS release 6.5
Python 2.6.6

Different Errors included:
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0
pkg_resources.DistributionNotFound: argparse>=1.1.0
pkg_resources.DistributionNotFound: click>=3.3

This fixed the issue for us:
pip install -U setuptools

curator -v
curator 2.1.2

untergeek · 2015-03-09T17:59:52Z

Thank you! I think you finally figured it out. It's an outdated version of the setuptools!

If that's the case, this problem will persist in v3. I will update the FAQ and docs in the wiki.

wjimenez5271 · 2015-04-21T00:18:43Z

Ditto @RJ3000, pip install -U setuptools worked for me also on Centos 6.6

untergeek · 2015-04-21T18:03:31Z

See the new FAQ

wjimenez5271 · 2015-04-21T18:07:58Z

@untergeek thats good. I wonder if perhaps it should be mentioned in the README as well? I'll be happy to submit a PR...

untergeek · 2015-04-21T18:11:39Z

I will be happy to merge an addition there, but would prefer it to be more in the vein of:

DistributionNotFound errors?

See this FAQ

I want to keep people moving to the official documentation, rather than have lengthy stuff in the development README.

wjimenez5271 · 2015-04-21T20:31:28Z

OK good thought.

wjimenez5271 · 2015-04-21T20:39:39Z

OK see #361

untergeek self-assigned this Mar 25, 2014

untergeek added the bug label Mar 25, 2014

untergeek closed this as completed Mar 25, 2014

untergeek mentioned this issue Oct 20, 2014

Does curator run with elasticsearch installed via RPM #202

Closed

untergeek mentioned this issue Apr 21, 2015

Curator install from source error #354

Closed

tony mentioned this issue Jul 9, 2015

pkg_resources.DistributionNotFound: argparse on RHEL-based distros tmux-python/tmuxp#111

Closed

Master branch doesn't work with Elasticsearch 1.0.1 #56

Master branch doesn't work with Elasticsearch 1.0.1 #56

Comments

akkie commented Mar 3, 2014

honzakral commented Mar 3, 2014

akkie commented Mar 3, 2014

honzakral commented Mar 3, 2014

untergeek commented Mar 3, 2014

akkie commented Mar 3, 2014

msolujic commented Mar 4, 2014

kevinava commented Mar 13, 2014

honzakral commented Mar 13, 2014

untergeek commented Mar 13, 2014

kevinava commented Mar 13, 2014

kevinava commented Mar 14, 2014

nwood888 commented Mar 24, 2014

cleesmith commented Mar 24, 2014

untergeek commented Mar 25, 2014

cleesmith commented Mar 25, 2014

untergeek commented Mar 25, 2014

cleesmith commented Mar 25, 2014

untergeek commented Mar 25, 2014

cleesmith commented Mar 25, 2014

untergeek commented Mar 25, 2014

cleesmith commented Mar 25, 2014

untergeek commented Mar 25, 2014

Akshaykapoor commented Aug 7, 2014

untergeek commented Aug 8, 2014

1. Try re-installing the elasticsearch module

2. Complete reinstallation.

Akshaykapoor commented Aug 9, 2014

cdenneen commented Sep 26, 2014

untergeek commented Sep 26, 2014

cdenneen commented Sep 26, 2014

untergeek commented Sep 26, 2014

cdenneen commented Sep 26, 2014

cdenneen commented Sep 26, 2014

cdenneen commented Sep 26, 2014

untergeek commented Sep 26, 2014

cdenneen commented Sep 26, 2014

cdenneen commented Sep 26, 2014

untergeek commented Sep 26, 2014

RJ3000 commented Mar 9, 2015

untergeek commented Mar 9, 2015

wjimenez5271 commented Apr 21, 2015

untergeek commented Apr 21, 2015

wjimenez5271 commented Apr 21, 2015

untergeek commented Apr 21, 2015

DistributionNotFound errors?

wjimenez5271 commented Apr 21, 2015

wjimenez5271 commented Apr 21, 2015

1. Try re-installing the `elasticsearch` module