setup error in combination with setuptools install requires

[marscher]

A setuptools project like this:

from setuptools import setup
setup(name='test', version=0, install_requires=['psutil'])

causes an memory eating Python monster process. The stack trace (partial, because my 8GB ram were already full):

(Pdb) where
  /storage/mi/marscher/jenkins_ws/pyemma-doc-devel/setup.py(290)<module>()
-> setup(**metadata)
  /home/mi/marscher/miniconda/envs/build_doc/lib/python2.7/distutils/core.py(151)setup()
-> dist.run_commands()
  /home/mi/marscher/miniconda/envs/build_doc/lib/python2.7/distutils/dist.py(953)run_commands()
-> self.run_command(cmd)
  /home/mi/marscher/miniconda/envs/build_doc/lib/python2.7/distutils/dist.py(972)run_command()
-> cmd_obj.run()
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/develop.py(32)run()
-> self.install_for_development()
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/develop.py(132)install_for_development()
-> self.process_distribution(None, self.dist, not self.no_deps)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py(718)process_distribution()
-> [requirement], self.local_index, self.easy_install
  /home/mi/marscher/.local/lib/python2.7/site-packages/pkg_resources/__init__.py(836)resolve()
-> dist = best[req.key] = env.best_match(req, ws, installer)
  /home/mi/marscher/.local/lib/python2.7/site-packages/pkg_resources/__init__.py(1081)best_match()
-> return self.obtain(req, installer)
  /home/mi/marscher/.local/lib/python2.7/site-packages/pkg_resources/__init__.py(1093)obtain()
-> return installer(requirement)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py(638)easy_install()
-> return self.install_item(spec, dist.location, tmpdir, deps)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py(668)install_item()
-> dists = self.install_eggs(spec, download, tmpdir)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py(851)install_eggs()
-> return self.build_and_install(setup_script, setup_base)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py(1079)build_and_install()
-> self.run_setup(setup_script, setup_base, args)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py(1065)run_setup()
-> run_setup(setup_script, args)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(240)run_setup()
-> raise
  /home/mi/marscher/miniconda/envs/build_doc/lib/python2.7/contextlib.py(35)__exit__()
-> self.gen.throw(type, value, traceback)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(193)setup_context()
-> yield
  /home/mi/marscher/miniconda/envs/build_doc/lib/python2.7/contextlib.py(35)__exit__()
-> self.gen.throw(type, value, traceback)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(152)save_modules()
-> yield saved
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(126)__exit__()
-> self._saved = UnpickleableException.dump(type, exc)
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()
-> return cls.dump(cls, cls(repr(exc)))
  /home/mi/marscher/.local/lib/python2.7/site-packages/setuptools/sandbox.py(110)dump()

I think there is some endless recursion in sandbox.py, which might indicate that this is a setuptools bug. I've tested some older versions, but still the same behaviour (and only with psutil)

[marscher]

I forgot to mention that this happens only for develop mode for project a.

python setup.py develop

[giampaolo]

Mmmm can you please try 2.0.0 and latest 1.x versions? Also I don't understand the error. Is that the full traceback message?

[marscher]

I do not understand the error too. I had to kill/interrupt the process to obtain a stacktrace, because it does not terminate. But no matter how long I wait before forcefully terminating the process, the last element of the stack is always this setuptools.sandbox.dump() function.

Maybe this error is more related to setuptools than psutil. I also noticed that it only occurs if Cython is also installed. I think I'm going to post this in the setuptools repo.

Btw it does not matter whether 1.x, 2.0.0 or latest version of psutil is used.

[marscher]

https://bitbucket.org/pypa/setuptools/issues/448/develop-mode-in-anaconda-environment-does