Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ReadTheDocs is failing to build #26

Closed
richardotis opened this issue Nov 9, 2015 · 10 comments
Closed

ReadTheDocs is failing to build #26

richardotis opened this issue Nov 9, 2015 · 10 comments
Labels

Comments

@richardotis
Copy link
Collaborator

https://readthedocs.org/projects/pycalphad/builds/3465916/

Installed /home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/lib/python2.7/site-packages/tinydb-2.4-py2.7.egg
Searching for autograd
Reading https://pypi.python.org/simple/autograd/
Best match: autograd 1.1.2
Downloading https://pypi.python.org/packages/source/a/autograd/autograd-1.1.2.tar.gz#md5=db1cc306f37bb39f30d36c72a39ce1e9
Processing autograd-1.1.2.tar.gz
Writing /tmp/easy_install-Wtdg_w/autograd-1.1.2/setup.cfg
Running autograd-1.1.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Wtdg_w/autograd-1.1.2/egg-dist-tmp-Sj6EJJ
Traceback (most recent call last):
  File "setup.py", line 47, in <module>
    'Programming Language :: Python :: 3.4'
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 380, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 610, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 661, in install_item
    self.process_distribution(spec, dist, deps)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 709, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 836, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1081, in best_match
    return self.obtain(req, installer)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1093, in obtain
    return installer(requirement)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 629, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 659, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 842, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1070, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1056, in run_setup
    run_setup(setup_script, args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 240, in run_setup
    raise
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 193, in setup_context
    yield
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 164, in save_modules
    saved_exc.resume()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 139, in resume
    compat.reraise(type, exc, self._tb)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 152, in save_modules
    yield saved
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 193, in setup_context
    yield
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 237, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 267, in run
    return func()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 236, in runner
    _execfile(setup_script, ns)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 46, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-Wtdg_w/autograd-1.1.2/setup.py", line 79, in <module>

  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/bdist_egg.py", line 151, in run
    self.run_command("egg_info")
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 180, in run
    self.find_sources()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 207, in find_sources
    mm.run()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 291, in run
    self.add_defaults()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 320, in add_defaults
    sdist.add_defaults(self)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pycalphad/envs/develop/local/lib/python2.7/site-packages/setuptools/command/sdist.py", line 130, in add_defaults
    build_ext = self.get_finalized_command('build_ext')
  File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
    cmd_obj.ensure_finalized()
  File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
    self.finalize_options()
  File "/tmp/easy_install-Wtdg_w/autograd-1.1.2/setup.py", line 22, in finalize_options
    url='https://github.com/richardotis/pycalphad',
AttributeError: 'dict' object has no attribute '__NUMPY_SETUP__'
@richardotis richardotis added the bug label Nov 9, 2015
@richardotis
Copy link
Collaborator Author

@mattjj I can reproduce the same error we talked about earlier (#25) on ReadTheDocs, and locally. If I pre-install numpy and scipy into a clean environment (for time's sake) and then run python setup.py install on pycalphad, I get the same error complaining about something in autograd's setup.py.

@mattjj
Copy link

mattjj commented Nov 9, 2015

Just to clarify, the problem is that the name __builtins__ is defined but it's a dict instead of a module, right?

It sounds like __builtins__ is an implementation detail and relying on it is not good form. I tried to remedy this problem in the autograd setup.py file but failed and gave up. I'll let you know if I get it working but I'm not sure when I'll get a chance. Let me know if you make any progress on your end.

Do you know why it's failing in some environments but not on our Travis builds (which check 2.7 and 3.5)?

@mattjj
Copy link

mattjj commented Nov 9, 2015

Oh hey check this out and, even better, the doc page it links to.

@mattjj
Copy link

mattjj commented Nov 9, 2015

I think this could be a good solution! I'm going to try it.

@mattjj
Copy link

mattjj commented Nov 9, 2015

Is it feasible for you to try the builtins-when-setup-is-not-main branch of autograd? I applied the fix linked above and I think it might solve your build problems.

@richardotis
Copy link
Collaborator Author

I'll give it a shot.

@richardotis
Copy link
Collaborator Author

I just installed that branch into a clean environment with all of pycalphad's other dependencies and python setup.py install worked. If I try to do the same thing against the autograd on PyPI, it fails. However, if I install autograd's master branch, it also works. I'm thinking it's more a function of being preinstalled (and thus hooking into something setuptools is doing). I'm not sure there's a way to figure out if your fix works without involving PyPI somehow.

I'm going to play with the requirements.txt file on RTD and see if I can't bypass this issue by preinstalling autograd.

@mattjj
Copy link

mattjj commented Nov 9, 2015

I'm pretty confident that new setup.py will fix the error you were running into, and it didn't break anything in autograd, so I just merged it into the master branch.

@richardotis
Copy link
Collaborator Author

I updated the docs/requirements.txt file to include autograd in 98d09d7 and (after fixing a bug in 8f535c7) now RTD appears to build correctly. Thanks again for the help @mattjj

@mattjj
Copy link

mattjj commented Nov 12, 2015

Happy to help! Cool project!

echu referenced this issue in embotech/ecos-python Apr 27, 2018
Previously we would error when trying to install and numpy was not
installed. This now just goes ahead and installs numpy for you before
building the extension.
bocklund added a commit to bocklund/pycalphad that referenced this issue Aug 17, 2021
* ENH: Support restarting from a previous chain

A new calculation can be restarted from a previous chain.
As a side effect, the chain we output to file is the real chain and not the flat chain.

* ENH: Raise if tracefile/probfile will be overwritten

Because MCMC fitting is relatively long-running and RAM intensive, it is ideal to be able to restart from a previous calculation.

Currently, to restart, a user would manually update the parameters in the database to the parameters with the lowest probability from the previous run, then use that set of parameters to initialize a new set of walkers.

Since we save the state of the chains as the tracefile anyways, we can use the chains to restart a run in emcee and avoid tuning the walker initialization parameters on restart.

To make this (and analysis) easier, chains and log probabilities are now written out as binaries, rather than text files. This preserves the shape with the number of chains, so the chain file is now of shape (k, n, p), where k is number of walkers, n is the preallocated number of iterations, p is the number of parameters. The previous shape was (kn, p). The same is true for the log probability file, which is now of shape (k, n) rather than (kn).

This change also introduces a command line argument --restart which takes the path to a chain file. If the name of the input chain conflicts with the default name, an error will be raised.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants