Skip to content
This repository has been archived by the owner on Sep 1, 2023. It is now read-only.

Must provide nupic core release directory. --nupic-core-dir #619

Closed
rhyolight opened this issue Sep 22, 2015 · 57 comments
Closed

Must provide nupic core release directory. --nupic-core-dir #619

rhyolight opened this issue Sep 22, 2015 · 57 comments

Comments

@rhyolight
Copy link
Member

Originally reported as numenta/nupic-legacy#2570 and numenta/nupic-legacy#2543 but now failing with a different error that seems to be internal to nupic.core.

Just to clarify, the command pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.2-cp27-none-linux_x86_64.whl seems to work fine, but the subsequent pip install nupic then throws the following error.

CC @breznak @felippemr

(nupic_lab)nupic_lab$ pip uninstall nupic nupic.bindings -y
Cannot uninstall requirement nupic, not installed
Storing debug log for failure in /Users/felippe.gestec/.pip/pip.log
(nupic_lab)nupic_lab$ pip freeze
numpy==1.9.2
wsgiref==0.1.2
(nupic_lab)nupic_lab$ pip install nupic
Downloading/unpacking nupic
  Downloading nupic-0.3.1-py2-none-any.whl (5.3MB): 5.3MB downloaded
Downloading/unpacking PyMySQL==0.6.2 (from nupic)
  Downloading PyMySQL-0.6.2-py2.py3-none-any.whl (63kB): 63kB downloaded
Requirement already satisfied (use --upgrade to upgrade): numpy==1.9.2 in /Users/felippe.gestec/.virtualenvs/nupic_lab/lib/python2.7/site-packages (from nupic)
Downloading/unpacking nupic.bindings==0.2 (from nupic)
  Downloading nupic.bindings-0.2.tar.gz (593kB): 593kB downloaded
  Running setup.py (path:/Users/felippe.gestec/.virtualenvs/nupic_lab/build/nupic.bindings/setup.py) egg_info for package nupic.bindings
    Python Bindings directory: /Users/felippe.gestec/.virtualenvs/nupic_lab/build/nupic.bindings
    NUMPY VERSION: 1.9.2
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/Users/felippe.gestec/.virtualenvs/nupic_lab/build/nupic.bindings/setup.py", line 534, in <module>
        raise Exception("Must provide nupic core release directory. --nupic-core-dir")
    Exception: Must provide nupic core release directory. --nupic-core-dir
    Complete output from command python setup.py egg_info:
    Python Bindings directory: /Users/felippe.gestec/.virtualenvs/nupic_lab/build/nupic.bindings

NUMPY VERSION: 1.9.2

Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/Users/felippe.gestec/.virtualenvs/nupic_lab/build/nupic.bindings/setup.py", line 534, in <module>

    raise Exception("Must provide nupic core release directory. --nupic-core-dir")

Exception: Must provide nupic core release directory. --nupic-core-dir

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /Users/felippe.gestec/.virtualenvs/nupic_lab/build/nupic.bindings
Storing debug log for failure in /Users/felippe.gestec/.pip/pip.log
(nupic_lab)nupic_lab$ pip uninstall nupic nupic.bindings -y
Cannot uninstall requirement nupic, not installed
Storing debug log for failure in /Users/felippe.gestec/.pip/pip.log

@rhyolight
Copy link
Member Author

@oxtopus @scottpurdy Here is an updated bug report of the problem I mentioned in standup this morning. I think this is a P1 bug because I have multiple reports of this problem halting OS X binary installations.

@oxtopus
Copy link
Contributor

oxtopus commented Sep 22, 2015

nupic now depends on a more recent version of nupic.bindings (0.2.1 from https://github.com/numenta/nupic/blob/master/external/common/requirements.txt#L22). We'll need to validate that this is still a problem in recent nupic and nupic.core (I suspect not).

@rhyolight
Copy link
Member Author

@oxtopus Releasing 0.3.3 now...

@oxtopus
Copy link
Contributor

oxtopus commented Sep 22, 2015

This ought to be fixed in #593

Or, to put it another way, #593 shouldn't be considered done until it can build from source without having to specify --nupic-core-dir.

@rhyolight
Copy link
Member Author

@oxtopus That leads me to believe that 0.3.3 will not fix this issue 😭 I'll try it anyway.

@oxtopus
Copy link
Contributor

oxtopus commented Sep 22, 2015

Probably not.

@rhyolight
Copy link
Member Author

@oxtopus @scottpurdy @subutai @jcasner I think we need to make this issue our top priority. I've confirmed that Linux binary installations are not working, and I have strong reason to believe that OS X has the same problem.

@oxtopus
Copy link
Contributor

oxtopus commented Sep 22, 2015

👍 though, installing nupic.bindings separately should fix the problem.

@rhyolight
Copy link
Member Author

Yes it will, but I want a friction-less binary installation.

@oxtopus
Copy link
Contributor

oxtopus commented Sep 22, 2015

Meanwhile, easy_install nupic works for me in a fresh ubuntu 14.04 instance.

@scottpurdy
Copy link
Contributor

I think we should update the README to state that Linux users can A) use easy_install, B) manually install nupic.bindings from S3 Linux wheel, or C) install nupic.bindings from source/repo.

I think it will be better to work towards a source build than bring back all of the logic for downloading and including the binary linux egg for nupic.bindings in the setup.py of nupic.

@rhyolight and @oxtopus - thoughts?

@oxtopus
Copy link
Contributor

oxtopus commented Sep 22, 2015

👍 for install-from-source. Let's commit to moving away from setup.py hackery.

@scottpurdy
Copy link
Contributor

@rhyolight ?

@rhyolight
Copy link
Member Author

B) manually install nupic.bindings from S3 Linux wheel

This is what the README says to do now, and it is broken.

@rhyolight
Copy link
Member Author

Just a note... I have an Ubuntu laptop I've been running NuPIC on, but I haven't updated it in a few months. This morning I pulled in all the latest nupic changes and ran:

pip uninstall nupic nupic.bindings -y
pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.2.1-cp27-none-linux_x86_64.whl
pip install nupic

And it worked fine. However, yesterday when I started from scratch on Ubuntu 12, I ran directly into the exact reported issue. I don't know what would be different between a fresh Ubuntu and an older one.

@rcrowder
Copy link
Contributor

The same results with Matt in my Ubuntu 14.04 VM [1], and a Debian 8.2 VM [2], (default build-essential installs 4.9 with worked!). Tried Debian to see if it recreated Ryan's problem, but this morning 0.2.1 and nupic installed fine (after making sure known pip packages are installed first, such as upgrading default numpy, being super user for numpy install, etc.).

uname -a
1 Linux nupic-VirtualBox 3.16.0-49-generic #65~14.04.1-Ubuntu SMP Wed Sep 9 10:03:23 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
2 Linux debian 3.16.0-4-amd64 #1 Debian 3.16.7-ctk11-1+deb8u4 (2015-09-19) x86_64 GNU/Linux

@rhyolight
Copy link
Member Author

So I just ran though the exact same installation procedure as yesterday on Ubuntu again (fresh VM), and I did NOT run into this problem this time. I guess our release of 0.3.3 yesterday actually fixed this problem?!

@rhyolight
Copy link
Member Author

Something happened in one of these change sets that fixed the problem for me:

@rhyolight
Copy link
Member Author

I think I know what happened here.

Yesterday, after the 0.3.3 release, I noticed that the README instructions pointed to nupic.bindings 0.2, which was incorrect, so I changed it to 0.2.1. I think this was just an oversight in the 0.3.3 release, but it turns out that 0.2 has been the bindings release stated in the README for over two weeks, when it should have been updated to 0.2.1 a week ago. So for the past week, our README has been in a bad state. It is now fixed (as of 20 hours ago).

@felippemr
Copy link

But I can't install it...the same error with nupic.bindings persists.
Will you open another issue or release a new version?
What am I supposed to do to install it?

@rhyolight
Copy link
Member Author

@felippemr Are you getting the error you originally reported in numenta/nupic-legacy#2570?

Exception: Must provide path to nupic core release. export NUPIC_CORE_RELEASE=

Or are you getting the error reported above in this issue?

Exception: Must provide nupic core release directory. --nupic-core-dir

Either way, the console output from the latest 0.3.3 release would be helpful. Can you run the following commands (again, I know!) and give us the console output you get on OS X?

pip uninstall nupic nupic.bindings -y
pip install nupic --no-cache-dir --user

If I can see the output of pip install nupic, I can tell which versions are being fetched from pypi. Thanks.

@rhyolight
Copy link
Member Author

@felippemr Also, what version of pip are you using? (pip --version).

And thank you for persisting with us on this issue. I really want to make sure that there are no current installation problems. I'll be getting a new OS X machine today so I can make an installation video. I'll run into the same problems you are running into (hopefully, if the issue persists).

@felippemr
Copy link

I am getting this error:

Exception: Must provide nupic core release directory. --nupic-core-dir

When I do this pip uninstall nupic nupic.bindings -y pip install nupic --no-cache-dir --user, I get this:

(heimdall)(master *)numenta$ pip uninstall nupic nupic.bindings -y
Cannot uninstall requirement nupic, not installed
Storing debug log for failure in /Users/felippe.gestec/.pip/pip.log
(heimdall)(master *)numenta$ pip install nupic --no-cache-dir --user

Usage:
  pip install [options] <requirement specifier> ...
  pip install [options] -r <requirements file> ...
  pip install [options] [-e] <vcs project url> ...
  pip install [options] [-e] <local project path> ...
  pip install [options] <archive url/path> ...

no such option: --no-cache-dir

The output to pip install nupicis:

(heimdall)(master *)numenta$ pip install nupic
Downloading/unpacking nupic
  Downloading nupic-0.3.3-py2-none-any.whl (5.0MB): 5.0MB downloaded
Downloading/unpacking PyMySQL==0.6.2 (from nupic)
  Downloading PyMySQL-0.6.2-py2.py3-none-any.whl (63kB): 63kB downloaded
Downloading/unpacking nupic.bindings==0.2.1 (from nupic)
  Downloading nupic.bindings-0.2.1.tar.gz (593kB): 593kB downloaded
  Running setup.py (path:/Users/felippe.gestec/.virtualenvs/heimdall/build/nupic.bindings/setup.py) egg_info for package nupic.bindings
    Python Bindings directory: /Users/felippe.gestec/.virtualenvs/heimdall/build/nupic.bindings
    NUMPY VERSION: 1.9.2
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/Users/felippe.gestec/.virtualenvs/heimdall/build/nupic.bindings/setup.py", line 534, in <module>
        raise Exception("Must provide nupic core release directory. --nupic-core-dir")
    Exception: Must provide nupic core release directory. --nupic-core-dir
    Complete output from command python setup.py egg_info:
    Python Bindings directory: /Users/felippe.gestec/.virtualenvs/heimdall/build/nupic.bindings

NUMPY VERSION: 1.9.2

Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/Users/felippe.gestec/.virtualenvs/heimdall/build/nupic.bindings/setup.py", line 534, in <module>

    raise Exception("Must provide nupic core release directory. --nupic-core-dir")

Exception: Must provide nupic core release directory. --nupic-core-dir

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /Users/felippe.gestec/.virtualenvs/heimdall/build/nupic.bindings
Storing debug log for failure in /Users/felippe.gestec/.pip/pip.log

And finally my pip version is:

(heimdall)(master *)numenta$ pip --version
pip 1.5.4 from /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (python 2.7)

@rcrowder
Copy link
Contributor

@felippemr Before a reply from Matt, could you also run pip list and dump the output here please.

@rhyolight
Copy link
Member Author

Reopened as a P2, because I don't think this affects all users. This may be a pip issue. I will own this and work with @felippemr to get it resolved ASAP.

@felippemr are you comfortable upgrading pip? A newer version might solve this problem. Try: pip install --upgrade pip?

@felippemr
Copy link

(heimdall)(master *)numenta$ pip install --upgrade pip
Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl#md5=5ff9fec0be479e4e36df467556deed4d
  Downloading pip-7.1.2-py2.py3-none-any.whl (1.1MB): 1.1MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
(heimdall)(master *)numenta$ pip install https://pypi.python.org/packages/cp27/n/nupic.bindings/nupic.bindings-0.2.1-cp27-none-macosx_10_9_intel.whl
Collecting nupic.bindings==0.2.1 from https://pypi.python.org/packages/cp27/n/nupic.bindings/nupic.bindings-0.2.1-cp27-none-macosx_10_9_intel.whl
/Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading nupic.bindings-0.2.1-cp27-none-macosx_10_9_intel.whl (4.9MB)
    100% |████████████████████████████████| 4.9MB 89kB/s
Collecting pytest-cov==1.6 (from nupic.bindings==0.2.1)
  Downloading pytest-cov-1.6.tar.gz
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.9.2 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from nupic.bindings==0.2.1)
Collecting pytest-xdist==1.8 (from nupic.bindings==0.2.1)
  Downloading pytest-xdist-1.8.zip
Collecting pytest==2.5.1 (from nupic.bindings==0.2.1)
  Downloading pytest-2.5.1.tar.gz (577kB)
    100% |████████████████████████████████| 581kB 696kB/s
Collecting cov-core>=1.6 (from pytest-cov==1.6->nupic.bindings==0.2.1)
  Downloading cov-core-1.15.0.tar.gz
Collecting execnet>=1.0.8 (from pytest-xdist==1.8->nupic.bindings==0.2.1)
  Downloading execnet-1.4.1-py2.py3-none-any.whl (40kB)
    100% |████████████████████████████████| 40kB 6.1MB/s
Collecting py>=1.4.19 (from pytest==2.5.1->nupic.bindings==0.2.1)
  Using cached py-1.4.30-py2.py3-none-any.whl
Collecting coverage>=3.6 (from cov-core>=1.6->pytest-cov==1.6->nupic.bindings==0.2.1)
  Downloading coverage-4.0.tar.gz (166kB)
    100% |████████████████████████████████| 167kB 1.7MB/s
Collecting apipkg>=1.4 (from execnet>=1.0.8->pytest-xdist==1.8->nupic.bindings==0.2.1)
  Downloading apipkg-1.4-py2.py3-none-any.whl
Installing collected packages: py, pytest, coverage, cov-core, pytest-cov, apipkg, execnet, pytest-xdist, nupic.bindings
  Running setup.py install for pytest
  Running setup.py install for coverage
  Running setup.py install for cov-core
  Running setup.py install for pytest-cov
  Running setup.py install for pytest-xdist
Successfully installed apipkg-1.4 cov-core-1.15.0 coverage-4.0 execnet-1.4.1 nupic.bindings-0.2.1 py-1.4.30 pytest-2.5.1 pytest-cov-1.6 pytest-xdist-1.8
/Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
(heimdall)(master *)numenta$ pip install nupic
Collecting nupic
/Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading nupic-0.3.3-py2-none-any.whl (5.0MB)
    100% |████████████████████████████████| 5.0MB 89kB/s
Requirement already satisfied (use --upgrade to upgrade): pytest-cov==1.6 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): numpy==1.9.2 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from nupic)
Collecting asteval==0.9.1 (from nupic)
  Downloading asteval-0.9.1.tar.gz
Collecting prettytable==0.7.2 (from nupic)
  Downloading prettytable-0.7.2.tar.bz2
Requirement already satisfied (use --upgrade to upgrade): pytest==2.5.1 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from nupic)
Collecting psutil==1.0.1 (from nupic)
  Downloading psutil-1.0.1.tar.gz (156kB)
    100% |████████████████████████████████| 159kB 2.1MB/s
Collecting pycapnp==0.5.5 (from nupic)
  Downloading pycapnp-0.5.5.tar.gz (427kB)
    100% |████████████████████████████████| 430kB 178kB/s
Collecting PyYAML==3.10 (from nupic)
  Downloading PyYAML-3.10.tar.gz (241kB)
    100% |████████████████████████████████| 241kB 1.6MB/s
Requirement already satisfied (use --upgrade to upgrade): pytest-xdist==1.8 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from nupic)
Collecting unittest2==0.5.1 (from nupic)
  Downloading unittest2-0.5.1.tar.gz (62kB)
    100% |████████████████████████████████| 65kB 3.5MB/s
Collecting pyproj==1.9.3 (from nupic)
  Downloading pyproj-1.9.3.tar.gz (2.7MB)
    100% |████████████████████████████████| 2.7MB 121kB/s
Collecting DBUtils==1.1 (from nupic)
  Downloading DBUtils-1.1.tar.gz (109kB)
    100% |████████████████████████████████| 110kB 209kB/s
Collecting python-dateutil==2.1 (from nupic)
  Downloading python-dateutil-2.1.tar.gz (152kB)
    100% |████████████████████████████████| 155kB 299kB/s
Collecting pillow==2.3.0 (from nupic)
  Downloading Pillow-2.3.0.zip (2.4MB)
    100% |████████████████████████████████| 2.4MB 142kB/s
Requirement already satisfied (use --upgrade to upgrade): nupic.bindings==0.2.1 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from nupic)
Collecting validictory==0.9.1 (from nupic)
  Downloading validictory-0.9.1.tar.gz
Collecting ordereddict==1.1 (from nupic)
  Downloading ordereddict-1.1.tar.gz
Collecting PyMySQL==0.6.2 (from nupic)
  Downloading PyMySQL-0.6.2-py2.py3-none-any.whl (63kB)
    100% |████████████████████████████████| 65kB 3.2MB/s
Collecting coverage==3.7.1 (from nupic)
Collecting mock==1.0.1 (from nupic)
  Downloading mock-1.0.1.tar.gz (818kB)
    100% |████████████████████████████████| 819kB 510kB/s
Requirement already satisfied (use --upgrade to upgrade): cov-core>=1.6 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from pytest-cov==1.6->nupic)
Requirement already satisfied (use --upgrade to upgrade): py>=1.4.19 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from pytest==2.5.1->nupic)
Requirement already satisfied (use --upgrade to upgrade): execnet>=1.0.8 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from pytest-xdist==1.8->nupic)
Collecting six (from python-dateutil==2.1->nupic)
  Downloading six-1.9.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /Users/felippe.gestec/.virtualenvs/heimdall/lib/python2.7/site-packages (from execnet>=1.0.8->pytest-xdist==1.8->nupic)
Installing collected packages: asteval, prettytable, psutil, pycapnp, PyYAML, unittest2, pyproj, DBUtils, six, python-dateutil, pillow, validictory, ordereddict, PyMySQL, coverage, mock, nupic
  Running setup.py install for asteval
  Running setup.py install for prettytable
  Running setup.py install for psutil
  Running setup.py install for pycapnp
    Complete output from command /Users/felippe.gestec/.virtualenvs/heimdall/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/pip-build-LDNNAm/pycapnp/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/pip-_JorSQ-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/felippe.gestec/.virtualenvs/heimdall/bin/../include/site/python2.7/pycapnp:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.9-x86_64-2.7
    creating build/lib.macosx-10.9-x86_64-2.7/capnp
    copying capnp/__init__.py -> build/lib.macosx-10.9-x86_64-2.7/capnp
    copying capnp/_gen.py -> build/lib.macosx-10.9-x86_64-2.7/capnp
    copying capnp/version.py -> build/lib.macosx-10.9-x86_64-2.7/capnp
    copying capnp/__init__.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp
    copying capnp/c++.capnp -> build/lib.macosx-10.9-x86_64-2.7/capnp
    copying capnp/schema.capnp -> build/lib.macosx-10.9-x86_64-2.7/capnp
    creating build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/__init__.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/helpers.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/non_circular.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/asyncHelper.h -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/capabilityHelper.h -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/checkCompiler.h -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/fixMaybe.h -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/rpcHelper.h -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    copying capnp/helpers/serialize.h -> build/lib.macosx-10.9-x86_64-2.7/capnp/helpers
    creating build/lib.macosx-10.9-x86_64-2.7/capnp/includes
    copying capnp/includes/__init__.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/includes
    copying capnp/includes/capnp_cpp.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/includes
    copying capnp/includes/schema_cpp.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/includes
    copying capnp/includes/types.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/includes
    creating build/lib.macosx-10.9-x86_64-2.7/capnp/lib
    copying capnp/lib/__init__.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/lib
    copying capnp/lib/capnp.pxd -> build/lib.macosx-10.9-x86_64-2.7/capnp/lib
    copying capnp/lib/__init__.py -> build/lib.macosx-10.9-x86_64-2.7/capnp/lib
    copying capnp/lib/pickle_helper.py -> build/lib.macosx-10.9-x86_64-2.7/capnp/lib
    copying capnp/lib/capnp.pyx -> build/lib.macosx-10.9-x86_64-2.7/capnp/lib
    creating build/lib.macosx-10.9-x86_64-2.7/capnp/templates
    copying capnp/templates/module.pyx -> build/lib.macosx-10.9-x86_64-2.7/capnp/templates
    copying capnp/templates/setup.py.tmpl -> build/lib.macosx-10.9-x86_64-2.7/capnp/templates
    running build_ext
    creating var
    creating var/folders
    creating var/folders/gz
    creating var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp
    creating var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T
    creating var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/tmpksOTRn
    cc -arch x86_64 -c /var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/tmpksOTRn/vers.cpp -o var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/tmpksOTRn/vers.o --std=c++11
    cc -arch x86_64 -undefined dynamic_lookup var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/tmpksOTRn/vers.o -o /var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/tmpksOTRn/vers
    building 'capnp.lib.capnp' extension
    creating build/temp.macosx-10.9-x86_64-2.7
    creating build/temp.macosx-10.9-x86_64-2.7/capnp
    creating build/temp.macosx-10.9-x86_64-2.7/capnp/lib
    /usr/bin/gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -I/opt/X11/include -arch x86_64 -I. -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c capnp/lib/capnp.cpp -o build/temp.macosx-10.9-x86_64-2.7/capnp/lib/capnp.o --std=c++11
    In file included from capnp/lib/capnp.cpp:236:
    ./capnp/helpers/checkCompiler.h:9:10: fatal error: 'capnp/dynamic.h' file not found
    #include "capnp/dynamic.h"
             ^
    1 error generated.
    error: command '/usr/bin/gcc' failed with exit status 1

    ----------------------------------------
Command "/Users/felippe.gestec/.virtualenvs/heimdall/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/pip-build-LDNNAm/pycapnp/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/pip-_JorSQ-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/felippe.gestec/.virtualenvs/heimdall/bin/../include/site/python2.7/pycapnp" failed with error code 1 in /private/var/folders/gz/lc93rd8d2tlfrbm88gpm9rhh0000gp/T/pip-build-LDNNAm/pycapnp

@felippemr
Copy link

I was able to install nupic.bindings but received a diffenrent error when trying to install nupic....but I think that the problem with nupic.bindings was my pip version....

Thank you guys!

@rhyolight
Copy link
Member Author

Ok Great! The error you are getting is because you probably need to install XCode command line tools, which will give you a C compiler that is necessary for Capnproto installation. Then try starting from scratch again:

pip uninstall nupic nupic.bindings -y
pip install nupic --user

How does that work?

@felippemr
Copy link

(heimdall)(master *)heimdall$ python
Python 2.7.10 (default, Jul 13 2015, 12:05:58)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import nupic
>>>

@felippemr
Copy link

Thank you guys!

@rhyolight
Copy link
Member Author

😌 whew

@garikoitz
Copy link

Hi Matt, I've just seen that you closed this ticket but I am having the same problems as explained here. (OSX 10.10.5). I had the same problem:
Exception: Must provide nupic core release directory. --nupic-core-dir
with the previous version, and with 0.3.3 I have the same error as above:
nupic.bindings-0.2.1-cp27-none-macosx_10_9_intel.whl is not a supported wheel on this platform.
I did export MACOSX_DEPLOYMENT_TARGET=10.9 too, but it is not working.
I have the latest pip version.
I can send pip list output, but it is a little bit long since I have anaconda installed. Should I send it?

So this is the output if it is of help...

gari@GariMBP ~ $ pip uninstall nupic nupic.bindings -y
Cannot uninstall requirement nupic, not installed
gari@GariMBP ~ $ pip install nupic --user
Collecting nupic
  Using cached nupic-0.3.3-py2-none-any.whl
Collecting pytest-cov==1.6 (from nupic)
  Using cached pytest-cov-1.6.tar.gz
Requirement already satisfied (use --upgrade to upgrade): numpy==1.9.2 in ./anaconda/lib/python2.7/site-packages (from nupic)
Collecting asteval==0.9.1 (from nupic)
  Using cached asteval-0.9.1.tar.gz
Collecting prettytable==0.7.2 (from nupic)
  Using cached prettytable-0.7.2.tar.bz2
Collecting pytest==2.5.1 (from nupic)
  Using cached pytest-2.5.1.tar.gz
Collecting psutil==1.0.1 (from nupic)
  Using cached psutil-1.0.1.tar.gz
Collecting pycapnp==0.5.5 (from nupic)
  Using cached pycapnp-0.5.5.tar.gz
Collecting PyYAML==3.10 (from nupic)
  Using cached PyYAML-3.10.tar.gz
Collecting pytest-xdist==1.8 (from nupic)
  Using cached pytest-xdist-1.8.zip
Collecting unittest2==0.5.1 (from nupic)
  Using cached unittest2-0.5.1.tar.gz
Collecting pyproj==1.9.3 (from nupic)
  Using cached pyproj-1.9.3.tar.gz
Collecting DBUtils==1.1 (from nupic)
  Using cached DBUtils-1.1.tar.gz
Collecting python-dateutil==2.1 (from nupic)
  Using cached python-dateutil-2.1.tar.gz
Collecting pillow==2.3.0 (from nupic)
  Using cached Pillow-2.3.0.zip
Collecting nupic.bindings==0.2.1 (from nupic)
  Could not find a version that satisfies the requirement nupic.bindings==0.2.1 (from nupic) (from versions: )
  Some externally hosted files were ignored as access to them may be unreliable (use --allow-external nupic.bindings to allow).
No matching distribution found for nupic.bindings==0.2.1 (from nupic)

and

gari@GariMBP ~ $ pip install https://pypi.python.org/packages/cp27/n/nupic.bindings/nupic.bindings- 0.2.1-cp27-none-macosx_10_9_intel.whl --user
nupic.bindings-0.2.1-cp27-none-macosx_10_9_intel.whl is not a supported wheel on this platform.
gari@GariMBP ~ $ echo  $MACOSX_DEPLOYMENT_TARGET 
10.9
gari@GariMBP ~ $ pip --version
pip 7.1.2 from /Users/gari/anaconda/lib/python2.7/site-packages (python 2.7)

Any idea on what to test next? Maybe it can be some incompatibiliy with the anaconda environment?
thanks for your help

@rhyolight
Copy link
Member Author

Wow, it never ends!! 😝 I'm reopening and starting from scratch on OS X now to try to replicate this.

@rhyolight rhyolight reopened this Sep 23, 2015
@rhyolight
Copy link
Member Author

@garikoitz This is probably because you are running Anaconda python. There are many opportunities to confuse OS X when you have multiple versions of python installed on the system. I just ran through a complete install of NuPIC on a new OS X Yosemite with the system version of Python with no problems. However, I did notice this in your logs:

Some externally hosted files were ignored as access to them may be unreliable (use --allow-external nupic.bindings to allow).

Did you try that?

@garikoitz
Copy link

Thanks Matt!
I tried it now, and this is the log:

gari@GariMBP soft $ pip install nupic --allow-external nupic.bindings
Collecting nupic
  Using cached nupic-0.3.3-py2-none-any.whl
Collecting pytest-cov==1.6 (from nupic)
  Using cached pytest-cov-1.6.tar.gz
Requirement already satisfied (use --upgrade to upgrade): numpy==1.9.2 in /Users/gari/anaconda/lib/python2.7/site-packages (from nupic)
Collecting asteval==0.9.1 (from nupic)
  Using cached asteval-0.9.1.tar.gz
Collecting prettytable==0.7.2 (from nupic)
  Using cached prettytable-0.7.2.tar.bz2
Collecting pytest==2.5.1 (from nupic)
  Using cached pytest-2.5.1.tar.gz
Collecting psutil==1.0.1 (from nupic)
  Using cached psutil-1.0.1.tar.gz
Collecting pycapnp==0.5.5 (from nupic)
  Using cached pycapnp-0.5.5.tar.gz
Collecting PyYAML==3.10 (from nupic)
  Using cached PyYAML-3.10.tar.gz
Collecting pytest-xdist==1.8 (from nupic)
  Using cached pytest-xdist-1.8.zip
Collecting unittest2==0.5.1 (from nupic)
  Using cached unittest2-0.5.1.tar.gz
Collecting pyproj==1.9.3 (from nupic)
  Using cached pyproj-1.9.3.tar.gz
Collecting DBUtils==1.1 (from nupic)
  Using cached DBUtils-1.1.tar.gz
Collecting python-dateutil==2.1 (from nupic)
  Using cached python-dateutil-2.1.tar.gz
Collecting pillow==2.3.0 (from nupic)
  Using cached Pillow-2.3.0.zip
Collecting nupic.bindings==0.2.1 (from nupic)
  Could not find a version that satisfies the requirement nupic.bindings==0.2.1 (from nupic) (from versions: )
  Some externally hosted files were ignored as access to them may be unreliable (use --allow-external nupic.bindings to allow).
No matching distribution found for nupic.bindings==0.2.1 (from nupic)
gari@GariMBP soft $ pip install https://pypi.python.org/packages/cp27/n/nupic.bindings/nupic.bindings- 0.2.1-cp27-none-macosx_10_9_intel.whl --user --allow-external nupic.bindings
Requirement '0.2.1-cp27-none-macosx_10_9_intel.whl' looks like a filename, but the file does not exist
0.2.1-cp27-none-macosx_10_9_intel.whl is not a valid wheel filename.

Should I create another independent Python environment for nupic to work in? I am relying in anaconda por work related projects so I cannot remove it.
thanks again :)

@rhyolight
Copy link
Member Author

@garikoitz If pip cannot recognize your OS and download the right wheel file, you may need to compile yourself.

I'll be creating more tutorial videos over the next couple of days to show people how to build and install from scratch on OS X and Ubuntu.

@rcrowder
Copy link
Contributor

@garikoitz I notice a ' ' (space character) in the whl install (between nupic.bindings- and 0.2.1-cp27. Could you try installing the bingings again with out that space. Thanks

@garikoitz
Copy link

Hi Matt, I cloned the code and compiled it, and this is the end of the output:

byte-compiling build/bdist.macosx-10.5-x86_64/egg/nupic/swarming/permutationhelpers.py to permutationhelpers.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/nupic/swarming/permutations_runner.py to permutations_runner.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/nupic/swarming/regression.py to regression.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/nupic/test/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/nupic/test/abstract_temporal_memory_test.py to abstract_temporal_memory_test.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/nupic/test/test_framework_helpers.py to test_framework_helpers.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/nupic/utils.py to utils.pyc
creating build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying src/nupic.egg-info/PKG-INFO -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying src/nupic.egg-info/SOURCES.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying src/nupic.egg-info/dependency_links.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying src/nupic.egg-info/namespace_packages.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying src/nupic.egg-info/not-zip-safe -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying src/nupic.egg-info/requires.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying src/nupic.egg-info/top_level.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
creating 'dist/nupic-0.3.4.dev0-py2.7.egg' and adding 'build/bdist.macosx-10.5-x86_64/egg' to it
removing 'build/bdist.macosx-10.5-x86_64/egg' (and everything under it)
Processing nupic-0.3.4.dev0-py2.7.egg
removing '/Users/gari/.local/lib/python2.7/site-packages/nupic-0.3.4.dev0-py2.7.egg' (and everything under it)
creating /Users/gari/.local/lib/python2.7/site-packages/nupic-0.3.4.dev0-py2.7.egg
Extracting nupic-0.3.4.dev0-py2.7.egg to /Users/gari/.local/lib/python2.7/site-packages
nupic 0.3.4.dev0 is already the active version in easy-install.pth

Installed /Users/gari/.local/lib/python2.7/site-packages/nupic-0.3.4.dev0-py2.7.egg
Processing dependencies for nupic==0.3.4.dev0
Searching for nupic.bindings==0.2.1
Reading https://pypi.python.org/simple/nupic.bindings/
No local packages or download links found for nupic.bindings==0.2.1
error: Could not find suitable distribution for Requirement.parse('nupic.bindings==0.2.1')

Is it possible to install nupic.bindings in some other way?
thanks!
Gari

@garikoitz
Copy link

@rcrowder I think it was a formatting problem I had when I sent the code. In any case I did it again right now and this is the result:

gari@GariMBP soft $ pip install https://pypi.python.org/packages/cp27/n/nupic.bindings/nupic.bindings-0.2.1-cp27-none-macosx_10_9_intel.whl --user
nupic.bindings-0.2.1-cp27-none-macosx_10_9_intel.whl is not a supported wheel on this platform.

thanks for your help!

@rcrowder
Copy link
Contributor

@garikoitz Saw that space first in an email update, and thought it might have been something like that. Cheers.

@breznak
Copy link
Member

breznak commented Sep 24, 2015

@rhyolight sorry, didn't read the whole thread. Just an update: I didn't reproduce the must provide nupic-core-dir error anymore.

@garikoitz
Copy link

Hi again, I've tried everything but nupic.bindings installation is not working for me.
Are there instructions to compile them manually?
Thanks again

@oxtopus
Copy link
Contributor

oxtopus commented Sep 24, 2015

The instructions in the nupic.core README are current.

I'm working on a document for developer setup at https://github.com/numenta/nupic/wiki/Numenta-Developer-Setup. It's specific to Numenta employees, and makes assumptions about your environment that may not be true, but it may be helpful. You should also be able to use the example in https://github.com/oxtopus/nupic-build-matrix/blob/master/ubuntu/14.04/clang/Dockerfile#L38-L58

@rhyolight
Copy link
Member Author

I think that numenta/nupic-legacy#2603 captures this issue better, so I am closing this.

@rhyolight
Copy link
Member Author

@garikoitz This video might help you install from source code: https://www.youtube.com/watch?v=sxb_Zrls7Zs

@garikoitz
Copy link

Thanks!
I could follow exactly the same steps as you did, and it seems that
everything is installed right now (I could import the nupic.bindings and
nupic in python), BUT:
1.- Now I have a problem with pytest:

gari@GariMBP* nupic $* ./scripts/run_nupic_tests -u

=============================================================================
test session starts
=============================================================================

platform darwin -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.2 --
/Users/gari/anaconda/bin/python

rootdir: /Users/gari/soft/nupic, inifile:

plugins: cov, xdist

collecting 6 items__gari@GariMBP* nupic $*

gari@GariMBP* nupic $*

gari@GariMBP* nupic $*

gari@GariMBP* nupic $*

gari@GariMBP* nupic $* pip install pytest

Requirement already satisfied (use --upgrade to upgrade): pytest in
/Users/gari/anaconda/lib/python2.7/site-packages

Requirement already satisfied (use --upgrade to upgrade): py>=1.4.29 in
/Users/gari/anaconda/lib/python2.7/site-packages (from pytest)

gari@GariMBP* nupic $* pip install --upgrade pytest

Collecting pytest

Using cached pytest-2.8.0-py2.py3-none-any.whl

Requirement already up-to-date: py>=1.4.29 in
/Users/gari/anaconda/lib/python2.7/site-packages (from pytest)

Installing collected packages: pytest

Found existing installation: pytest 2.7.2

Uninstalling pytest-2.7.2:

  Successfully uninstalled pytest-2.7.2

Successfully installed pytest-2.8.0

gari@GariMBP* nupic $* ./scripts/run_nupic_tests -u

INTERNALERROR> Traceback (most recent call last):

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/main.py", line
86, in wrap_session

INTERNALERROR> config._do_configure()

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/config.py", line
829, in _do_configure

INTERNALERROR>
self.hook.pytest_configure.call_historic(kwargs=dict(config=self))

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 729, in call_historic

INTERNALERROR> self._hookexec(self, self._nonwrappers + self._wrappers,
kwargs)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 338, in _hookexec

INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 333, in

INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 596, in execute

INTERNALERROR> res = hook_impl.function(*args)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/xdist/plugin.py", line
60, in pytest_configure

INTERNALERROR> multicall.execute()

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 596, in execute

INTERNALERROR> res = hook_impl.function(*args)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/terminal.py",
line 47, in pytest_configure

INTERNALERROR> config.pluginmanager.register(reporter,
'terminalreporter')

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/config.py", line
206, in register

INTERNALERROR> ret = super(PytestPluginManager, self).register(plugin,
name)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 370, in register

INTERNALERROR> hook._maybe_apply_history(hookimpl)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 747, in _maybe_apply_history

INTERNALERROR> res = self._hookexec(self, [method], kwargs)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 338, in _hookexec

INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 333, in

INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()

INTERNALERROR> File
"/Users/gari/anaconda/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py",
line 593, in execute

INTERNALERROR> args = [all_kwargs[argname] for argname in
hook_impl.argnames]

INTERNALERROR> KeyError: 'nodeid'

gari@GariMBP* nupic $*

*I've tried with two different versions of pytest, but I got two different
error, not working. It seems that they are working on it. *

pytest-dev/pytest#1034
pytest-dev/pytest#1034

2.- But then, if the problem was only the testing, I would expect hotgym
to work, but it doesn't either (it seems a different problem):

gari@GariMBP* simple $* ./hotgym.py

Fatal Python error: PyThreadState_Get: no current thread

Abort trap: 6

What do you recommend? I am thinking in creating a separate python
environment and trying to compile everything again...

thanks!

*Gari *

On Thu, Sep 24, 2015 at 2:13 PM Matthew Taylor notifications@github.com
wrote:

@garikoitz https://github.com/garikoitz This video might help you
install from source code: https://www.youtube.com/watch?v=sxb_Zrls7Zs


Reply to this email directly or view it on GitHub
#619 (comment).

@rhyolight
Copy link
Member Author

@garikoitz I had problems with py.test as well. The solution for me was to ensure that there was no system-level py.test installed (if you installed with sudo, then uninstall it with sudo). NuPIC provides its own version of py.test, and that is the version you should be using. The py.test executable file is installed as well, you just need to find it and put it on your path. See the video at 24m50s.

@garikoitz
Copy link

thanks! I didn't know that, will test it and let you know.

On Fri, Sep 25, 2015 at 8:32 AM Matthew Taylor notifications@github.com
wrote:

@garikoitz https://github.com/garikoitz I had problems with py.test as
well. The solution for me was to ensure that there was no system-level
py.test installed (if you installed with sudo, then uninstall it with
sudo). NuPIC provides its own version of py.test, and that is the version
you should be using. The py.test executable file is installed as well, you
just need to find it and put it on your path. See the video at 24m50s
https://youtu.be/sxb_Zrls7Zs?t=24m50s.


Reply to this email directly or view it on GitHub
#619 (comment).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants