Default NuPIC configuration is not found

[rhyolight]

After the pip merge, it seems that the default NuPIC config files aren't being used when imported from python. So you get errors like this:

ValueError: Invalid classifier implementation (%s). Value must be "py" or "cpp".

When creating models. The easy fix is to set the NTA_DATA_PATH env var:

export NTA_DATA_PATH=$NUPIC/resources

We don't want to force users to do this, we a part of the pip installation needs to include a way to get this configuration file in the right place, and NuPIC using it automatically.

[rhyolight]

[11:33:36] synaesthetik rhyolight: i'm getting an error
[11:33:49] synaesthetik ValueError: Invalid classifier implementation (%s). Value must be "py" or "cpp".
[11:44:06] synaesthetik do the bins need to be .py
[11:44:10] synaesthetik i don't get it
[11:56:04] rhyolight synaesthetik: I recognize that.... hmm...
[11:56:25] rhyolight you're getting that on the run.py script?
[12:07:15] synaesthetik rhyolight: yeah
[12:34:16] rhyolight synaesthetik: can you pastebin the whole command and stdout?
[13:23:07] synaesthetik http://bpaste.net/show/472792/
[13:23:52] rhyolight synaesthetik: I think I know the problem, one I've had before
[13:24:03] rhyolight The problem is that the nupic default configuration is not being used
[13:24:11] synaesthetik oh
[13:24:12] rhyolight I need to file a bug for it.
[13:24:34] synaesthetik how do I use the default config
[13:24:34] rhyolight it's a problem with the pip install, it will be a problem for most people, probably.
[13:25:36] rhyolight quick fix (probably)... set NTA_DATA_PATH=$NUPIC/resources
[13:26:00] rhyolight The NTA_DATA_PATH environment variable needs to point to that resources directory
[13:26:21] synaesthetik should I add that to .bash_profile
[13:27:36] rhyolight yeah for now
[13:35:51] synaesthetik ahh that worked rhyolight
[13:36:03] rhyolight great

[rhyolight]

[11:33:36] synaesthetik rhyolight: i'm getting an error
[11:33:49] synaesthetik ValueError: Invalid classifier implementation (%s). Value must be "py" or "cpp".
[11:44:06] synaesthetik do the bins need to be .py
[11:44:10] synaesthetik i don't get it
[11:56:04] rhyolight synaesthetik: I recognize that.... hmm...
[11:56:25] rhyolight you're getting that on the run.py script?
[12:07:15] synaesthetik rhyolight: yeah
[12:34:16] rhyolight synaesthetik: can you pastebin the whole command and stdout?
[13:23:07] synaesthetik http://bpaste.net/show/472792/
[13:23:52] rhyolight synaesthetik: I think I know the problem, one I've had before
[13:24:03] rhyolight The problem is that the nupic default configuration is not being used
[13:24:11] synaesthetik oh
[13:24:12] rhyolight I need to file a bug for it.
[13:24:34] synaesthetik how do I use the default config
[13:24:34] rhyolight it's a problem with the pip install, it will be a problem for most people, probably.
[13:25:36] rhyolight quick fix (probably)... set NTA_DATA_PATH=$NUPIC/resources
[13:26:00] rhyolight The NTA_DATA_PATH environment variable needs to point to that resources directory
[13:26:21] synaesthetik should I add that to .bash_profile
[13:27:36] rhyolight yeah for now
[13:35:51] synaesthetik ahh that worked rhyolight
[13:36:03] rhyolight great

[david-ragazzi]

The problem is that /config folder is not included in the nupic package and how you noted someone has to download the repo and set NTA_DATA_PATH to it.

The solution would be install this folder with the nupic package (e.g. \nupic\config) and if NTA_DATA_PATH is empty, we use this default configuration that will be located in the default python location.

[david-ragazzi]

The problem is that /config folder is not included in the nupic package and how you noted someone has to download the repo and set NTA_DATA_PATH to it.

The solution would be install this folder with the nupic package (e.g. \nupic\config) and if NTA_DATA_PATH is empty, we use this default configuration that will be located in the default python location.

[rhyolight]

I think we can assume that NTA_DATA_PATH will be empty, so we only need to include the config with the python package then?

[rhyolight]

I think we can assume that NTA_DATA_PATH will be empty, so we only need to include the config with the python package then?

[david-ragazzi]

opsss... I'm confusing the things.. Actually I mean NTA_CONF_PATH.. Forget NTA_DATA_PATH.. NTA_CONF_PATH is the variable where it is supposed we find config files..

[david-ragazzi]

opsss... I'm confusing the things.. Actually I mean NTA_CONF_PATH.. Forget NTA_DATA_PATH.. NTA_CONF_PATH is the variable where it is supposed we find config files..

[rhyolight]

Well, it is the NTA_DATA_PATH that fixes the problem. I remember having a
similar issue when setting up the regression tests.

---

Matt Taylor
OS Community Flag-Bearer
Numenta

On Thu, Jul 17, 2014 at 2:55 PM, David Ragazzi notifications@github.com
wrote:

opsss... I'm confusing the things.. Actually the dir I mean is
NTA_CONF_PATH

—
Reply to this email directly or view it on GitHub
#1122 (comment).

[rhyolight]

Well, it is the NTA_DATA_PATH that fixes the problem. I remember having a
similar issue when setting up the regression tests.

---

Matt Taylor
OS Community Flag-Bearer
Numenta

On Thu, Jul 17, 2014 at 2:55 PM, David Ragazzi notifications@github.com
wrote:

opsss... I'm confusing the things.. Actually the dir I mean is
NTA_CONF_PATH

—
Reply to this email directly or view it on GitHub
#1122 (comment).

[david-ragazzi]

This is really weird... Problems related to default configuration files that happened to me usually were related to NTA_CONF_DATA

[david-ragazzi]

This is really weird... Problems related to default configuration files that happened to me usually were related to NTA_CONF_DATA

[rhyolight]

I'm putting in #1124 as a temporary measure, so at least people will be able to run if they read the instructions.

[rhyolight]

I'm putting in #1124 as a temporary measure, so at least people will be able to run if they read the instructions.

[rhyolight]

Seems like NTA_DATA_PATH should be getting set in CMakeLists.txt:

set_environment_variable(NTA_DATA_PATH "${REPOSITORY_DIR}/resources" OFF)

[rhyolight]

Seems like NTA_DATA_PATH should be getting set in CMakeLists.txt:

set_environment_variable(NTA_DATA_PATH "${REPOSITORY_DIR}/resources" OFF)

[rhyolight]

Initially, my NTA_DATA_PATH was set to $NUPIC/resources, but I don't have it in my bash config files, so not sure where it is coming from. Every time I start a new shell, NTA_DATA_PATH is being set again. I can't figure out how it's getting set yet.

So, I just did unset NTA_DATA_PATH, then ran python setup.py install. After the install, NTA_DATA_PATH was still unset. Then I ran nupic on a sample experiment with NTA_DATA_PATH unset, and it ran fine!

I am wondering if the user who reported this problem had NTA_DATA_PATH set from a previous installation, and it was pointing to the wrong place?

[rhyolight]

Initially, my NTA_DATA_PATH was set to $NUPIC/resources, but I don't have it in my bash config files, so not sure where it is coming from. Every time I start a new shell, NTA_DATA_PATH is being set again. I can't figure out how it's getting set yet.

So, I just did unset NTA_DATA_PATH, then ran python setup.py install. After the install, NTA_DATA_PATH was still unset. Then I ran nupic on a sample experiment with NTA_DATA_PATH unset, and it ran fine!

I am wondering if the user who reported this problem had NTA_DATA_PATH set from a previous installation, and it was pointing to the wrong place?

[ChrisRJernigan]

I know this probably seems really obvious but I had to remember to set $NUPIC before I set $NTA_DATA_PATH since I was starting with a clean Mavericks install and the environment variable instructions have been removed from the readme. I just added this to my .bash_profile (and remember to source .bash_profile after you edit it):

export NUPIC=<path to nupic directory>
export NTA_DATA_PATH=$NUPIC/resources

[ChrisRJernigan]

I know this probably seems really obvious but I had to remember to set $NUPIC before I set $NTA_DATA_PATH since I was starting with a clean Mavericks install and the environment variable instructions have been removed from the readme. I just added this to my .bash_profile (and remember to source .bash_profile after you edit it):

export NUPIC=<path to nupic directory>
export NTA_DATA_PATH=$NUPIC/resources

[rhyolight]

the environment variable instructions have been removed from the readme

I added them back yesterday.

Trying to replicate this error in another OS X account right now.

[rhyolight]

the environment variable instructions have been removed from the readme

I added them back yesterday.

Trying to replicate this error in another OS X account right now.

[ChrisRJernigan]

I've tried that before with some success.

[ChrisRJernigan]

I've tried that before with some success.

[rhyolight]

Strangely, I am not having this issue on a fresh OS X account. There is no NTA_DATA_PATH env var set at all, before or after installation. It builds and runs file. I've run swarms, created models, and it all works. I'm not sure why @ChrisRJernigan's installation is in a bad state. @ChrisRJernigan, can you try to unset NTA_DATA_PATH and make sure no rc files are setting it, then try to run nupic again? I'd like to see if it works or not.

It is also suspicious that this issue didn't fail on Travis-CI when it ran the sample experiments. This may be a red herring.

[rhyolight]

Strangely, I am not having this issue on a fresh OS X account. There is no NTA_DATA_PATH env var set at all, before or after installation. It builds and runs file. I've run swarms, created models, and it all works. I'm not sure why @ChrisRJernigan's installation is in a bad state. @ChrisRJernigan, can you try to unset NTA_DATA_PATH and make sure no rc files are setting it, then try to run nupic again? I'd like to see if it works or not.

It is also suspicious that this issue didn't fail on Travis-CI when it ran the sample experiments. This may be a red herring.

[ChrisRJernigan]

I think I figured it out. It was because I had forgotten to set $NUPIC.
It still works when I unset NTA_DATA_PATH
But then once I unset $NUPIC I get the ValueError: Invalid classifier implementation (%s). Value must be "py" or "cpp" error

[ChrisRJernigan]

I think I figured it out. It was because I had forgotten to set $NUPIC.
It still works when I unset NTA_DATA_PATH
But then once I unset $NUPIC I get the ValueError: Invalid classifier implementation (%s). Value must be "py" or "cpp" error

[ChrisRJernigan]

Haha so this whole time we're thinking it was fixed when I set NTA_DATA_PATH because at the same time I had just set $NUPIC

This issue should be renamed as OOPS NEVERMIND

[ChrisRJernigan]

Haha so this whole time we're thinking it was fixed when I set NTA_DATA_PATH because at the same time I had just set $NUPIC

This issue should be renamed as OOPS NEVERMIND

[cogmission]

Matt,

I successfully installed Nupic a couple of weeks ago on my laptop, I tried to do it here at work and I have the following problems. (All OSX Mavericks related)

1. The instructions jump you around too much. There needs to be a single page where you start at the top, and proceed to the bottom.

2. I started here: https://github.com/numenta/nupic

   From here under the section entitled: Installation there is another section directly under it entitled “Installing and Building NuPIC”.

   So then I “Get the code” using the git command listed there...

   Under that listing there are “git” specific instructions and the a section entitled “NuPIC Installation” under which it tells you to first see the “installation instructions” <— lol

   So then I go back to the original page… ?

   Nothing told me to click on nupic-darwin64, but I happened to know I needed it. So I cloned it…
   

   Now I have nupic, nupic-darwin64, pip, brew and at this point I now do what?

   So now I try running the pip command (my version is 1.5.6 so I run the latter)

pip install --allow-all-external --allow-unverified PIL --allow-unverified psutil -r external/common/requirements.txt

which fails with…    ImportError: cannot import name getstatusoutput

David

On Jul 18, 2014, at 2:52 PM, Matthew Taylor notifications@github.com wrote:

Strangely, I am not having this issue on a fresh OS X account. There is no NTA_DATA_PATH env var set at all, before or after installation. It builds and runs file. I've run swarms, created models, and it all works. I'm not sure why @ChrisRJernigan's installation is in a bad state. @ChrisRJernigan, can you try to unset NTA_DATA_PATH and make sure no rc files are setting it, then try to run nupic again? I'd like to see if it works or not.

It is also suspicious that this issue didn't fail on Travis-CI when it ran the sample experiments. This may be a red herring.

—
Reply to this email directly or view it on GitHub.

[cogmission]

Matt,

I successfully installed Nupic a couple of weeks ago on my laptop, I tried to do it here at work and I have the following problems. (All OSX Mavericks related)

1. The instructions jump you around too much. There needs to be a single page where you start at the top, and proceed to the bottom.

2. I started here: https://github.com/numenta/nupic

   From here under the section entitled: Installation there is another section directly under it entitled “Installing and Building NuPIC”.

   So then I “Get the code” using the git command listed there...

   Under that listing there are “git” specific instructions and the a section entitled “NuPIC Installation” under which it tells you to first see the “installation instructions” <— lol

   So then I go back to the original page… ?

   Nothing told me to click on nupic-darwin64, but I happened to know I needed it. So I cloned it…
   

   Now I have nupic, nupic-darwin64, pip, brew and at this point I now do what?

   So now I try running the pip command (my version is 1.5.6 so I run the latter)

pip install --allow-all-external --allow-unverified PIL --allow-unverified psutil -r external/common/requirements.txt

which fails with…    ImportError: cannot import name getstatusoutput

David

On Jul 18, 2014, at 2:52 PM, Matthew Taylor notifications@github.com wrote:

Strangely, I am not having this issue on a fresh OS X account. There is no NTA_DATA_PATH env var set at all, before or after installation. It builds and runs file. I've run swarms, created models, and it all works. I'm not sure why @ChrisRJernigan's installation is in a bad state. @ChrisRJernigan, can you try to unset NTA_DATA_PATH and make sure no rc files are setting it, then try to run nupic again? I'd like to see if it works or not.

It is also suspicious that this issue didn't fail on Travis-CI when it ran the sample experiments. This may be a red herring.

—
Reply to this email directly or view it on GitHub.

[ChrisRJernigan]

David you don't need nupic-darwin64 if you are running
pip install -r external/common/requirements.txt

Seems redundant.

[ChrisRJernigan]

David you don't need nupic-darwin64 if you are running
pip install -r external/common/requirements.txt

Seems redundant.

[rhyolight]

@ChrisRJernigan Yes, it seems this issue can be closed. Better safe than sorry. I'll create another PR to revert the README now before I close it.

@cogmission Thanks for the feedback. I can probably simplify this a lot now that pip works.

[rhyolight]

@ChrisRJernigan Yes, it seems this issue can be closed. Better safe than sorry. I'll create another PR to revert the README now before I close it.

@cogmission Thanks for the feedback. I can probably simplify this a lot now that pip works.

[cogmission]

Yes that’s all honkey/dory but how do I solve my problem? :P

lol,

David

On Jul 18, 2014, at 3:41 PM, ChrisRJernigan notifications@github.com wrote:

David you don't need nupic-darwin64 if you just run pip install -r external/common/requirements.txt

—
Reply to this email directly or view it on GitHub.

[cogmission]

Yes that’s all honkey/dory but how do I solve my problem? :P

lol,

David

On Jul 18, 2014, at 3:41 PM, ChrisRJernigan notifications@github.com wrote:

David you don't need nupic-darwin64 if you just run pip install -r external/common/requirements.txt

—
Reply to this email directly or view it on GitHub.

[ChrisRJernigan]

David maybe try using sudo
I think I remember using it on that command

[ChrisRJernigan]

David maybe try using sudo
I think I remember using it on that command

[cogmission]

Ok, now I get: RuntimeError: Broken toolchain: cannot link a simple C program

On Jul 18, 2014, at 3:48 PM, ChrisRJernigan notifications@github.com wrote:

David maybe try using sudo

—
Reply to this email directly or view it on GitHub.

[cogmission]

Ok, now I get: RuntimeError: Broken toolchain: cannot link a simple C program

On Jul 18, 2014, at 3:48 PM, ChrisRJernigan notifications@github.com wrote:

David maybe try using sudo

—
Reply to this email directly or view it on GitHub.

[cogmission]

I have Xcode 5.1.1 and the command line tools installed?

On Jul 18, 2014, at 3:50 PM, David Ray david.ray@barchart.com wrote:

Ok, now I get: RuntimeError: Broken toolchain: cannot link a simple C program

On Jul 18, 2014, at 3:48 PM, ChrisRJernigan notifications@github.com wrote:

David maybe try using sudo

—
Reply to this email directly or view it on GitHub.

[cogmission]

I have Xcode 5.1.1 and the command line tools installed?

On Jul 18, 2014, at 3:50 PM, David Ray david.ray@barchart.com wrote:

Ok, now I get: RuntimeError: Broken toolchain: cannot link a simple C program

On Jul 18, 2014, at 3:48 PM, ChrisRJernigan notifications@github.com wrote:

David maybe try using sudo

—
Reply to this email directly or view it on GitHub.

[rhyolight]

@cogmission Can you retrace your steps and paste the commands and resuting stdout into an email to nupic-discuss? I would try not to use sudo.

[rhyolight]

@cogmission Can you retrace your steps and paste the commands and resuting stdout into an email to nupic-discuss? I would try not to use sudo.

[cogmission]

Ok…

On Jul 18, 2014, at 3:51 PM, Matthew Taylor notifications@github.com wrote:

@cogmission Can you retrace your steps and paste the commands and resuting stdout into an email to nupic-discuss? I would try not to use sudo.

—
Reply to this email directly or view it on GitHub.

[cogmission]

Ok…

On Jul 18, 2014, at 3:51 PM, Matthew Taylor notifications@github.com wrote:

@cogmission Can you retrace your steps and paste the commands and resuting stdout into an email to nupic-discuss? I would try not to use sudo.

—
Reply to this email directly or view it on GitHub.

[rhyolight]

Sorry to direct you somewhere else, but this not the best bad place to have
this discussion.

---

Matt Taylor
OS Community Flag-Bearer
Numenta

On Fri, Jul 18, 2014 at 1:53 PM, David Ray notifications@github.com wrote:

Ok…

On Jul 18, 2014, at 3:51 PM, Matthew Taylor notifications@github.com
wrote:

@cogmission Can you retrace your steps and paste the commands and
resuting stdout into an email to nupic-discuss? I would try not to use
sudo.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#1122 (comment).

[rhyolight]

Sorry to direct you somewhere else, but this not the best bad place to have
this discussion.

---

Matt Taylor
OS Community Flag-Bearer
Numenta

On Fri, Jul 18, 2014 at 1:53 PM, David Ray notifications@github.com wrote:

Ok…

On Jul 18, 2014, at 3:51 PM, Matthew Taylor notifications@github.com
wrote:

@cogmission Can you retrace your steps and paste the commands and
resuting stdout into an email to nupic-discuss? I would try not to use
sudo.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#1122 (comment).

[cogmission]

No problem at all…

but where should I start from?

David

On Jul 18, 2014, at 3:54 PM, Matthew Taylor notifications@github.com wrote:

Sorry to direct you somewhere else, but this not the best bad place to have
this discussion.

---

Matt Taylor
OS Community Flag-Bearer
Numenta

On Fri, Jul 18, 2014 at 1:53 PM, David Ray notifications@github.com wrote:

Ok…

On Jul 18, 2014, at 3:51 PM, Matthew Taylor notifications@github.com
wrote:

@cogmission Can you retrace your steps and paste the commands and
resuting stdout into an email to nupic-discuss? I would try not to use
sudo.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#1122 (comment).

—
Reply to this email directly or view it on GitHub.

[cogmission]

No problem at all…

but where should I start from?

David

On Jul 18, 2014, at 3:54 PM, Matthew Taylor notifications@github.com wrote:

Sorry to direct you somewhere else, but this not the best bad place to have
this discussion.

---

Matt Taylor
OS Community Flag-Bearer
Numenta

On Fri, Jul 18, 2014 at 1:53 PM, David Ray notifications@github.com wrote:

Ok…

On Jul 18, 2014, at 3:51 PM, Matthew Taylor notifications@github.com
wrote:

@cogmission Can you retrace your steps and paste the commands and
resuting stdout into an email to nupic-discuss? I would try not to use
sudo.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#1122 (comment).

—
Reply to this email directly or view it on GitHub.

[rhyolight]

@cogmission let's assume you've cloned the repo, so start with the first step in the README installation instructions?

[rhyolight]

@cogmission let's assume you've cloned the repo, so start with the first step in the README installation instructions?