package_data is missing for linux wheels

[rhyolight]

You can see this on our regression tests:

IOError: [Errno 2] No such file or directory: '/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/nupic/swarming/jsonschema/jobParamsSchema.json'

The missing json file is declared in package_data, but not included in the wheel. Wheels built on OS X seem to have all the package_data files.

The script that creates the wheel and uploads it to S3 is here. The wheel command being used to package is pip wheel --wheel-dir=dist/wheels ..

[rhyolight]

After investigating on Ubuntu, I've found that wheels I'm creating are missing the following package_data:

    package_data = {
      "nupic.support.resources.images": ["*.png", "*.gif",
                                         "*.ico", "*.graffle"],
      "nupic.swarming.jsonschema": ["*.json"]
    },

Strangely, the rest of the package_data is included in the wheel.

[rhyolight]

@oxtopus Can you think of any reason why some package_data would be missing on one platform and not another?

[rhyolight]

@oxtopus Never mind. I figured it out. It works if I add an empty nupic/swarming/jsonschema/__init__.py.

[vitaly-krugl]

@rhyolight, was it possibly missing from MANIFEST.in? We shouldn't have to add __init__.py files in data directories. Also see #2621

[oxtopus]

On the subject of MANIFEST.in and superfluous package_data, according to https://docs.python.org/3.4/distutils/sourcedist.html#specifying-the-files-to-distribute:

If you do supply your own MANIFEST, you must specify everything: the default set of files described above does not apply in this case.

I interpret that to mean that if you use MANIFEST.in (which we do), then it doesn't matter what is in package_data. The files need to be added in MANIFEST.in. In which case, we can remove package_data from setup.py.

[rhyolight]

I find it all immensely confusing. 😕