BUILD from sdist broken if cython available #1606

Closed
josef-pkt opened this Issue Apr 19, 2014 · 10 comments

Projects

None yet

2 participants

@josef-pkt
Member

I think there is a mistake in setup.py

the sdist doesn't distribute .pyx files anymore
However, if the install python has cython available, then it checks for .pyx files not for .c files.

File "c:\users\josef\appdata\local\temp\pip-cak2dv-build\setup.py", line 323, in check_cython_extensions
    """ % src)

Exception: Cython-generated file 'statsmodels\nonparametric/_smoothers_lowess.pyx' not found.

        Cython is required to compile statsmodels from a development branch.
        Please install Cython or download a source release of statsmodels.
@rgommers
Member

The .pyx files should still be included in the sdist output; source releases should contain all sources. The generated C should be shipped in addition to, not instead of pyx.

I suggest to replace the custom Cython invokation in setup.py by tools/cythonize from scipy.

@josef-pkt
Member

we removed the .pyx files from the sdist in #1200

@rgommers
Member

Ah, missed that PR. That was just the quickest way right, not a principled decision? I'd think you want to keep them in there.

@josef-pkt
Member

Thinking about it, we should revisit #1200 and #1195
we don't want to install the .pyx, but we can still include them in the sdist.
keep them out of manifest.in, but add them to package_data
I don't see why they are not included as sources

@rgommers
Member

There's no need to add them to package_data, just don't exclude them in MANIFEST.in and use tools/cythonize.py to use C files unless the pyx timestamp says it has changes in it. That's also useful for the dev version.

@josef-pkt
Member

If they are in MANIFEST.in, then they will also be installed. If they are only in package data, then they will be in the sdist but will not be installed, IIRC.

@rgommers
Member

That's the wrong way around. package_data includes files in MANIFEST plus installs them: https://docs.python.org/3/distutils/setupscript.html#distutils-installing-package-data

@rgommers
Member

I'm a big fan of initiatives like "bug week"! You want me to have a go at this one? I'll make sure to double check the install behavior of what I said above.

@josef-pkt
Member

Please do

The way I see it:
fix the pyx check
add .pyx to sdist but not to installed directories
or both

I guess I mixed up MANIFEST.in and package_data. I never understood the python documentation for this, and still don't, and always only succeeded with lots of trial end error.

@rgommers rgommers added a commit to rgommers/statsmodels that referenced this issue May 11, 2014
@rgommers rgommers BLD: fix use of setuptools include_package_data. Closes gh-1606. 173c1d3
@rgommers
Member

OK bug week is over but I got something done. Issue was some setuptools-specific behavior described in https://pythonhosted.org/setuptools/setuptools.html#including-data-files.

@rgommers rgommers closed this in 1c110ef May 22, 2014
@PierreBdR PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this issue Sep 2, 2014
@rgommers rgommers BLD: fix use of setuptools include_package_data. Closes gh-1606. 45efc41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment