New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using an out of place build directory with src paths like "../../../src/foo.pyx" doesn't work #2764

Open
telamonian opened this Issue Dec 17, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@telamonian
Copy link
Contributor

telamonian commented Dec 17, 2018

As per this Stackoverflow question, say you have a setup.py that looks like this:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

import numpy as np

extensions = [
    Extension(
        'boolFunc',
        ['../../../src/boolFunc.pyx'],
        include_dirs=[np.get_include()],
    )]

setup(
    ext_modules=cythonize(extensions,
                          build_dir='build_dir'),
)

When you try to build the Cython extension with, say, the following call:

python setup.py build_ext -b /Users/foo/proj/lib -t .

The cythonize routine will try to build the path to the boolFunc.c file as:

./build_dir/../../../src/boolFunc.c

and the path to boolFunc.o will be built similarly.

At this point, insanity breaks loose. Best case scenario, setup.py crashes. Worst case, instead of saving the the temporary .c and .o files to the specified build_dir, they end up in a semi-random (seeming) place on the filesystem. Actually, I think I was getting both things happening (the .c file would be created somewhere weird, and then setup.py would crash).

@scoder

This comment has been minimized.

Copy link
Contributor

scoder commented Dec 21, 2018

IMHO, insanity already starts when people use paths like '../../../src/boolFunc.pyx' as their sources.
I don't think there is a correct/safe way to handle such a case. People can happily refer to their sources from src/, ../src/, ../../src/ etc., and then complain that Cython shovels them all in the same build directory and weird things start happening. Would that be Cython's fault? I don't think so.

I agree that the PR improves the specifically described situation. It does not solve the general problem that people come up with weird build setups and expect the tools to magically make them work.

@telamonian

This comment has been minimized.

Copy link
Contributor

telamonian commented Dec 21, 2018

As they say, there's no accounting for taste. And I completely agree that the most general case is hopeless. Which is exactly why "weird build" functionality should be added in on the basis of reasonable(-ish) use cases. This is also why you should completely ignore the perverse ones (I thought of the src/, ../src/, ../../src/ etc. case too as I was writing the PR. I'm gonna say that one's their own damn fault).

An "up-and-to-the-left-of-the-source" build is obviously not ideal, but I've run into similar situations myself. They usually involve in-house code that's needed as a dependency by 3 or 4 other in-house projects. I suppose the "correct" way to deal with those situations is something like Git submodule, but honestly that's not really that great a solution either. So I sympathize with the guy who asked about this on SO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment