Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scipy.weave.inline and Microsoft Visual C++: Generated filenames are too long. #3216

Closed
vanpact opened this issue Jan 15, 2014 · 6 comments
Closed
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected
Milestone

Comments

@vanpact
Copy link
Contributor

vanpact commented Jan 15, 2014

The function inline() from scipy.weave store temporarily the compiled files with a filename consisting mainly of a sha256 hash. However Microsoft Visual C++ supports only filename of less than 258 characters including the extension for compilation. Hence the compilation fail. ( https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/2156195-fix-260-character-file-name-length-limitation ).

I can think of two easy solutions to fix the problem:

  1. Trunk the hash by changin the line
    catalog.py:95: return base + sha256(expr).hexdigest()
    to: return base + sha256(expr).hexdigest()[:100]

  2. Use another hash (md5?sha-1?sha-224?sha-3 224bits?)

@rgommers
Copy link
Member

Hmm not good. Using sha-224 looks a little cleaner than truncation the output of sha-256, but whatever works I guess. @pv @TomasTomecek any preference for a solution?

@TomasTomecek
Copy link
Contributor

I was talking to some security guys and they are fine with truncating those hashes (I'm not really a security person, wasn't sure about collisions and stuff).

I'm also fine with sha-224 since it's FIPS approved: link to pdf

@pv
Copy link
Member

pv commented Jan 16, 2014

Let's just truncate the hash in the file names to a sane length (the same length for all platforms so that we avoid an unnecessary platform dependency).

@vanpact
Copy link
Contributor Author

vanpact commented Jan 16, 2014

@juliantaylor
Copy link
Contributor

no security evaluations need to be made here, the reason this was changed is to silence a bogus warning on fips pedantic systems about the mere use of md5.
md5 is sufficient as a hash key as used in this context, it is not a digital signature nor encryption.

as sha256 is a cryptographic hash it should be uniform in its distribution so we are free to truncate as much as we see fit to have acceptable low collisions.

rgommers added a commit to rgommers/scipy that referenced this issue Jan 22, 2014
…gh-3216.

This issue was introduced in 021e0ee.  It's MSVC specific; MSVC
has a 260 char file name (including full path).  So truncation the
has to 50 chars, plus filename prefix (``sc_`` or ``compiler_``), leaves
200 chars for the path which should be enough.
rgommers added a commit to rgommers/scipy that referenced this issue Jan 22, 2014
…gh-3216.

This issue was introduced in 021e0ee.  It's MSVC specific; MSVC
has a 260 char file name (including full path).  So truncation the
has to 50 chars, plus filename prefix (``sc_`` or ``compiler_``), leaves
200 chars for the path which should be enough.
@rgommers
Copy link
Member

Fix in gh-3232.

rgommers added a commit that referenced this issue Jan 25, 2014
BUG: fix weave issue with too long file names with MSVC.  Closes gh-3216...
rgommers added a commit that referenced this issue Jan 25, 2014
This issue was introduced in 021e0ee.  It's MSVC specific; MSVC
has a 260 char file name (including full path).  So truncation the
has to 50 chars, plus filename prefix (``sc_`` or ``compiler_``), leaves
200 chars for the path which should be enough.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected
Projects
None yet
Development

No branches or pull requests

5 participants