Workaround for windows file path size limit issue

[isuruf]

Adding --no-build-id would shorten the file path and therefore enable
boost builds and others that have really long file paths

[shadowwalkersb]

How about, also using --croot option to set the build root directory to the shortest path possible, maybe C:\?

[isuruf]

Thanks. I'll make the change

[jakirkham]

Sorry I think I'm missing some context. What is the big picture here?

[isuruf]

See this issue, conda-forge/boost-feedstock#27

the build dir changed from conda-bld/work from conda-bld/recipe_[unixtime] which is much longer, got the same problem for another recipe

With the changes in conda-build 2.x, unpacking the boost tarball fails as the path size becomes greater than 260 chars which is the limit for a file path size on windows. This PR reduces C:\\Miniconda-x64\\conda-bld\\recipe_1483772488890\\work to C:\\work

[jakirkham]

I see. Thanks for the clarification.

[jakirkham]

Could you please take a look at this, @msarahan ?

[jakirkham]

So one thing I worry about is this makes a change to the prefix for all Windows builds. I don't know offhand if this would affect anything negatively, but it is a concern. Would it be possible for us to have some sort of setting in conda-forge.yml to configure this?

[isuruf]

I guess I can remove --croot and it will be the same as conda-build 1.x

[isuruf]

FYI, conda-build 2.x did change the prefix for all builds including Linux and OS X, so any problem we may get with this, we will have to deal with anyways.

[msarahan]

--croot is a much better option than --no-build-id. Although both are tested and should work, I trust croot a lot more. If you can get away with just using --croot, I think you'll be less likely to run into issues.

[isuruf]

For anybody wondering what the differences are,

C:\\Miniconda-x64\\conda-bld\\                           # conda-build 1.x
C:\\Miniconda-x64\\conda-bld\\recipe_1483772488890       # conda-build 2
C:\\Miniconda-x64\\conda-bld\\                           # conda-build 2 with --no-build-id
C:\\recipe_1483772488890                                 # conda-build 2 with --croot
C:\\                                                     # conda-build 2 with --croot and --no-build-id

[jakirkham]

FYI, conda-build 2.x did change the prefix for all builds including Linux and OS X, so any problem we may get with this, we will have to deal with anyways.

Yep, we are aware. That is why it took so long to upgrade conda-build. We rebuilt (or at least tried) anything with a binary prefix we could find. There are a few still needing to be addressed though. 😞

xref: conda-forge/conda-forge-build-setup-feedstock#40

[jakirkham]

Alright, if Mike recommends --croot, I'm more than happy to go that way. Did that test out for you with Boost? Based on your comment, I would think it should.

[jakirkham]

Is there .condarc setting we could use to do this @msarahan or are we stuck using options to conda-build?

[msarahan]

http://conda.pydata.org/docs/config.html#specify-conda-build-output-root-directory-conda-build

[jakirkham]

That's a good point. To make sure I'm understanding you correctly, we could just explicitly override this in .condarc to be C:\\ for Windows, right?

In that case, I'd think we might be better off to do this in conda-forge-build-setup instead of conda-smithy.

[msarahan]

Sure, just set the environment variable mentioned there. I personally wouldn't do just C:, but I really think it won't matter. You wouldn't ordinarily put stuff at /, would you? I don't think it'll break anything, so go ahead if you want, just kind of smells.

[jakirkham]

We can tack on test or something. My main worry is we start tweaking this a lot and am not interested in doing a release here everytime someone wants the root path changed.

[jakirkham]

What do you think about this suggestion, @isuruf? Will this work for you? Maybe we can give it a try in your PR first.

[isuruf]

I'd prefer changing .condarc as that will be easier to debug than environment variable, but I'll give it a try in boost-feedstock

[jakirkham]

So one can do that by adding this to the .condarc.

conda-build:
    root-dir: <dir>

However, AFAICT we can't add that by calling conda config. ( conda/conda#4302 ) Perhaps we can just append similar text to .condarc. Feels kind of hacky. So setting the environment variable is probably the best solution near term.

[jakirkham]

Replaced with a similar adjustment in conda-forge-build-setup.

xref: conda-forge/conda-forge-build-setup-feedstock#50