Python 2.7 Conda builds

[patricksnape]

At the moment, there only seems to be conda builds for Py34 and not for Py27.

One suggestion may be that since you are git tagging your releases, you could set up appveyor and travis to automatically upload builds on tagged releases?

[almarklein]

At the moment, there only seems to be conda builds for Py34 and not for Py27.

That is correct.

One suggestion may be that since you are git tagging your releases, you could set up appveyor and travis to automatically upload builds on tagged releases?

That would be nice and saving me time to fire up my VM :)

Personally, I have no experience with building and uploading build artifacts. If you have, maybe you'd be interested in submitting a PR for this?

Another option is to keep the recipe in Continuums conda-recipes repo up to date, but I am not sure how regular they build it (I don't they build every package in the list).

[patricksnape]

It is a bit difficult to test the PR, primarily due to the fact that I don't have access to the configuration of the continuous integration servers.

I work on a project called Menpo, and we use a small script hosted here.

We host the conda build scripts inside the main python repository, and then we essentially just run this command, which does some work to find out if the current build is a tag or not. If it is a tag, it uploads it to binstar automatically. Otherwise, it uploads every build to a special channel called master that hosts the latest passing build.

The only thing that really changes is the manner in which the meta.yaml works out what version to build. We use the CONDACI_VERSION environment variable to specify this in the meta.yaml.

The last thing to mention is that you need to set up your Binstar key as encrypted variables inside your Appveyor and Travis builds.

In the future, we would like to extend the script to also automatically release to pypi and build and upload wheels.

[patricksnape]

As a small bump to this - I think we are keen to shift to imageio away from pillow - so I've been working on conda builds. My biggest question is:

If I create a conda package that ships freeimage and ffmpeg, can I tell imageio to use those without resorting to an environment variable? Since I won't know the environment path at runtime I would prefer if imageio could be overridden to prefer a specific relative path without needing environment variables. Is this currently possible? I don't really want people to need to download freeimage etc as I like the idea of a one shot install.

[almarklein]

Note that we're working on a conda package for freeimage and imageio.

conda-archive/conda-recipes#362
conda-archive/conda-recipes#335

[patricksnape]

@almarklein OK - I commented over at the freeimage recipe. I have better working builds on Windows (actually builds freeimage).

The question still stands - if the freeimage conda package is installed - can imageio know about it without the environment variable? What if imageio maybe used a config file and then fell back to the environment?

[almarklein]

Yes, as long as its in one of the places that imageio looks. Which seems to be the case by looking over your recipe.

https://github.com/imageio/imageio/blob/master/imageio/core/findlib.py#L63

[almarklein]

Does your recipe work on osx as well? If its better, do you mind doing a PR to conda-packages?

[patricksnape]

https://travis-ci.org/menpo/conda-freeimage
https://ci.appveyor.com/project/jabooth/conda-imageio
http://jenkins.menpo.org/job/conda-freeimage/

But it's a bit hacky at the moment because I'm using the latest
libfreeimage and just renaming it so that imageio doesn't redownload it.
Would be good if you updated imageio to the latest freeimage.

On Tue, 8 Sep 2015 22:14 Almar Klein notifications@github.com wrote:

Does your recipe work on osx as well? If its better, do you mind doing a
PR to conda-packages?

—
Reply to this email directly or view it on GitHub
#78 (comment).

[almarklein]

Would be good if you updated imageio to the latest freeimage.

I'll make a note. But wont have much time this week.

[patricksnape]

That findlib.py isn't in the current release is it?

[almarklein]

Its been there from the start :)

[patricksnape]

😞 Well it doesn't seem to work? If I remove the freeimage library that gets copied by imageio during installation, and the conda environment includes the freeimage library inside conda/lib - it still re-downloads freeimage. At the moment it seems I have to call a shim that soft links the freeimage library into resources in order for it to work?

Also, ffmpeg doesn't have this logic? And I can't get it to use my ffmpeg without resorting to soft linking?

I'm happy to help contribute to fix this issues by the way! As I'm keen for my project to use imageio - particularly for video loading. However, I'm unsure about how you want this all to work with conda? Should we be adding lookups whereby it tries to work out of it is in a conda environment (and the lib folder is a certain number of directories up?). Also - do you want to enforce the naming scheme you currently have? If so I need to update my recipes for freeimage and ffmpeg.

[almarklein]

What's the name of your lib? Something that contains freeimage or libfreeimage should be fine (https://github.com/imageio/imageio/blob/master/imageio/plugins/_freeimage.py#L452). I'd try putting a breakpoint or some print-statements in find_lib to see what's going on. It should work.

[patricksnape]

@almarklein Turns out I was being a fool - you are totally correct - I was calling the wrong method to get the freeimage path, rather than instantiating the Freeimage class, so I was bypassing all the clever logic. I think that this means I can just chuck my logic for copying the files. Still, I'm happy to push the freeimage updates (I guess just updating to look for the latest version), I can send you the binaries if you want? In fact, you can get them straight off of conda if you want!

The project is called menpo. Ironically, we had implemented a lot of similar logic for file importing, but your image/video logic is just much better than pillow so we can't thank you enough for imageio 😄

[almarklein]

Yes, updating to the latest freeimage would be great. Are these the latest binaries? https://anaconda.org/menpo/freeimage/files Is there no linux32 binary?

Menpo looks cool! The group in which I did my PhD did similar things with faces for purpose of biometrics (but I have not done anything related).

[patricksnape]

Unfortunately I've never managed to find a Linux 32-bit build server :( We use travis for everything and they only support 64-bit Ubuntu. I've been pushing hard for automated builds so that everything is reproducible and my environment doesn't leak into package building. Sorry about that :( Maybe we can setup jenkins on a 32-bit VM in our department, similar to how we are running our OSX box...

Those are the latest files - they seem to be working fine for me. The only issue is the names of the files that get dropped out - they are fine for renaming because they have no dependencies, but maybe @jakirkham would prefer to try and merge my freeimage recipe with his first so that we can rely on the official Ananconda freeimage rather than my hosted one?

Thanks! Hopefully we can start to make some strong headway on videos now that imageio is going to be doing all the hard work!

[rgommers]

Unfortunately I've never managed to find a Linux 32-bit build server :( We use travis for everything and they only support 64-bit Ubuntu.

This thread is old, but maybe this is still of interest: https://github.com/numpy/numpy/blob/master/tools/travis-test.sh#L175 is how numpy sets up 32-bit Linux builds on TravisCI.

[almarklein]

Thanks @rgommers, though I think we can close this, since we're now on conda-forge. Plus we don't rely on FreeImage anymore.