added molpx recipe

[marscher]

No description provided.

[marscher]

cc @gph82

[Lnaden]

I'm pretty sure the noarch_python will fail. There is a bug involving how the file name string is formed such that it looses its python version identity as part of the string, and I can't figure out why yet.

[Lnaden]

Fail is not the right word, it might build, but there will be no way to tell in the file name what python version is was built for

[marscher]

Can you please point me to the lines of code, which are failing?

[marscher]

I think there is a bug in passing the Config instance to the get_output_file_path function. It is currently passed as a positional argument (2nd), but the signature of this function expects the config as the third argument, so actually the config is passed as argument for no_download_source. Shall we give it a try with an fixed version?

[Lnaden]

Sure. For context though, this is all based on what I think the code is suppose to be doing.

We modify the Python version and the Numpy version of the package here to ensure that Py/NPy versions we are building against are the targeted once, and not those of the system default (e.g. building Python 2.7 version when system is 3.6). There was a change to the conda-build code which changed the behavior of that block which I managed to fix in this PR.

As I understand, the noarch_python flag has been broken for a while, before I started helping with this code, based on all the commented out noarch_python lines in the meta.yaml files. I don't know what was broken so I am still not sure what the proper fix is. You can see in the Travis log down at line 3057 (if that does not directly take you there) that it tries 4 times to build a package called molpx-0.1.3-py_0.tar.bz2 where the missing information should be pyXX_0.tar.bz2. The 0 is just the build number.

[Lnaden]

If you have an idea on how to fix it, I'm all for trying it!

[Lnaden]

I am guessing its the get_output_file_path(m, cfg). Would we need to change that to get_output_file_paths(m, config=config)?

[Lnaden]

And you are about 5 min ahead of me I see

[marscher]

sorry :) I should have replied earlier.

[marscher]

Your last fix to noarch was perfectly fine, but as the config gets "ignored" it could not have worked though.

[Lnaden]

Hmm... Doing some local tests it looks like its still going to mess up the file path. We'll see if it shows up in the Travis build

[Lnaden]

It also looks like it is going to mess up the file path, even if we dont do anything to the config file. I think I might have an answer. I'm going to do some local test

[marscher]

Mhm, locally the path locks fine (noarch/pkgname.tbz2)

[Lnaden]

Shouldn't it also have a python version name attached to it so we now that the builds were compiled against python 2 and all the python 3?

[marscher]

On 15.03.2017 15:02, Levi Naden wrote: Shouldn't it also have a python version name attached to it so we now that the builds were compiled against python 2 and all the python 3?

The mission critical point about noarch_python is also that it is invariant against the python major version ;)

[marscher]

It seems like the platform is set somewhere again to the actual building platform: TEST START: C:\Miniconda\conda-bld\win-32\molpx-0.1.3-py_0.tar.bz2 Now I see what you meant... :)

[Lnaden]

The mission critical point about noarch_python is also that it is invariant against the python major version

I was wondering if that were the case, but I have my doubts as attempting to go up to Python 3.6 caused many, many packages to break here. So I would only accept that logic if the package did not need python at all, which is where I get concerned that we would be skipping tests on all versions of python to ensure code stability.

As I understand from this issue, the linking happens at test time so I can see how that might create the folder there. Check the final output file name that is proposed for the upload and see if it says win-32 or noarch

[marscher]

On 15.03.2017 15:13, Levi Naden wrote: The mission critical point about noarch_python is also that it is invariant against the python major version I was wondering if that were the case, but I have my doubts as attempting to go up to Python 3.6 caused many, many packages to break here. So I would only accept that logic if the package did not need python at all, which is where I get concerned that we would be skipping tests on all versions of python to ensure code stability.

Python 3.6 has a different format for compiled bytecode, yes. But the trick for noarch is then to simply not pre-compile it and let the interpreter create those files in site-pkgs on the first import (which is what I suspect conda is doing). There are no pyc files in a noarch (python) package.

As I understand from this issue <conda/conda-build#317>, the linking happens at test time so I can see how that might create the folder there. Check the final output file name that is proposed for the upload and see if it says |win-32| or |noarch|

I think this is being borked by overwriting the config of the metadata instance...

[Lnaden]

I am actually not seeing this problem on my local testing. Setting noarch_python correctly gives the noarch directory as the output, even after doing the parse_again(). I am however, far more concerned about packages which specify the noarch_python then requires numpy x.x which I am not sure are going to be tested correctly. Are you sure it does not just assign an architecture for the test since it kind of has to?

I am using 'auograd' as my test since it has both python and numpy requirements, but also had its noarch_python line commented out.

[marscher]

On 15.03.2017 15:29, Levi Naden wrote: I am actually not seeing this problem on my local testing. Setting |noarch_python| correctly gives the |noarch| directory as the output, even after doing the |parse_again()|.

This is very strange, I see the same behavior.

I am however, far more concerned about packages which specify the |noarch_python| then requires |numpy x.x| which I am not sure are going to be tested correctly. I am using 'auograd' as my test since it has both python and numpy requirements, but also had its |noarch_python| line commented out.

If a package is pure (it contains only Python code), it does not matter if it depends also on NumPy. The x.x setting should actually be used to require a certain ABI version.

[Lnaden]

If it depends also on NumPy. The x.x setting should actually be used to require a certain ABI version.

The package I am using as test does have the x.x requirement on Numpy. You are saying that it should not matter for the builds here, but the end-user will need to specify a numpy version at test/run install time (which they would have to do anyways). I'm also not sure what the ABI acronym is. Sorry if these are somewhat naive questions, I'm relatively new to package building and distribution.

[Lnaden]

I also know right now the conda-build-all script cannot handle noarch packages which also require numpy x.x since it will try to build that package and fail. The script in general would need to be modified to handle that case. Of course there is talk if we should even be trying to maintain our own conda-build-all script for this exact type of problem (a la #735).

[rmcgibbo]

noarch and numpy x.x don't make sense together

…

On Wed, Mar 15, 2017 at 10:57 AM, Levi Naden ***@***.***> wrote: I also know right now the conda-build-all script cannot handle noarch packages which also require numpy x.x since it will try to build that package and fail. The script in general would need to be modified to handle that case. Of course there is talk if we should even be trying to maintain our own conda-build-all script for this exact type of problem (a la #735 <#735>). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#741 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAnI_mXQEgwCZ2p3eyYEb5psnE86a8eGks5rl_xmgaJpZM4Md4sM> .

-- -Robert

[Lnaden]

noarch and numpy x.x don't make sense together

Okay, that clears a number of things up, it just means that the autograd package meta.yaml file is flawed. Thanks!

[rmcgibbo]

ABI = 'application binary interface' The point here is really Numpy exposes both a Python API and a C API. Using the C API requires including header files from numpy at build time, and once you've built against a certain version of Numpy you _must_ use that version of numpy in production (basically). So with numpy x.x in the conda requirements, a bunch of different numpy versions get added to the build matrix, so you build packages with build tags like "win-64/mdtraj-1.8.0-np112py27_1.tar.bz2" and "win-64/mdtraj-1.8.0-np111py27_1.tar.bz2". The build artifacts are specific to the numpy version. On Wed, Mar 15, 2017 at 11:51 AM, Robert T. McGibbon <rmcgibbo@gmail.com> wrote:

…

noarch and numpy x.x don't make sense together On Wed, Mar 15, 2017 at 10:57 AM, Levi Naden ***@***.***> wrote: > I also know right now the conda-build-all script cannot handle noarch > packages which also require numpy x.x since it will try to build that > package and fail. The script in general would need to be modified to handle > that case. Of course there is talk if we should even be trying to maintain > our own conda-build-all script for this exact type of problem (a la #735 > <#735>). > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#741 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAnI_mXQEgwCZ2p3eyYEb5psnE86a8eGks5rl_xmgaJpZM4Md4sM> > . > -- -Robert

-- -Robert

[rmcgibbo]

And that's why noarch and numpy x.x don't make sense together. noarch is all about _not_ being tied to any ABIs (including the Python ABI), so that you only need 1 build artificat that works for every version of Python. numpy x.x is taking you in the opposite direction, producing more specific artifacts that are tied not only to a particular (operating system, Python version), but also tied to a numpy version.

…

On Wed, Mar 15, 2017 at 11:55 AM, Levi Naden ***@***.***> wrote: noarch and numpy x.x don't make sense together Okay, that clears a number of things up, it just means that the autograd package meta.yaml file is flawed. Thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#741 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAnI_k8w9N1umHl0B29_I9DB2Tsdbe60ks5rmAnhgaJpZM4Md4sM> .

-- -Robert