New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support optional dependencies spec from conda 4.4 #2001

Merged
merged 1 commit into from May 9, 2017

Conversation

Projects
None yet
5 participants
@msarahan
Copy link
Contributor

msarahan commented May 8, 2017

fixes #1964

@msarahan

This comment has been minimized.

Copy link
Contributor

msarahan commented May 8, 2017

@kalefranz please review - especially make sure that the test case adequately captures the functionality that you had in mind.

@msarahan msarahan force-pushed the msarahan:add_optional_deps branch from 6909ae9 to f4749db May 8, 2017

@kalefranz
Copy link
Member

kalefranz left a comment

LGTM

@kalefranz

This comment has been minimized.

Copy link
Member

kalefranz commented May 9, 2017

codecov has been giving me problems recently on Appveyor too. I pinned to pip install codecov==2.0.5 a few days ago. That seemed to help.

@msarahan

This comment has been minimized.

Copy link
Contributor

msarahan commented May 9, 2017

Thanks. Merging.

@msarahan msarahan merged commit e60b399 into conda:master May 9, 2017

1 of 2 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@msarahan msarahan deleted the msarahan:add_optional_deps branch May 9, 2017

@sodre sodre referenced this pull request May 11, 2017

Merged

Add airflow 1.8.0 recipe to conda-forge. #2882

7 of 7 tasks complete
@183amir

This comment has been minimized.

Copy link
Contributor

183amir commented Sep 26, 2017

There is this option in conda build:

build:
  pin_depends: strict

Can this be used for run_constrained too? Or is something similar available?

@msarahan

This comment has been minimized.

Copy link
Contributor

msarahan commented Sep 26, 2017

no, pin_depends is something completely different. See this for its origin: #741

The idea for it was to constrain dependencies tightly. optional dependencies are completely different - they say "if this dependency is installed, make sure it meets these additional constraints"

@183amir

This comment has been minimized.

Copy link
Contributor

183amir commented Sep 26, 2017

So I have this recipe:

package:
  name: bob-devel
  version: {{ os.environ.get('CI_BUILD_TAG', 'vx.y.z')[1:] }}

build:
  number: 0
  pin_depends: strict

requirements:
  build:
    - numpy x.x
  run:
    - python
    - numpy x.x
    # Our dependency list.
    # Everything is pinned to allow better reproducibility.
    # The pinnings below are coming from anaconda 4.4.0
    - curl 7.52.1
    - cython 0.25.2
    - cytoolz 0.8.2
    - dask 0.14.3
    - freetype 2.5.5
    - hdf5 1.8.17
    - imagesize 0.7.1
    - ipython 5.3.0
    - jpeg 9b
    - libpng 1.6.27
    - libtiff 4.0.6
    - matplotlib 2.0.2
    - mkl 2017.0.1
    - nose 1.3.7
    - numba 0.33.0
    - numba 0.33.0
    - numexpr 2.6.2
    - pandas 0.20.1
    - pillow 4.1.1
    - pip 9.0.1
    - scikit-image 0.13.0
    - scikit-learn 0.18.1
    - scipy 0.19.0
    - setuptools 27.2.0
    - six 1.10.0
    - sphinx 1.5.6
    - sqlalchemy 1.1.9
    - toolz 0.8.2
    # these pinnings of indirect dependencies make sure bob-devel is compatible with anaconda 4.4.0, hopefully
    - bokeh 0.12.5
    - nbconvert 5.1.1
    - testpath 0.3
    # the rest of the dependencies
    - boost 1.61.0
    - caffe 1.0.0rc5  # [linux]
    - cmake 3.6.3
    - coverage 4.3.4
    - cyvlfeat 0.4.5
    - dlib 19.0
    - docopt 0.6.2
    - ffmpeg 2.8.10
    - gcc 4.*  # [linux]
    - giflib 5.1.4
    - ipdb 0.10.2
    - kaldi r7271.1a4dbf6  # [linux]
    - keras 2.0.8  # [linux]
    - libblitz 0.10
    - libgcc 4.*  # [linux]
    - libmatio 1.5.6
    - libogg 1.3.2
    - libsvm 3.21
    - madmom 0.15.1
    - menpo 0.8.1
    - menpofit 0.5.0
    - menpowidgets 0.3.0p6
    - mne 0.14.1
    - opencv 3.1.0
    - pkg-config 0.28
    - pkgtools 0.7.3
    - pyedflib 0.1.9
    - schema 0.6.5
    - sox 14.4.2
    - sphinx_rtd_theme 0.2.4
    - tensorflow 1.2.1  # [linux]
    - toolchain 2.3.2
    - virtualenv 15.1.0
    - vlfeat 0.9.20
    - zc.buildout 2.9.2

test:
  imports:
    - tempfile

about:
  home: http://www.idiap.ch/software/bob
  license: BSD 3-Clause
  summary: A meta-package to define an environment for Bob package development

extra:
  recipe-maintainers:
    - anjos
    - amohammadi

And I pin the versions that I want in it. But the great thing about this is that it pins the indirect dependencies too. It's like taking a snapshot of the environment while only pinning the relevant ones in the build dependencies.

Now what I really want is actually this frozen environment (with direct and indirect pins) to be in run_constrained instead of run. This way I don't have to install all the dependencies when I install bob-devel but if in future I decide to install one of those dependencies in my environment, I will get a version that is mandated by bob-devel.

Does that make sense?

@msarahan

This comment has been minimized.

Copy link
Contributor

msarahan commented Sep 26, 2017

Yes, I can see the draw in that. I'd gladly consider a PR. It may also be better to do this with configuration on the conda side somehow - just depends which way is easier to share with other people.

@183amir

This comment has been minimized.

Copy link
Contributor

183amir commented Sep 26, 2017

Is it possible to do this with conda configurations?
I would also see that making a package like bob-devel a build dependency as well would be useful.

@msarahan

This comment has been minimized.

Copy link
Contributor

msarahan commented Sep 26, 2017

I think it's possible, but you should file an issue on the conda tracker and ask for examples and docs.

@kalefranz

This comment has been minimized.

Copy link
Member

kalefranz commented Sep 26, 2017

If I understand correctly, this is exactly how the pinned_packages configuration parameter works in conda 4.4 now. Note you can have an environment-only .condarc file (at the prefix root of the environment). The --env flag on conda config also manipulates this file.

@183amir

This comment has been minimized.

Copy link
Contributor

183amir commented Sep 27, 2017

Hi @kalefranz while having this as the conda configuration is useful. I don't think you can benefit from that during the conda-build process. Right now, with the introduction of the variants, it is very easy to pin direct dependencies during build time. But, it is not possible to pin the indirect recipes during build time. But I am thinking that I can achieve this with the new run_constrained feature.

@isuruf

This comment has been minimized.

Copy link
Contributor

isuruf commented Apr 13, 2018

Is this documented somewhere?

@jakirkham

This comment has been minimized.

Copy link
Contributor

jakirkham commented May 22, 2018

Also not seeing docs. Is there an issue for this or am I just missing it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment