Package for conda #232

Closed
pydanny opened this Issue Jul 24, 2014 · 25 comments

Projects

None yet

4 participants

@pydanny
Collaborator
pydanny commented Jul 24, 2014

Don Rie suggested this to me in a private email.

Conda build instructions: http://conda.pydata.org/docs/build.html

@westurner

These could also be helpful: https://github.com/conda/conda-recipes

@westurner

A cookiecutter-conda-recipe would be useful.

@pydanny
Collaborator
pydanny commented Jan 16, 2015

@westurner Yes it would! Should the recipe go into a different project?

@westurner

So, there are at least one meta.yml files to create for conda:

  1. audreyr/cookiecutter/meta.yml (so that cookiecutter can be built as a conda package)
  2. <...>-recipe/<...>/meta.yml (so that templated projects can be built as conda packages (which do not run setup.py at install time))

see: http://conda.pydata.org/docs/build.html

@pydanny
Collaborator
pydanny commented Jan 26, 2015

I built a conda recipe for cookiecutter: pydanny/conda-recipes@0197eb3

I uploaded it to https://binstar.org/pydanny/cookiecutter. I don't know how to make it cross-platform. 😒

@westurner

I built a conda recipe for cookiecutter: pydanny/conda-recipes@0197eb3

Nice. Thanks! This may be a bit easier to maintain:

source:
  git_tag: 0.9.1
  git_url: https://github.com/audreyr/cookiecutter.git

I don't know how to make it cross-platform.

http://conda.pydata.org/docs/commands.html#conda-build-commands

http://conda.pydata.org/docs/commands/build/conda-convert.html

@pydanny
Collaborator
pydanny commented Jan 26, 2015

I changed the source as you prescribed and ran conda convert on cookiecutter-0.9.1.tar.g inside the recipe directory. This is what I got:

conda convert cookiecutter-0.9.1.tar.gz --platform linux-32
Error: cannot convert: /Users/danny/opensource/conda-recipes/cookiecutter/cookiecutter-0.9.1.tar.gz

What am I doing wrong?

@westurner

Something like this?

Adapted from: http://conda.pydata.org/docs/build.html

package:
  name: cookiecutter
  version: {{ environ['GIT_DESCRIBE_TAG'] }}

build:
  number: {{ environ.get('GIT_DESCRIBE_NUMBER', 0) }}

  # Note that this will override the default build string with the Python
  # and NumPy versions
  string: {{ environ.get('GIT_BUILD_STR', '') }}

The docs for this could certainly be more clear.

@westurner

Is cookiecutter-0.9.1.tar.gz a conda package?

@westurner

-p all may also work?

@pydanny
Collaborator
pydanny commented Jan 26, 2015

I got it! First of all, I discovered that my cookiecutter file was actually a PyPI distro. So I found the right file and used that. After some research and trials my conversion command is:

$ conda-convert /Users/danny/miniconda3/conda-bld/osx-64/cookiecutter-0.9.1-0_ga057b2c.tar.bz2 -p all -o cookiecutter/

Then I ran $ binstar upload on all the generated files. The result can be seen at https://binstar.org/pydanny/cookiecutter.

Is this good? I can install from here but I don't know about Linux or Windows users

@westurner

Nice. I'll have a minute to check installation from Linux later this evening.

This seems like a for-loop that could be just one command?

@westurner

A conda skeleton -t python https://github.com/audreyr/cookiecutter/ could be useful, as well.

@westurner

Linux-64:

$ conda install -c pydanny cookiecutter
Fetching package metadata: ...
Solving package specifications: 
Error: Could not find some dependencies for cookiecutter: binaryornot >=0.2.0

...

$ conda install -c pydanny binaryornot cookiecutter
Fetching package metadata: ...
Error: No packages found in current linux-64 channels matching: binaryornot
@pydanny
Collaborator
pydanny commented Jan 28, 2015

Thanks! I'll update the binaryornot package

@pydanny
Collaborator
pydanny commented Jan 28, 2015

Okay, binaryornot updated with the following:

$ conda-build binaryornot
$ convert binaryornot-0.3.0-0_ge797740.tar.bz2 -p all
$ binstar upload linux-32/binaryornot-0.3.0-0_ge797740.tar.bz2
@pydanny
Collaborator
pydanny commented Jan 28, 2015

Please try again!

@pydanny
Collaborator
pydanny commented Jan 28, 2015

How I generated the conda package for cookiecutter:

$ conda skeleton pypi cookiecutter

This will create a conda recipe in a new cookiecutter directory. Change the generated cookiecutter/meta.yml to include:

source:
  git_tag: 0.9.1
  git_url: https://github.com/audreyr/cookiecutter.git

package:
  name: cookiecutter
  version: {{ environ['GIT_DESCRIBE_TAG'] }}

build:
  number: {{ environ.get('GIT_DESCRIBE_NUMBER', 0) }}

  # Note that this will override the default build string with the Python
  # and NumPy versions
  string: {{ environ.get('GIT_BUILD_STR', '') }}

To build and convert, use the following commands:

$ conda-build cookiecutter
$ convert cookiecutter-0.9.1-STUFF -p all

Finally, upload to binstar:

$ binstar upload linux-32/cookiecutter-0.9.1-BUILDNUM
$ binstar upload linux-64/cookiecutter-0.9.1-BUILDNUM
$ binstar upload osx-64/cookiecutter-0.9.1-BUILDNUM
$ binstar upload win-32/cookiecutter-0.9.1-BUILDNUM
$ binstar upload win-64/cookiecutter-0.9.1-BUILDNUM

Note: BUILDNUM is the build number created by the recipe.

@westurner

With linux-64 Python2:

$ conda install -c pydanny cookiecutter
Fetching package metadata: ...
Solving package specifications: ..
Error: Unsatisfiable package specifications.
Generating hint: 
[2/2                 ] |##########################################################################################################| 100%

Hint: the following combinations of packages create a conflict with the
remaining packages:
  - python 2.7*
  - cookiecutter
# conda install -c pydanny cookiecutter

With linux-64 Python 3:

$ conda install -c pydanny cookiecutter
Fetching package metadata: ...
Solving package specifications: .
Package plan for installation in environment ~/-wrk/-ce34/cookiecutter34:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    yaml-0.1.4                 |                0         205 KB
    binaryornot-0.3.0          |       0_ge797740           6 KB
    markupsafe-0.23            |           py34_0          31 KB
    pyyaml-3.11                |           py34_0         353 KB
    jinja2-2.7.3               |           py34_1         351 KB
    cookiecutter-0.9.1         |       0_ga057b2c          28 KB
    ------------------------------------------------------------
                                           Total:         975 KB

The following NEW packages will be INSTALLED:

    binaryornot:  0.3.0-0_ge797740
    cookiecutter: 0.9.1-0_ga057b2c
    jinja2:       2.7.3-py34_1    
    markupsafe:   0.23-py34_0     
    pyyaml:       3.11-py34_0     
    yaml:         0.1.4-0         

Proceed ([y]/n)? y

Fetching packages ...
yaml-0.1.4-0.t 100% |########################################################################################| Time: 0:00:00 498.95 kB/s
binaryornot-0. 100% |########################################################################################| Time: 0:00:00   5.18 MB/s
markupsafe-0.2 100% |########################################################################################| Time: 0:00:00 191.24 kB/s
pyyaml-3.11-py 100% |########################################################################################| Time: 0:00:00 569.07 kB/s
jinja2-2.7.3-p 100% |########################################################################################| Time: 0:00:01 228.16 kB/s
cookiecutter-0 100% |########################################################################################| Time: 0:00:00 463.66 kB/s
Extracting packages ...
[      COMPLETE      ] |##########################################################################################################| 100%
Linking packages ...
[      COMPLETE      ] |##########################################################################################################| 100%
# conda install -c pydanny cookiecutter
@pydanny
Collaborator
pydanny commented Jan 29, 2015

conda/conda-recipes@3b5f5d3

Next step: Document installation via conda.

@srooks10
srooks10 commented Sep 2, 2015

Hi, one needs one conda package per python version. The main problem in your recipe (both for binaryornot and cookiecutter is this line:

string: {{ environ.get('GIT_BUILD_STR', '') }}

Just remove it and one package per python version gets created. I've put my recipe for cookiecutter 1.0.0 here with more corrections (entry point definition and dependencies) .
To use it you will also have to correct the recipe for binaryornot and package click at version 3.3.

I hope it will relieve your sadness 😉
Thanks a lot for your work on cookiecutter!

@bollwyvl
Contributor

tl:dr; cookiecutter is now installable via conda:

conda install -c conda-forge cookiecutter

Hi, folks! I have been dinking around with cookiecutter for some time... great stuff ❤️!

Over on project jupyter, we're trying to raise our game for developers of extensions: since these almost always require Node Stuff, we'll probably also be working some yeoman stuff, but cookiecutter is a natural fit for easy-to-get-started projects.

As part of that, since anaconda is the suggested end-user way to get python for jupyter, conda packages have become an attractive approach to distribution. We (me, some members of the community, my employer @continuumio) are also advocating conda as a way to build and distribute jupyter extensions... even for pypi.

How did we get here?

conda-forge is a way to move much of the pain of cross-platform (linux, osx, windows) building and testing to a distributed, community-based, automated approach, requiring less fiddling-per-package, and removing bottlenecks. I put together a PR that added the whole chain in one go (not really recommended, but the chain just kept going), and it was pretty smooth!

Now that the initial recipe-writing has been done, based on all your guys' great work, that PR was broken up into "feedstocks": community-managed repos, and are available to all users of anaconda (dynamic anaconda.org and pypi badges for visual comparison... looks like I already need to update poyo, or just pulled the wrong version!):

Call for Maintainers!

If you are interested in helping maintain these recipes, just raise a PR against the respective repo, adding your github username to the recipe/meta.yml#/extra/recipe-maintainers, e.g. cookiecutter-feedstock. Once the PR is accepted, you'll automatically be added as a team member on the conda-forge organization! It is still preferred to fork to your own username and submit PRs that way :)

What will I have to do?

This work will mainly consist of changing two lines in meta.yml whenever a new release comes out:

  • version (as tagged in github)
  • sha256 (the sha256 of the downloaded file)
  • pro-tip: just update version first, conda build then grab the new hash!

These feedstocks also re-run all of the tests... so if the test approach changes (new test dependencies, which can still be pip-installable), a little more work will be required! All in all, this shouldn't be too much overhead in return for getting shiny, tested, cross-platform builds!

Thanks again everyone!

@pydanny
Collaborator
pydanny commented May 20, 2016 edited

@bollwyvl Awesome work! At some point hopefully in the next week let's try and get Cookiecutter to pass tests on Windows.

In the meantime, I'm closing this issue.

@pydanny pydanny closed this May 20, 2016
@bollwyvl
Contributor

Awesome work! At some point hopefully in the next week let's try and get Cookiecutter to pass tests on Windows.

Thanks! I agree re: windows! Thanks for stepping up on conda-forge/cookiecutter-feedstock#1!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment