A conda-smithy repository for scrapy.
Latest commit 71ee90f Feb 14, 2017 @rolando rolando committed on GitHub Merge pull request #10 from rolando-contrib/scrapy-1.3.2
Update recipe for 1.3.2 release.
Failed to load latest commit information.
appveyor.yml MNT: Re-rendered with conda-smithy 1.7.0 Jan 5, 2017


About scrapy

Home: http://scrapy.org/

Package license: BSD 3-Clauses

Feedstock license: BSD 3-Clause

Summary: A high-level Python Screen Scraping framework

Scrapy is an open source and collaborative framework for extracting the data you need from websites in a fast, simple, yet extensible way.

Current build status

Linux: Circle CI OSX: TravisCI Windows: AppVeyor

Current release info

Version: Anaconda-Server Badge Downloads: Anaconda-Server Badge

Installing scrapy

Installing scrapy from the conda-forge channel can be achieved by adding conda-forge to your channels with:

conda config --add channels conda-forge

Once the conda-forge channel has been enabled, scrapy can be installed with:

conda install scrapy

It is possible to list all of the versions of scrapy available on your platform with:

conda search scrapy --channel conda-forge

About conda-forge

conda-forge is a community-led conda channel of installable packages. In order to provide high-quality builds, the process has been automated into the conda-forge GitHub organization. The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock.

A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services. Thanks to the awesome service provided by CircleCI, AppVeyor and TravisCI it is possible to build and upload installable packages to the conda-forge Anaconda-Cloud channel for Linux, Windows and OSX respectively.

To manage the continuous integration and simplify feedstock maintenance conda-smithy has been developed. Using the conda-forge.yml within this repository, it is possible to re-render all of this feedstock's supporting files (e.g. the CI configuration files) with conda smithy rerender.


feedstock - the conda recipe (raw material), supporting scripts and CI configuration.

conda-smithy - the tool which helps orchestrate the feedstock. Its primary use is in the construction of the CI .yml files and simplify the management of many feedstocks.

conda-forge - the place where the feedstock and smithy live and work to produce the finished article (built conda distributions)

Updating scrapy-feedstock

If you would like to improve the scrapy recipe or build a new package version, please fork this repository and submit a PR. Upon submission, your changes will be run on the appropriate platforms to give the reviewer an opportunity to confirm that the changes result in a successful build. Once merged, the recipe will be re-built and uploaded automatically to the conda-forge channel, whereupon the built conda packages will be available for everybody to install and use from the conda-forge channel. Note that all branches in the conda-forge/scrapy-feedstock are immediately built and any created packages are uploaded, so PRs should be based on branches in forks and branches in the main repository should only be used to build distinct package versions.

In order to produce a uniquely identifiable distribution:

  • If the version of a package is not being increased, please add or increase the build/number.
  • If the version of a package is being increased, please remember to return the build/number back to 0.

Checklist for new releases:

  1. Update the version string.
  2. Update the sha256sum of the pypi release tarball. This can be done by downloading the tarball from pypi and compute the sha256sum.
  3. Reset the build/number to 0. This number is only increased when we re-build a package with no version change.
  4. Update requirements if needed. This can be done by checking the setup.py changes between releases.
  5. Do a test build locally to ensure everything is OK.
  6. Create a branch in your fork and submit the PR.