This is the website that runs on https://scrapy.org.
This website is made with Jekyll. The most universal way to install it is:
gem install jekyll
To start the web server locally auto-reloading when files change use:
jekyll serve --watch
The website is updated via the Travis build when merging to the
Adding your company to the website
Are you a company or individual using Scrapy for your product or project? Add yourself to the "Companies using Scrapy" list by sending a pull request:
- add a "company card" in
_data/companies/list/<companyslug>.ymlas a YAML file; you need to set a few fields: a name, a "logouser" for your logo filename, a homepage and a blurb in markdown syntax on how you're using Scrapy. Do add links to any article or tweet spreading your love of Scrapy
- add your logo to
- add the slug of your company card to the list in
Are you providing Scrapy consulting?
If you are a company providing Scrapy consulting or spider development, you can submit a pull request to add your company to the "Scrapy Pros" list.
The companies are sorted by their overall contribution to the Scrapy project and ecosystem. There's a small process to manage that:
- In order to appear in the list, a company must score at least five points in the table below.
- The company must submit an issue to scrapy.org repo whenever they want to update their contributions and ranking.
- This issue must contain references to all the contributions that the company made since the last time their rank has been updated.
This is the criteria that will be used:
|Pull request accepted in scrapy repo||50|
|Creation or contribution to public Scrapy plugins||20|
|Instructional materials such as screencasts, tutorials or blog posts||10|
|Blog posts that spread the word about Scrapy||5|
|Landing page in the company website describing the Scrapy services||2|
|Scrapy listed in the company's technologies stack (if any)||2|
|Answer to StackOverflow question regarding Scrapy||1|
In practice, to be listed, you'll need to:
- add/update your "company card" in
_data/companies/list/<companyslug>.ymlas a YAML file; it needs a
urlfield (e.g. the landing page of your consulting services offer), a
descriptionfield describing your services (use markdown), and a
- add your logo to
- add/update your contributions in
and send a pull request with these changes.