Skip to content

greenpeace/planet4-gpi-export

Repository files navigation

  1. Have python and pip working

  2. Install virtualenv ( https://virtualenv.pypa.io/en/stable/installation/ )

  3. make a directory where your project will run

    mkdir p3-gpi-export

  4. Tell it to run in a vritualenv

    virtualenv p3-gpi-export

  5. Go in the directory

    cd p3-gpi-export

  6. Start the virtualenv

    source bin/activate

  7. Install scrapy

    pip install Scrapy

  8. Install dateparser

    pip install dateparser

  9. Initiate a scrapy project

    scrapy startproject p3_gpi_export

  10. Download the crawler

    git clone https://github.com/greenpeace/planet4-gpi-export

  11. Copy the crawler file in the scrapy project crawlers

    cp planet4-gpi-export/all_spider.py p3_gpi_export/p3_gpi_export/spiders/all_spider.py

  12. Go in the directory of the scrapy project

    cd p3_gpi_export/

  13. Run the crawler

    scrapy crawl all

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published