Analyze your jobs on Scrapy Cloud.

The tool is hosted on Github Pages: https://zytedata.github.io/sctools/

Table of Contents

• Screenshots
  • Display multiple jobs stats as a table
  • Sort jobs by some metric
  • Filter jobs by spider args
  • Customize which stats to show
  • Select which scrapy stats to show
  • Reorder the columns by drag and drop
  • Visualizing the Jobs stats
• License

Screenshots

Display multiple jobs stats as a table

Sort jobs by some metric

Filter jobs by spider args

Customize which stats to show

Select which scrapy stats to show

Reorder the columns by drag and drop

Drag and drop to reorder the columns:

Visualizing the jobs stats

Number of items crawled over time:

Development Guide

Install pnpm

curl -fsSL https://get.pnpm.io/install.sh | sh -

Install clojure

Follow this document.

Launch the dev server

pnpm watch

It could take a while running for the first time.

In another terminal:

pnpm watch-css

Now visit http://127.0.0.1:3345 in a web browser.

License

Copyright © 2021 Lucy Wang

Distributed under the the Apache License, Version 2.0.