Skip to content
A utility to export OWASP ZAP databases as Scrapy filesystem cache directories
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src
.gitignore
README.md
pom.xml

README.md

What

A utility to export OWASP ZAP databases as Scrapy filesystem cache directories

How

mvn -e clean package
mvn dependency:copy -DoutputDirectory=target -Dartifact=org.hsqldb:hsqldb:2.2.9
java -jar target/dumper-1.0.jar -dump myawesome.session
find .scrapy/httpcache/myawesome.session -type f

Why

OWASP ZAP is a great tool, and very kindly holds on to every byte it sees, but harvesting those bytes and putting them into a directory that Scrapy understands is tedious. Tedium is great for computers, so here we are.

It's a glorified script, so don't expect the pinnacle of software engineering although there are a few tests ... for my sanity.

Details

It operates in two basic modes: dump the world and dump just a specific URL.

Dump the world does what you'd expect, and is the -dump argument.

Dump just a specific URL is also what you'd expect, modulo that the URL you specify must match exactly with what's in the ZAP database. This mode will not output any files if the URL you specified was found, but ZAP has no bytes for it (in a 301 redirect, for example).

For convenience the tool also emits a tab separated file of the interesting columns from the ZAP database. It will put the Scrapy cache hash in the FINGERPRINT column if one was produced.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.