Web Flayer

Web Flayer is a data mining tool that makes use of the Salt loader system to determine which sites to process, and how. Think of it as wget with plugins.

Requirements

The following python libraries are required:

• beautifulsoup4
• psycopg2
• pyyaml
• requests
• salt
• termcolor

Web Flayer also requires PostgreSQL. Schema can be found in the schema directory.