Skip to content


Folders and files

Last commit message
Last commit date

Latest commit


Repository files navigation

Web Flayer

Web Flayer is a data mining tool that makes use of the Salt loader system to determine which sites to process, and how. Think of it as wget with plugins.


The following python libraries are required:

  • beautifulsoup4
  • psycopg2
  • pyyaml
  • requests
  • salt
  • termcolor

Web Flayer also requires PostgreSQL. Schema can be found in the schema directory.