Skip to content
Data mining tool
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc
flayer
pkg
plugins
salt
schema
scripts
.gitignore
.pylintrc
CHANGELOG.rst
LICENSE
README.rst
setup.py

README.rst

Web Flayer

Web Flayer is a data mining tool that makes use of the Salt loader system to determine which sites to process, and how. Think of it as wget with plugins.

Requirements

The following python libraries are required:

  • beautifulsoup4
  • psycopg2
  • pyyaml
  • requests
  • salt
  • termcolor

Web Flayer also requires PostgreSQL. Schema can be found in the schema directory.

You can’t perform that action at this time.