Skip to content

techhat/webflayer

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
doc
 
 
 
 
pkg
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Web Flayer

Web Flayer is a data mining tool that makes use of the Salt loader system to determine which sites to process, and how. Think of it as wget with plugins.

Requirements

The following python libraries are required:

  • beautifulsoup4
  • psycopg2
  • pyyaml
  • requests
  • salt
  • termcolor

Web Flayer also requires PostgreSQL. Schema can be found in the schema directory.

About

Data mining tool

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages