Skip to content

A 3-Step Program to gather links of job postings, parse the postings, and analyze them

Notifications You must be signed in to change notification settings

shib-maximalist/monster.de-crawler-analyzer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Monster.de Crawler & Analyzer

Dependencies

  • requests (Handling Requests to Server)
  • lxml (Parsing HTML)
  • beautifulsoup 4 (Parsing HTML)
  • npyscreen (Console based UI)
  • curses (windows: Binaries)
  • whoosh (Information Retrieval)
  • regex (pip install regex, not the python included one)

Features

  • Gathers deep links of job postings from monster.de for a given search term
  • Saves links to file
  • Parses and stores the postings content
  • Correlation analysis of tokens inside of Postings
  • Display of analysis results

How to Run

Start UserDialog.py

Important: Because most IDE's reroute stdout to the build in console which npyscreen prohibits this program must be launches via comandline and not from inside an IDE.

About

A 3-Step Program to gather links of job postings, parse the postings, and analyze them

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages