Skip to content

TransparencyToolkit/TSJobCrawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is a crawler for job listings that require security clearance.

To run-

t = TSJobCrawler.new("search term" (or nil), request_manager, cm_hash or nil)

t.crawl_jobs

For example-

Headless.ly do

r = RequestManager.new(nil, [0, 0], 1)

t = TSJobCrawler.new(nil, r, nil)

t.crawl_jobs

File.write("test.json", t.gen_json)

end

If you input nil for the search term, it downloads as many job listings as possible. Unless you have a lot of RAM, you should run it through Harvester if you want to download as many listings as possible as then you can take advantage of incremental result reporting.

About

Collects listings for jobs that require security clearance.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published