Skip to content

malignantz/DualScraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Dual Scraper

Instructions

  1. node --use_strict --harmony server.js
  2. Go to / to set authssid (token available from cookies of logged in browser)
  3. Submit token and wait for scraping to finish. Note: this may take up to 3 minutes. Browser will hang. JSON data will be displayed when scraping is complete
  4. /orgs & /users contain the json data and output it to stdout & request response
  5. /api/user/:user will respond with user createdDate, pledgeStatus and a list of organizations. Detailed organization information only available after scraping.
  6. /user/:user has user-friendly HTML response with organization links

Getting authssid Token from Google Chrome

  1. Open Developer Tools - Win/Linux: F12, macOS: Cmd+option+J
  2. Copy authssid token value and paste into textbox at / dev_tools_screenshot

Please star/watch repo or donate if you like Dual Scraper!

BTC: 1DJK3yVNNpqCyA5vssUwWKp4HEEWarY2m5

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published