Skip to content

pooya/github_crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Github Crawler

This repo includes a bunch of helper functions which are wrappers over github api v3 and a disco job that crawls github.

  1. Use a better way for sampling(e.g. snowball sampling)
  2. Use OAuth to increase the rate limit.
  3. Handle exceeding rate limit and sleep.
  4. Prune the final output only the extensions we care about.

About

github crawler based on Disco

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages