Skip to content

C0deboy/stats-scraper

Repository files navigation

Programing languages statistics scraper

Java tool rewrited to Kotlin which collects (scraps from web) statistics for programing languages for my site jaki-jezyk-programowania.pl.

Currently, the tool is fetching data for each language from:

  • Github
    • top 10 projects
    • number of projects
    • number of projects with more than 1000 stars
  • Meetup.com
    • number of members
    • number of meetups
  • StackOverflow
    • number of tagged questions
  • Wikipedia
    • latest language version
  • Tiobe INDEX
    • position at last year
    • position at this year
  • Spectrum ranking
    • position at last year
    • position at this year

Everything is stored in two json files: - statistics.json - languagesVersions.json

If statistics.json already exists, then it will be renamed (date will be appended) and fresh one will be created as statistics.json.

Running

Note:

If you will run this tool more than once within short time then errors occurs due to Github api restrictions. 

Provide github authentication token under src\main\resources\config.properties if u want to fetch data from Github

Follow this guide if u don't have token.

GithubAuthToken=token 22sadasdsa34r32412342134214324123

Otherwise, you need to pass parameter remove line where GithubDataScraper is added at Main class.

Sample log output of the tool:

Sample output of tool Sample output of tool 2

Built With

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages