Skip to content
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Hackathon Scraper

Hi all, this repo is to help out getting OKRs for DevRel folks!


For this, you need to install selenium with firefox to run the gitcoin portion. You can find out how to do that here

Otherwise, just add the flag --no-gitcoin

To use

git clone cd hackathon_scraper

If you want to run only the github report.

If you'd like to just get the github report of basic keywords, here is what you'll need to do (you'll need selenium and firefox) --keywords chainlink --days-back 7 --send-mail-to

I run mine in a crontab so it runs automatically, and I get a report on repo activity every week.

Here is how it works:

  1. Every week run python --hackathon-keyword <KEYWORD_HERE> With `<KEYWORD_HERE> being the type of hackathon you're looking for

This will save a file to next_week.json which will have a list of hackathons that you'll look to check out once submissions are availible.

  1. After each week, run python --github-keyword <GITHUB_KEYWORD>

This will pull from next_week.json and give you the github projects from submissions that have the project you're looking for!

Example: At the start of the week I run: python --hackathon-keyword blockchain This will show me all the blockchain hackathons that are ending in 7 days or less.

Then I run: python --github-keyword chainlink This will show me all the hackathon submissions that recently ended that have their github repo have that keyword in it.

Sample output:

$ python --github-keyword chainlink
Total submissions: 53 + 45 = 98
2020-06-16 23:31:14.815999
 No projects had that keyword :(
$ python --github-keyword const --input-file tests/test_data.json 
Total submissions: 5 + 45 = 50
2020-06-16 23:40:15.635757
 Number of devpost hackathon projects 2
INFO:root:[{'url': '', 'prizes': 165375, 'platform': 'devpost'}, {'url': '', 'prizes': 165375, 'platform': 'devpost'}]
INFO:root: Average prize pool of submissions: 165375.0
 Number of gitcoin hackathon projects 3
INFO:root:['/hackathon/GitxChange/projects/', '/hackathon/GitxChange/projects/', '/hackathon/GitxChange/projects/']
INFO:root: Average prize pool of submissions: 110250.0

Have fun!


Note: Due to github rate limiting you have to test each one at a time

Run run cd tests; pytest -v or to run a single test cd tests; pytest -v -k test_repo_has_keyword


  • Github organizations accounted for
  • If a hackathon has multiple pages of submissions it only gets the first page
  • Gitlab accounted for
  • Devfolio Integration
  • Gitcoin integration
  • More analytics metrics
  • Github OAuth key added so we don't get rate limited and have to run induvidual tests

The report aspect

  1. Set up schedule on your mac (if you are using one) and set the schedule to wake up at the specific time
  2. setup the crontab job with crontab -e

sample crontab: 1 7 * * MON cd ~/code/hackathon_scraper/ && python


No description, website, or topics provided.




No releases published


You can’t perform that action at this time.