A Node.js application for pulling Github organisation statistics into a SQLite or PostgreSQL database.
This project was built with simplicity and ease of use in mind. We simply wanted GitHub data in a relational database which we could then create visualisations for using Metabase (https://www.metabase.com/).
There are other projects out there like GhCrawler which collect and handle much more detailed GitHub data than this project, that is not our focus.
Data collected by Roadblock
Roadblock automatically collects all data from all the organisations which the token user has a public membership of, so there is no configuration of which organisations to collect from.
When Roadblock runs, it will collect:
- Members of organisations
- Pull Requests
- Community Profiles
- External contributions
These instructions will help you get a copy of the project up and running on your local machine for development and testing purposes.
To run, you must have a Github token setup, you can setup a new token here: https://github.com/settings/tokens. Token needs access to repo, read:org, read:user and user:email.
You can also pass in a list of orgs which it should specifically query, it will only attempt to do so if the token gives the user access to this.
> Clone this project to your local machine > npm install > npm start [GITHUBTOKEN] [ORG,ORG] [TYPE,TYPE]
- ORG can be left blank or can be a commaseperated string of orgs to index
- TYPE can be left blank or be a commaseparated string of the following:
Both ORG and TYPE can also use negative filters so "!topics," will collect all repository data except topics "!zalando,!stups," will go through all organisations that the profile have access to, except zalando and stups.
script will run between 10 and 20 minutes and store collected data in either the default SQLite database or in a SQLite database you configure.
Database and json summaries will be stored in the /data folder
What software you need to install:
- Metabase (optional) - to visualise the collected data
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
This project is licensed under the MIT License - see the LICENSE.md file for details