Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
cmd
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Discover

A weekly GitHub newsletter tailored for you. Discover what your peers are doing on GitHub.

Discover monitors the people you follow on GitHub and sends you a weekly newsletter with their activity. Discover projects your friends are working on, new repositories they recently discovered, and issues or releases you might have missed.

It consists of three parts.

Crawler

The crawler is the part that monitors GitHub activity. It constantly crawls the activity of the people you follow and feeds our Graph DB. Based on that, then, we are able to export data based on your connections.

API

The API is serving our website.

Extraction (Better name pending?!)

Extraction is the part that queries our GraphDB, prepares the email template and sends it out as a weekly newsletter.

Technologies

This project is using:

  • Written in Go
  • Neo4j for our GraphDB
  • docker/docker-compose
  • PostgreSQL for everything relational
  • Redis for cache
  • Mailgun for emails
  • GitHub duh?

Usage

Consult the table below and export your preferences as environment variables.

export GITHUB_TOKEN=...
make run-crawler
make run-extraction
make run-api

Available env vars:

Variable Description Required Default
GITHUB_TOKEN GitHub token for the crawler yes
LOG_LEVEL Log level: error, info, debug, trace no info
QUEUE_STORE_DIR path for dqueue persistence; defaults to ~/go-discover
SUGGESTION_STORE_TYPE no sqlite3
SUGGESTION_STORE_DSN no ./local/suggestions.db
NEO4J_HOST no http://localhost:7474/db/data
REDIS_HOST no localhost:6379
API_BIND_ADDRESS no 0.0.0.0:8080
GITHUB_CLIENT_SECRET GitHub OAuth secret yes
GITHUB_CLIENT_ID GitHub OAuth ID yes
GITHUB_CALLBACK_URL GitHub OAuth callback URL no http://localhost:8080/github/callback
MAILGUN_DOMAIN yes
MAILGUN_APIKEY yes
MAIL_SENDER_ADDRESS yes
LOCK_USER_DURATION no 12h
LOCK_REPOSITORY_DURATION no 24h

Development

make tools
make deps
make lint
make test

Available make targets:

  • make deps - installs dependencies
  • make tools - installs required tools under ./bin
  • make lint - lints package using ./bin/golangci-lint
  • make build-api - builds cmd/api as ./bin/api
  • make build-crawler - builds cmd/crawler as ./bin/crawler
  • make build-extraction - builds cmd/extraction as ./bin/extraction
  • make run-api - builds and runs cmd/api
  • make run-crawler - builds and runs cmd/crawler
  • make run-extraction - builds and runs cmd/extraction
  • make test - tests package
  • make clean - removes temp files

Contribute

Want to contribute to this project?

We absolutely welcome contributions to this project. Take a look at our open issues and submit a PR if you see one you can help with.

Feel free to get in touch with us to discuss more about this project and how you can get involved. We love hearing from people!

Creators

Name Twitter
Kostas Bariotis @kbariotis
George Antoniadis @geoah

About

Discover what your peers are doing in GitHub

Topics

Resources

Releases

No releases published

Packages

No packages published

Languages