Skip to content
Community tool to manage open PRs for puppet modules
Ruby HTML CSS JavaScript Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
app
bin
config
db
images
lib
log
public
storage
test
tmp
.dockerignore
.gitignore
.rubocop.yml
.rubocop_todo.yml
.ruby-version
.travis.yml
Dockerfile
Gemfile
Gemfile.lock
LICENSE
Procfile
README.md
Rakefile
config.ru
docker-compose.yml
docker_push.sh
entrypoint.sh

README.md

Vox Pupuli Tasks - The Webapp for community management

Build Status AGPL v3 License microbadger layercounter display version GitHub release Docker Stars Docker Pulls Codacy Badge

Table of contents

Purpose

As a collaborator at Vox Pupuli we have basically two different kinds of main tasks:

We currently have a few tools for those jobs:

Reviewing open Pull Requests

Collaborators review a lot of code in many pull requests. But there are even more pull requests that are open but don't need any attention. A collaborator spends a lot of time to figure out which pull request actually needs attention.

One of the goals of this project is to provide a proper UI that displays filtered pull requests. Some examples:

It's not required to review code in a pull request if a merge conflict exists. If the PR is properly labeled, we can exclude it from the UI. The service gets notifications from GitHub for each activity on a PR. If a conflict appears, a label will be added. It will also automatically be removed if the conflict disappears after a rebase.

Instead of dealing with all open PRs over and over, collaborators can spend their time to reviewing pull requests that actually need it.

Some more examples are documented as open issues, in particular issue 4

Yak shaving Puppet modules

This is the second big tasks for collaborators. Update dependencies in metadata.json files, allow new Puppet versions, drop legacy operating systems. There are many many tasks that collaborators do from time to time and this project tries to make it as easy as possible or even automate stuff where it's suitable.

Usage

You can currently access a beta version at voxpupu.li. This is a MVP that we redeploy with enhancements every few days at the moment. The app uses GitHubs OAuth for authentication:

login screenshot

The application is developed by Robert 'flipez' Müller and Tim 'bastelfreak' Meusel. The current OAuth app is registed to their personal account but will soon be migrated to the Vox Pupuli GitHub organisation. You do not need to grant the application access to any repository. This OAuth setup is only used to authenticate the user.

In the future it's possible to restrict the login or certain features to people that are members of specific GitHub organisations or teams. We didn't want to reimplement a whole usermanagement so we rely on GitHub OAuth.

After the login, you see the following page:

startpage screenshot

Each yak shaving task is a row. It's prefixed with the number of modules that are in this category. For example:

single yak shaving action

You can click on each PR. The app displays all the information that GitHub provides. It will also list open pull requests. It will be possible to filter this. The filtering will also work for all open pull requests in a namespace. The design and scope is currently discussed and implemented in Issue #4.

Besides being an OAuth application, this Ruby on Rails website is also a registered GitHub Application. This means that GitHub sends notifications for user interactions to the Rails app. The app gets information about every new pull request, new label, new code or comments in a pull request and much more. We currently store those notifications in a self hosted Sentry. The displayed data in the frontend comes from polling the GitHub API and from analysing the notifications. In the future we all add more automisation to the app, which will be based on the notifications. Use cases for automisation are discussed and developed at:

Other open issues might also be good candidates for live interactions based on notifications. Please comment the open issues or raise new ones if you have crazy ideas.

Existing Automatisation

We aim to automate different use cases. Each usecase gets a dedicated milestone at GitHub to track the issues and pull requests.

Merge Conflicts - Milestone 1

At the moment, the application handles appearing and disappearing merge conflicts. Since PR #35 went live, we are able to detect if a Pull request went from a mergeable into a non-mergeable state. In this case we check if the label merge-conflicts is present in the repository. Afterwards we add it to the pull request.

bot adds label

Also our bot Account adds a comment to the pull request. GitHub does not send notifications to the author whe a label was added, but for comments.

bot adds comment

Local Setup

To start the app locally, do the following (assumes that you've ruby, bundler and yarn available, also redis needs to be started):

git clone git@github.com:voxpupuli/vox-pupuli-tasks.git
cd vox-pupuli-tasks
bundle install --jobs $(nproc) --with development test --path vendor/bundle
bundle exec yarn install --frozen-lockfile --non-interactive
export SECRET_KEY_BASE=$(bundle exec rails secret)
bundle exec rails assets:precompile
# somehow generate config/master.key
RAILS_ENV=development bundle exec rails db:migrate
bundle exec foreman start

Secrets are stored as an encrypted yaml file. You can edit them by doing:

bundle exec rails credentials:edit

This only works properly if one od the developers sent you the /config/master.key file.

Foreman will take care of the actual rails application, but it will also start sidekiq.

Production Setup

The production setup is a homage to microservices:

poo logo

The setup is deployed as docker microservices. This repository contains a docker-compose.yaml for this.

We deploy multiple containers:

Cerebro

This is a webinterface for Elasticsearch. The service is available at localhost:9001. We highly suggest that you deploy an nginx in front of it with proper TLS certiicates. To access the elasticsearch container, you can use this URL:

http://localhost:9001/#/overview?host=http:%2F%2Felasticsearch:9200

The docker-compose.yaml sets elasticsearch as FQDN for the container.

Elasticsearch

We use the semantic logger to log all rails data to elasticsearch. Logs are important, and writing them to a file in a container is a bad idea.

Kibana

Kibana is our frontend for elasticsearch. It's available at localhost on port 5601.

Sentry

ToDo: Describe how we forward errors to Sentry

GitHub App Setup

As mentioned in the usage section, this Ruby on Rails application can be registered as a GitHub App. To do this, a few things need to be configured.

User authorization callback URL

The full URL to redirect to after a user authorizes an installation. For our instance this is https://voxpupu.li/auth/github/callback

Request user authorization (OAuth) during installation

Requests that the installing user grants access to their identity during installation of the application.

This allows us to validate if a user is in a specific GitHub organisation or Team.

Webhook URL

Events will POST to this URL. For our instance this is https://voxpupu.li/incoming/github

Permissions

We need Read and write access to issues because we add/remove labels and also comment on them. More information can be found at the GitHub developer docs.

issue perms

The same applies for the Pull requests. More information can be found at the GitHub developer docs

pr perms

Events

We also need to tell GitHub which events we would like to get:

events

API docs for:

Contribution

We have a helpful rake task available to run a ruby linter. It will inform you about styleguide violations. Please execute it before you provide a pull request:

bundle exec rake rubocop

This will execute the linter. You can also advice him to automatically fix things (works often, but not on all issues):

bundle exec rake rubocop::auto_correct

License

This project is licensed under GNU Affero General Public License version 3

Sponsor

This project is sponsored by Hetzner Cloud. They provide us with free cloud instances to host the application.

hcloud

Are you interested as well in sponsoring parts of the Vox Pupuli organisation? Get in touch with the Project Management Committee.

You can’t perform that action at this time.