Skip to content

Commit

Permalink
Merge pull request #2071 from isaacwengler/augur-new
Browse files Browse the repository at this point in the history 
feat: add code complexity endpoint metrics
  • Loading branch information
@sgoggins
sgoggins committed Jan 1, 2023
2 parents 804b1ee + 20f7183 commit ff38f9c
Show file tree
Hide file tree
Showing 8 changed files with 26,802 additions and 227 deletions.
14 changes: 8 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Augur NEW Release v0.43.3
[![first-timers-only](https://img.shields.io/badge/first--timers--only-friendly-blue.svg?style=flat-square)](https://www.firsttimersonly.com/) We follow the [First Timers Only](https://www.firsttimersonly.com/) philosophy of tagging issues for first timers only, and walking one newcomer through the resolution process weekly. [You can find these issues tagged with "first timers only" on our issues list.](https://github.com/chaoss/augur/labels/first-timers-only).

[![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme) [![Build Docker images](https://github.com/chaoss/augur/actions/workflows/build_docker.yml/badge.svg)](https://github.com/chaoss/augur/actions/workflows/build_docker.yml) [![Hits-of-Code](https://hitsofcode.com/github/chaoss/augur?branch=main)](https://hitsofcode.com/github/chaoss/augur/view?branch=main) [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/2788/badge)](https://bestpractices.coreinfrastructure.org/projects/2788)

## NEW RELEASE ALERT!
[If you want to jump right in, updated docker build/compose and bare metal installation instructions are available here](docs/new-install.md)
### Design Description:
* Display and Sort by Number of Repos on Groups Page: https://github.com/CadenHicks/Group_1_Semester_Project/issues/7

* Add Filter Functionality to Table on Repos Page: https://github.com/CadenHicks/Group_1_Semester_Project/issues/5

Augur is now releasing a dramatically improved new version to the main branch. It is also available here: https://github.com/chaoss/augur/releases/tag/v0.43.3
- The `main` branch is a stable version of our new architecture, which features:
Expand All @@ -16,10 +17,11 @@ Augur is now releasing a dramatically improved new version to the main branch. I
- Data collection completeness assurance enabled by a structured, relational data set that is easily compared with platform API Endpoints
- The next release of the new version will include a hosted version of Augur where anyone can create an account and add repos “they care about”. If the hosted instance already has a requested organization or repository it will be added to a user’s view. If its a new repository or organization, the user will be notified that collection will take (time required for the scale of repositories added).

## What is Augur?
* Add Project Health Description to Repos Page: https://github.com/CadenHicks/Group_1_Semester_Project/issues/4

* Add Project Health Description to Insights Page: https://github.com/CadenHicks/Group_1_Semester_Project/issues/3

Augur is a software suite for collecting and measuring structured data
about [free](https://www.fsf.org/about/) and [open-source](https://opensource.org/docs/osd) software (FOSS) communities.
* Adding Trending Tab: https://github.com/CadenHicks/Group_1_Semester_Project/issues/1

We gather trace data for a group of repositories, normalize it into our data model, and provide a variety of metrics about said data. The structure of our data model enables us to synthesize data across various platforms to provide meaningful context for meaningful questions about the way these communities evolve.
Augur’s main focus is to measure the overall health and sustainability of open source projects, as these types of projects are system critical for nearly every software organization or company. We do this by gathering data about project repositories and normalizing that into our data model to provide useful metrics about your project’s health. For example, one of our metrics is Burstiness. Burstiness – how are short timeframes of intense activity, followed by a corresponding return to a typical pattern of activity, observed in a project?
Expand Down
254 changes: 254 additions & 0 deletions augur/api/routes/complexity.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,254 @@
#SPDX-License-Identifier: MIT
from flask import Response
import sqlalchemy as s
import pandas as pd
from augur.api.util import metric_metadata
import json
import os
import requests

AUGUR_API_VERSION = 'api/unstable'

def create_routes(server):

@server.app.route('/{}/complexity/project_languages'.format(AUGUR_API_VERSION), methods=["GET"])
def get_project_languages():
project_languages_sql = s.sql.text("""
SELECT
e.repo_id,
augur_data.repo.repo_git,
augur_data.repo.repo_name,
e.programming_language,
e.code_lines,
e.files
FROM
augur_data.repo,
(SELECT
d.repo_id,
d.programming_language,
SUM(d.code_lines) AS code_lines,
COUNT(*)::int AS files
FROM
(SELECT
augur_data.repo_labor.repo_id,
augur_data.repo_labor.programming_language,
augur_data.repo_labor.code_lines
FROM
augur_data.repo_labor,
( SELECT
augur_data.repo_labor.repo_id,
MAX ( data_collection_date ) AS last_collected
FROM
augur_data.repo_labor
GROUP BY augur_data.repo_labor.repo_id) recent
WHERE
augur_data.repo_labor.repo_id = recent.repo_id
AND augur_data.repo_labor.data_collection_date > recent.last_collected - (5 * interval '1 minute')) d
GROUP BY d.repo_id, d.programming_language) e
WHERE augur_data.repo.repo_id = e.repo_id
ORDER BY e.repo_id
""")
results = pd.read_sql(project_languages_sql, server.engine)
data = results.to_json(orient="records", date_format='iso', date_unit='ms')
return Response(response=data,
status=200,
mimetype="application/json")

@server.app.route('/{}/complexity/project_files'.format(AUGUR_API_VERSION), methods=["GET"])
def get_project_files():
project_files_sql = s.sql.text("""
SELECT
e.repo_id,
augur_data.repo.repo_git,
augur_data.repo.repo_name,
e.files
FROM
augur_data.repo,
(SELECT
d.repo_id,
count(*) AS files
FROM
(SELECT
augur_data.repo_labor.repo_id
FROM
augur_data.repo_labor,
( SELECT
augur_data.repo_labor.repo_id,
MAX ( data_collection_date ) AS last_collected
FROM
augur_data.repo_labor
GROUP BY augur_data.repo_labor.repo_id) recent
WHERE
augur_data.repo_labor.repo_id = recent.repo_id
AND augur_data.repo_labor.data_collection_date > recent.last_collected - (5 * interval '1 minute')) d
GROUP BY d.repo_id) e
WHERE augur_data.repo.repo_id = e.repo_id
ORDER BY e.repo_id
""")
results = pd.read_sql(project_files_sql, server.engine)
data = results.to_json(orient="records", date_format='iso', date_unit='ms')
return Response(response=data,
status=200,
mimetype="application/json")

@server.app.route('/{}/complexity/project_lines'.format(AUGUR_API_VERSION), methods=["GET"])
def get_project_lines():
project_lines_sql = s.sql.text("""
SELECT
e.repo_id,
augur_data.repo.repo_git,
augur_data.repo.repo_name,
e.total_lines,
e.average_lines
FROM
augur_data.repo,
(SELECT
d.repo_id,
SUM(d.total_lines) AS total_lines,
AVG(d.total_lines)::INT AS average_lines
FROM
(SELECT
augur_data.repo_labor.repo_id,
augur_data.repo_labor.total_lines
FROM
augur_data.repo_labor,
( SELECT
augur_data.repo_labor.repo_id,
MAX ( data_collection_date ) AS last_collected
FROM
augur_data.repo_labor
GROUP BY augur_data.repo_labor.repo_id) recent
WHERE
augur_data.repo_labor.repo_id = recent.repo_id
AND augur_data.repo_labor.data_collection_date > recent.last_collected - (5 * interval '1 minute')) d
GROUP BY d.repo_id) e
WHERE augur_data.repo.repo_id = e.repo_id
ORDER BY e.repo_id
""")
results = pd.read_sql(project_lines_sql, server.engine)
data = results.to_json(orient="records", date_format='iso', date_unit='ms')
return Response(response=data,
status=200,
mimetype="application/json")

@server.app.route('/{}/complexity/project_comment_lines'.format(AUGUR_API_VERSION), methods=["GET"])
def get_project_comment_lines():
comment_lines_sql = s.sql.text("""
SELECT
e.repo_id,
augur_data.repo.repo_git,
augur_data.repo.repo_name,
e.comment_lines,
e.avg_comment_lines
FROM
augur_data.repo,
(SELECT
d.repo_id,
SUM(d.comment_lines) AS comment_lines,
AVG(d.comment_lines)::INT AS avg_comment_lines
FROM
(SELECT
augur_data.repo_labor.repo_id,
augur_data.repo_labor.comment_lines
FROM
augur_data.repo_labor,
( SELECT
augur_data.repo_labor.repo_id,
MAX ( data_collection_date ) AS last_collected
FROM
augur_data.repo_labor
GROUP BY augur_data.repo_labor.repo_id) recent
WHERE
augur_data.repo_labor.repo_id = recent.repo_id
AND augur_data.repo_labor.data_collection_date > recent.last_collected - (5 * interval '1 minute')) d
GROUP BY d.repo_id) e
WHERE augur_data.repo.repo_id = e.repo_id
ORDER BY e.repo_id
""")
results = pd.read_sql(comment_lines_sql, server.engine)
data = results.to_json(orient="records", date_format='iso', date_unit='ms')
return Response(response=data,
status=200,
mimetype="application/json")

@server.app.route('/{}/complexity/project_blank_lines'.format(AUGUR_API_VERSION), methods=["GET"])
def get_project_blank_lines():
blank_lines_sql = s.sql.text("""
SELECT
e.repo_id,
augur_data.repo.repo_git,
augur_data.repo.repo_name,
e.blank_lines,
e.avg_blank_lines
FROM
augur_data.repo,
(SELECT
d.repo_id,
SUM(d.blank_lines) AS blank_lines,
AVG(d.blank_lines)::int AS avg_blank_lines
FROM
(SELECT
augur_data.repo_labor.repo_id,
augur_data.repo_labor.blank_lines
FROM
augur_data.repo_labor,
( SELECT
augur_data.repo_labor.repo_id,
MAX ( data_collection_date ) AS last_collected
FROM
augur_data.repo_labor
GROUP BY augur_data.repo_labor.repo_id) recent
WHERE
augur_data.repo_labor.repo_id = recent.repo_id
AND augur_data.repo_labor.data_collection_date > recent.last_collected - (5 * interval '1 minute')) d
GROUP BY d.repo_id) e
WHERE augur_data.repo.repo_id = e.repo_id
ORDER BY e.repo_id
""")
results = pd.read_sql(blank_lines_sql, server.engine)
data = results.to_json(orient="records", date_format='iso', date_unit='ms')
return Response(response=data,
status=200,
mimetype="application/json")


@server.app.route('/{}/complexity/project_file_complexity'.format(AUGUR_API_VERSION), methods=["GET"])
def get_project_file_complexity():
project_file_complexity_sql = s.sql.text("""
SELECT
e.repo_id,
augur_data.repo.repo_git,
augur_data.repo.repo_name,
e.sum_code_complexity,
e.average_code_complexity
FROM
augur_data.repo,
(SELECT
d.repo_id,
SUM(d.code_complexity) AS sum_code_complexity,
AVG(d.code_complexity)::int AS average_code_complexity
FROM
(SELECT
augur_data.repo_labor.repo_id,
augur_data.repo_labor.code_complexity
FROM
augur_data.repo_labor,
( SELECT
augur_data.repo_labor.repo_id,
MAX ( data_collection_date ) AS last_collected
FROM
augur_data.repo_labor
GROUP BY augur_data.repo_labor.repo_id) recent
WHERE
augur_data.repo_labor.repo_id = recent.repo_id
AND augur_data.repo_labor.data_collection_date > recent.last_collected - (5 * interval '1 minute')) d
GROUP BY d.repo_id) e
WHERE augur_data.repo.repo_id = e.repo_id
ORDER BY e.repo_id
""")
results = pd.read_sql(project_file_complexity_sql, server.engine)
data = results.to_json(orient="records", date_format='iso', date_unit='ms')
return Response(response=data,
status=200,
mimetype="application/json")

Loading

0 comments on commit ff38f9c

Please sign in to comment.