Operations Engineering Reports

This repository houses our GitHub repository standards reports flask application, developed and maintained by the Ministry of Justice's Operations Engineering Team. It provides a unified platform to visualise and track the compliance of GitHub repositories with the GitHub Repository Standards as defined by the Ministry of Justice.

Table of Contents

1. Prerequisites
2. Reports
3. Development
4. Deployment

Prerequisites

To develop, deploy or run this app you will need to install:

• Python >= 3.11
• Docker and Docker Compose (Optional, for running a local AWS DynamoDB instance or development/production servers with Docker Compose)
• Helm and Kubectl (Optional, for deploying to Cloud Platform)

Note: This project uses AWS DynamoDB for data storage. While it's not necessary for basic development, you might want to set up AWS DynamoDB Local for a full-fledged local testing environment.

Reports

At the Ministry of Justice, all our source code is open by default and stored on GitHub. We believe in transparency and accessibility in our codebase, and we've established some key standards to ensure we uphold these principles. The guidelines outlined in the technical standards documentation help us maintain a secure, transparent and collaborative codebase. Adhering to these standards is crucial to ensure that our GitHub repositories remain accessible, safe, and easy to contribute to.

For more information, please review the detailed MoJ GitHub standards here.

What are reports?

GitHub repository standards reports are generated by the Operations Engineering team to track the compliance of GitHub repositories with the MoJ GitHub organisation. These reports are generated using a combination of GitHub's REST and GraphQL API. A report is generated for each repository, and contains the following information:

• Repository name: The name of the GitHub repository.
• Repository URL: The GitHub URL of the repository.
• Repository owner: Whether or not the repository is owned by a team or an individual.
• Repository description: A boolean value that indicates whether a repository has a description.
• Repository visibility: Public or private.
• Repository compliance status: A boolean value that indicates whether a repository is compliant with the MoJ GitHub standards.
• Repository compliance status reason: A list of reasons why a repository is non-compliant with the MoJ GitHub standards.

The report is then used to identify commonalities and trends across the organisation, and to identify areas of improvement. The Operations Engineering team uses this information to provide guidance and support to teams and projects, and to ensure that our GitHub repositories remain accessible, safe, and easy to contribute to.

If you wish to add a criteria to the report, please create an issue or speak to someone in the #ask-operations-engineering Slack channel.

How do you generate them?

A GitHub action runs daily to fetch all repositories in the Ministry of Justice GitHub organisation, and to generate a report for each repository. The report is then stored in an AWS DynamoDB table, and is used to generate the report dashboard.

This can be done manually by running the following command:

git clone https://github.com/ministryofjustice/operations-engineering.git
cd operations-engineering
pip install -r python/requirements.txt
python python/report_on_repository_standards.py --org ministryofjustice --oauth-token <your-github-token> --url <endpoint-url> --endpoint /api/v2/update-github-reports

How can I generate test reports?

To generate test data, you can use the ministryofjustice-test organisation. This organisation contains a number of test repositories that can be used to generate test reports.

Development

Running the app locally with Docker Compose and AWS DynamoDB Local

A makefile command is provided to run the app locally with Docker Compose and AWS DynamoDB Local. This will start the app and a local AWS DynamoDB instance.

make dev

Linting

To run the linter, run the following command:

make lint

Testing

To run the tests, run the following command:

make test

Formatting

To run the formatter, run the following command:

make format

Deployment

Development environment

The development namespace for this project is called operations-engineering-reports-dev.

To deploy the app to the development namespace, simply push to the main branch. The deployment will trigger a helm upgrade command to update the deployment.

If you wish to deploy manually, follow the steps outlined in the deploy-to-dev GitHub Action.

You can see the development app running at: https://operations-engineering-reports-dev.cloud-platform.service.justice.gov.uk/

and access Cloud Platform's namespace using:

kubectl get pods -n operations-engineering-reports-dev

or using the Helm commands:

helm list -n operations-engineering-reports-dev
helm history -n operations-engineering-reports-dev operations-engineering-reports-dev

Production environment

The production namespace on Cloud Platform for this project is called operations-engineering-reports-prod.

To deploy the app to the production namespace do the following:

• push to the main branch.
• create a new tag using git tag vn.n.n where n.n.n is the version number. Please follow semantic versioning.
• push the tag to the remote repository using git push origin --tags

This will trigger a GitHub Action that will create a release and deploy the app to the production namespace.

You can see the production app running at: https://operations-engineering-reports-prod.cloud-platform.service.justice.gov.uk/

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contact Us

If you have any questions or need further clarification, feel free to ask in the #ask-operations-engineering channel on Slack or email us at operations-engineering@digital.justice.gov.uk.