Merge pull request #202 from guardian/nicl/simplify

Tidy repo now that Cloudquery has taken over

.github/workflows/ci.yml

@@ -62,12 +62,6 @@ jobs:
           contentDirectories: |
             cdk.out:
               - packages/cdk/cdk.out
-            github-data-fetcher:
-              - packages/github-data-fetcher/dist/github-data-fetcher.zip
-            github-lens-api:
-              - packages/github-lens-api/dist/github-lens-api.zip
-            services-api:
-              - packages/services-api/dist/services-api.zip
             repocop:
               - packages/repocop/target/scala-3.2.1/repocop.jar
 

.github/workflows/snyk.yml

@@ -6,13 +6,12 @@ on:
       - main
   workflow_dispatch:
 
-
 jobs:
   security:
     uses: guardian/.github/.github/workflows/sbt-node-snyk.yml@main
     with:
       ORG: guardian-devtools
       SKIP_NODE: false
-      EXCLUDE: cdk,common,github-data-fetcher,github-lens-api,services-api
+      EXCLUDE: cdk
     secrets:
       SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

README.md

@@ -1,15 +1,20 @@
 # The Product & Engineering Service Catalogue
-A set of APIs cataloging services deployed via AWS CloudFormation, and their related metadata.
+A set of APIs cataloging services deployed via AWS CloudFormation, and their
+related metadata.
 
-In contrast with [Prism](https://github.com/guardian/prism), which collects data from a subset of AWS resources,
-Service Catalogue offers a more complete picture of production services, as we may provision a resource that Prism doesn't know about.
+In contrast with [Prism](https://github.com/guardian/prism), which collects data
+from a subset of AWS resources, Service Catalogue offers a more complete picture
+of production services, as we may provision a resource that Prism doesn't know
+about.
 
 ## Purpose
-The Guardian has hundreds of EC2, lambda, and other services in AWS, each is built from one of thousands of GitHub repositories, by one of many Product & Engineering (P&E) teams.
+The Guardian has hundreds of EC2, lambda, and other services in AWS, each is
+built from one of thousands of GitHub repositories, by one of many Product &
+Engineering (P&E) teams.
 
 We want to be able to answer the following questions:
 
-- For P&E teams: 
+- For P&E teams:
   - Which services do I own?
   - Which services follow DevX best practice/use tooling?
   - What does each service cost?
@@ -23,55 +28,11 @@ We want to be able to answer the following questions:
   - Which teams are struggling with reliability and need more support?
   - Which services belong to specific P&E product teams
 
-## How is it done?
-The project consists of the following sub-projects:
+## Services
 
-- [**GitHub Lens API**](packages/github-lens-api/README.md): A simple API to expose the cached GitHub data we have already fetched.
-- [**GitHub Data Fetcher**](packages/github-data-fetcher/README.md): A scheduled lambda to retrieve data from GitHub to serve in the Lens API.
-- [**GitHub Services API**](packages/services-api/README.md): The service API itself, combining data from the other services in this repository.
+The following packages form part of Service Catalogue:
 
-## Local Development
-1. Get AWS credentials from Janus for the `deployTools` profile
-2. Run setup:
+* [cloudquery](packages/cloudquery/README.md): A set of Cloudquery (ECS) tasks to collect the data
+* [repo-cop](packages/repocop/README.md): A service to warn teams about issues with the repositories they own
 
-   ```sh
-   ./scripts/setup.sh
-   ```
 
-   Local configuration uses [dotenv](https://www.npmjs.com/package/dotenv), reading the file `.env` at the project root.
-   The setup script will download an initial version of this file.
-
-3. Run a service:
-
-   You can run the following commands to start the various services locally:
-
-   ```sh
-   # Run github-data-fetcher lambda
-   npm run dev-data-fetcher
-
-   # Run github-lens-api
-   npm run dev-github-lens-api
-
-   # Run services-api
-   npm run dev-services-api
-   ```
-
-### Testing
-To run all the tests, `npm run test`. To run tests under a particular path, `npm run test -w <PATH_TO_DIRECTORY>`.
-
-## Known limitations / future improvements
-- Manually provisioned resources
-
-  This repository defines a service as something provisioned in AWS defined using Infrastructure as Code (IaC) (AWS CloudFormation).
-  Those resources provisioned outside IaC are not reported against. In the future, we should collect this data to help us answer questions such as:
-  - How can I completely recreate my service in a different region from scratch?
-
-- Multiple repositories
-
-  If a service defines its application code in one repository, and infrastructure in another, 
-  the Service Catalogue APIs will _not_ offer an aggregated view.
-  It will treat the repositories as independant entities.
-
-  The ultimate solution here is for teams to migrate to a single repository, as:
-    - Multiple repositories means at least one of them is manually deployed
-    - With a single repository, Riff-Raff can be used to continuously deploy all aspects of the service

jest.config.js

@@ -21,9 +21,5 @@ module.exports = {
 	testEnvironment: 'node',
 	projects: [
 		'cdk',
-		'github-lens-api',
-		'github-data-fetcher',
-		'common',
-		'services-api',
 	].map(generateProject),
 };

package.json

@@ -4,9 +4,6 @@
   "description": "A lens into your GitHub organisation",
   "scripts": {
     "test": "npm run test --workspaces --if-present",
-    "dev-data-fetcher": "npm -w packages/github-data-fetcher run dev",
-    "dev-github-lens-api": "npm -w packages/github-lens-api run dev",
-    "dev-services-api": "npm -w packages/services-api run dev",
     "synth": "npm run synth --workspace=cdk",
     "typecheck": "npm run typecheck --workspaces",
     "build": "npm run build --workspaces --if-present",

packages/cdk/bin/cdk.ts

@@ -1,26 +1,10 @@
 import 'source-map-support/register';
 import { GuRootExperimental } from '@guardian/cdk/lib/experimental/constructs/root';
 import { CloudQuery } from '../lib/cloudquery';
-import { GithubLens } from '../lib/github-lens';
 import { Repocop } from '../lib/repocop';
-import { ServicesApi } from '../lib/services-api';
 
 const app = new GuRootExperimental();
 
-new GithubLens(app, 'GithubLens-INFRA', {
-	stack: 'deploy',
-	stage: 'INFRA',
-	env: { region: 'eu-west-1' },
-	domainName: 'github-lens.gutools.co.uk',
-});
-
-new ServicesApi(app, 'ServicesApi-INFRA', {
-	stack: 'deploy',
-	stage: 'INFRA',
-	env: { region: 'eu-west-1' },
-	domainName: 'services.gutools.co.uk',
-});
-
 new Repocop(app, 'Repocop-INFRA', {
 	stack: 'deploy',
 	stage: 'INFRA',