From 7c6542ef27b1ab0c614a552dff12bfd9c50fc7d8 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Sun, 3 May 2020 11:36:47 +0100 Subject: [PATCH 1/4] Update contributing + some minor fixes --- .github/workflows/build.yml | 4 +- CONTRIBUTING.md | 115 +++++++++++++++++++++-- adoptopenjdk-api-v3-models/pom.xml | 4 - adoptopenjdk-api-v3-updater/pom.xml | 4 - gradle/wrapper/gradle-wrapper.properties | 1 - pom.xml | 2 +- 6 files changed, 108 insertions(+), 22 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 56c4209b..c1f6710a 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -17,7 +17,7 @@ jobs: - uses: actions/checkout@v1 - name: ktlint - run: curl -sSLO https://github.com/pinterest/ktlint/releases/download/0.35.0/ktlint && chmod a+x ktlint && ./ktlint + run: curl -sSLO https://github.com/pinterest/ktlint/releases/download/0.36.0/ktlint && chmod a+x ktlint && ./ktlint - name: Build app run: ./mvnw --batch-mode clean install jacoco:report jacoco:report-aggregate @@ -30,7 +30,7 @@ jobs: run: | cd adoptopenjdk-api-v3-frontend unzip target/adoptopenjdk-api-v3-frontend-*-runner.jar META-INF/quarkus-generated-openapi-doc.YAML - ../mvnw --batch-mode org.openapitools:openapi-generator-maven-plugin:4.2.2:generate \ + ../mvnw --batch-mode org.openapitools:openapi-generator-maven-plugin:4.3.0:generate \ -Dopenapi.generator.maven.plugin.inputSpec=META-INF/quarkus-generated-openapi-doc.YAML - name: Zip Javascript client diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e1bee8d7..fa3e1574 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -3,31 +3,126 @@ ## Overview The AdoptOpenJDK API V3 is a Kotlin application (fronted by Swagger and OpenAPI) which makes -calls via the GitHub API inb order to retrieve AdoptOpenJDK binaries and metadata. +calls via the GitHub API in order to retrieve AdoptOpenJDK binaries and metadata. Since the GitHub API is rate limited we use MongoDB as a caching mechanism. +## Source code management and branching + +We treat the AdoptOpenJDK org repository as the canonical repository for deploying the API from. +We use the `staging` branch to trial any changes in a Production like environment and then +eventually merge into `master` for a real Production deployment. + +**NOTE** Please ensure for any significant change that you Pull Request to the `staging` branch + ## Build ### Pre-Requisites -Java 11 is required to build. +Java 11 is a requirement to build to project. ### Build Tool -We use a Maven Wrapper (mvnw) to ensure that there's a consistent repeatable build. +We use a Maven Wrapper (mvnw) to ensure that there's a consistent, repeatable build. The +[POM File](./pom.xml) is the place to start. + +**NOTE:** We use a multi-module project structure. The root level POM contains a majority +of the configuration that the children inherit from. + +### Build Command -### Build +To perform a full build and test you run the following: `./mvnw clean install` -## Usage +If you wish to view all fo the Maven reporting about the project you run the following: + +`./mvnw clean install site` + +## Testing + +**WARN** This API is critical to the success of AdoptOpenJDK, so we have a strong preference that +for any new functionality, you must add tests. + +### Code Coverage + +TBD + +## Continuous Integration (CI) + +### Pull Requests + +There is a [Travis YAML](.github\workflows\build.yml) file which the Travis CI +system in GitHub uses to build and test a Pull Request. + +**NOTE:** Please update the dependencies in this file if you have changed the versions of: + +* The JDK +* ktlint +* openapi-generator-maven-plugin + +## API Definition and Usage + +We use Swagger to document the API. The Swagger documentation can be viewed at: [swagger-ui](https://api.adoptopenjdk.net/swagger-ui). +The Open API definition for this can be viewed at [openapi](https://api.adoptopenjdk.net/openapi). + +## Deployment / Continuous Deployment (CD) + +You can choose to deploy this API where you wish, for AdoptOpenJDK we use Continuous Deployment. + +### AdoptOpenJDK + +For AdoptOpenJDK, this API deploys to Red Hat OpenShift and is front ended by Cloud Flare as a CDN + +The Jenkins [AdoptOpenJDK CI Server](https://ci.adoptopenjdk.net) will automatically +deploy Pull Requests to OpenShift to Staging (the `staging` branch) or Production (the `master` branch.) + +## Code Architecture and Code + +The AdoptOpenJDK API V3 is a Kotlin application (fronted by Swagger and OpenAPI) which makes +calls via the GitHub API in order to retrieve AdoptOpenJDK binaries and metadata. + +Since the GitHub API is rate limited we use MongoDB as a caching mechanism. + +### Code Architecture + +We split the API into 4 modules: + +1. [models](adoptopenjdk-api-v3-models) - The core domain modeling for the API. +2. [persistence](adoptopenjdk-api-v3-persistence) - The layer that interacts with the Mongo DB cache. +3. [updater](adoptopenjdk-api-v3-updater) - The layer that interacts with the GitHub repositories (where we store JDK / JRE binaries). +4. [frontend](adoptopenjdk-api-v3-frontend) - The layer that responds to requests. + +### Models + +Contains the domain modeling for the API, including download stats. + +Contains the important [VersionParser](adoptopenjdk-api-v3-models/src/main/kotlin/net/adoptopenjdk/api/v3/parser/VersionParser.kt) and +its corresponding [VersionParserTest](adoptopenjdk-api-v3-models/src/test/kotlin/net/adoptopenjdk/api/VersionParserTest.kt). + +### Persistence + +The layer that interacts with the Mongo DB cache. + +### Updater + +The layer that interacts with the AdoptOpenJDK JDK/JRE binary repositories on GitHub. + +### Frontend + +Contains the important [Platforms JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/platforms.json) and +[Variants JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/variants.json). + +## Common Tasks + +In this section we list some common tasks and where to start. -The api is documented via swagger. The swagger documentation can be viewed at: [swagger-ui](https://api.adoptopenjdk.net/swagger-ui). -The open api definition for this can be viewed at [openapi](https://api.adoptopenjdk.net/openapi). +### I want support a new version string -## Deployment +If you need to add/edit/remove a supported version string then you need to update the [VersionParser](adoptopenjdk-api-v3-models/src/main/kotlin/net/adoptopenjdk/api/v3/parser/VersionParser.kt) and +its corresponding [VersionParserTest](adoptopenjdk-api-v3-models/src/test/kotlin/net/adoptopenjdk/api/VersionParserTest.kt). -The API is deployed to Red Hat OpenShift. +### I want to add a new variant such as OpenJDK's project amber or -The API is CDN fronted by Cloud Flare +You'll need to start at the [Platforms JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/platforms.json) and +[Variants JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/variants.json). \ No newline at end of file diff --git a/adoptopenjdk-api-v3-models/pom.xml b/adoptopenjdk-api-v3-models/pom.xml index 034a6ba3..0e49426f 100644 --- a/adoptopenjdk-api-v3-models/pom.xml +++ b/adoptopenjdk-api-v3-models/pom.xml @@ -29,8 +29,6 @@ org.apache.maven maven-artifact - - org.junit.jupiter junit-jupiter-api @@ -46,12 +44,10 @@ src/main/kotlin src/test/kotlin - org.apache.maven.plugins maven-surefire-plugin - 2.22.2 diff --git a/adoptopenjdk-api-v3-updater/pom.xml b/adoptopenjdk-api-v3-updater/pom.xml index 51e3092d..7210ba7f 100644 --- a/adoptopenjdk-api-v3-updater/pom.xml +++ b/adoptopenjdk-api-v3-updater/pom.xml @@ -21,10 +21,6 @@ org.apache.httpcomponents httpasyncclient - org.apache.httpcomponents httpcore diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties index 33dc4f1f..4c463175 100644 --- a/gradle/wrapper/gradle-wrapper.properties +++ b/gradle/wrapper/gradle-wrapper.properties @@ -2,5 +2,4 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists zipStoreBase=GRADLE_USER_HOME zipStorePath=wrapper/dists -#distributionUrl=https\://services.gradle.org/distributions/gradle-5.5.1-bin.zip distributionUrl=https\://services.gradle.org/distributions/gradle-5.6.4-bin.zip diff --git a/pom.xml b/pom.xml index 048c6663..e2b24914 100644 --- a/pom.xml +++ b/pom.xml @@ -340,6 +340,7 @@ maven-filtering 3.1.1 + org.apache.maven.plugins maven-install-plugin @@ -690,7 +691,6 @@ - report report-aggregate From 468e8a8e38bbfb17dd2022f4de8ceb799b19f617 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Sun, 3 May 2020 11:40:36 +0100 Subject: [PATCH 2/4] Update PR template --- .github/PULL_REQUEST_TEMPLATE.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index fbc5247b..6d89c97d 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -3,8 +3,9 @@ Thank you for your pull request. Please provide a description above and review the requirements below. --> -##### Checklist +# Checklist -- [ ] `mvn clean package` build and test completes -- [ ] documentation is changed or added \ No newline at end of file +- [ ] You added tests to cover the change +- [ ] `mvn clean install` build and test completes +- [ ] You changed or added to the documentation \ No newline at end of file From 31a91407fad19e11dd2d5a84f45432ce176dbc77 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Mon, 4 May 2020 12:30:33 +0100 Subject: [PATCH 3/4] Update CONTRIBUTING.md Co-authored-by: Nick Ebbitt --- CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fa3e1574..96799578 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,7 +19,7 @@ eventually merge into `master` for a real Production deployment. ### Pre-Requisites -Java 11 is a requirement to build to project. +Java 11 is a requirement to build the project. ### Build Tool @@ -125,4 +125,4 @@ its corresponding [VersionParserTest](adoptopenjdk-api-v3-models/src/test/kotlin ### I want to add a new variant such as OpenJDK's project amber or You'll need to start at the [Platforms JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/platforms.json) and -[Variants JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/variants.json). \ No newline at end of file +[Variants JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/variants.json). From 0c0a20967d502d4a6c5d96c43293493fbba515ff Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Mon, 4 May 2020 12:34:35 +0100 Subject: [PATCH 4/4] Fixes after review --- CONTRIBUTING.md | 41 +++++++---------------------------------- 1 file changed, 7 insertions(+), 34 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fa3e1574..2c09a01c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,7 +19,7 @@ eventually merge into `master` for a real Production deployment. ### Pre-Requisites -Java 11 is a requirement to build to project. +Java 11 is a requirement to build the project. ### Build Tool @@ -35,14 +35,14 @@ To perform a full build and test you run the following: `./mvnw clean install` -If you wish to view all fo the Maven reporting about the project you run the following: +If you wish to view all of the Maven reporting about the project you run the following: `./mvnw clean install site` ## Testing -**WARN** This API is critical to the success of AdoptOpenJDK, so we have a strong preference that -for any new functionality, you must add tests. +**WARN** This API is critical to the success of AdoptOpenJDK therefore it is +essential that tests are provided for all new functionality. ### Code Coverage @@ -52,8 +52,8 @@ TBD ### Pull Requests -There is a [Travis YAML](.github\workflows\build.yml) file which the Travis CI -system in GitHub uses to build and test a Pull Request. +There is a [GitHub Action](.github\workflows\build.yml) file which the CI system +in GitHub uses to build and test a Pull Request. **NOTE:** Please update the dependencies in this file if you have changed the versions of: @@ -84,34 +84,7 @@ calls via the GitHub API in order to retrieve AdoptOpenJDK binaries and metadata Since the GitHub API is rate limited we use MongoDB as a caching mechanism. -### Code Architecture - -We split the API into 4 modules: - -1. [models](adoptopenjdk-api-v3-models) - The core domain modeling for the API. -2. [persistence](adoptopenjdk-api-v3-persistence) - The layer that interacts with the Mongo DB cache. -3. [updater](adoptopenjdk-api-v3-updater) - The layer that interacts with the GitHub repositories (where we store JDK / JRE binaries). -4. [frontend](adoptopenjdk-api-v3-frontend) - The layer that responds to requests. - -### Models - -Contains the domain modeling for the API, including download stats. - -Contains the important [VersionParser](adoptopenjdk-api-v3-models/src/main/kotlin/net/adoptopenjdk/api/v3/parser/VersionParser.kt) and -its corresponding [VersionParserTest](adoptopenjdk-api-v3-models/src/test/kotlin/net/adoptopenjdk/api/VersionParserTest.kt). - -### Persistence - -The layer that interacts with the Mongo DB cache. - -### Updater - -The layer that interacts with the AdoptOpenJDK JDK/JRE binary repositories on GitHub. - -### Frontend - -Contains the important [Platforms JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/platforms.json) and -[Variants JSON](adoptopenjdk-api-v3-frontend/src/main/resources/JSON/variants.json). +See [./docs/STRUCTURE.md](Code Structure) doc for more details. ## Common Tasks