Skip to content
This repository has been archived by the owner. It is now read-only.
Permalink
master
Switch branches/tags
Go to file
 
 
Cannot retrieve contributors at this time

Structure

The AdoptOpenJDK API has 2 main components:

  1. Updater - Pulls data from a number of sources such as Github and Docker, parses the data and stores it to the DB.
  2. Front-end - Serve up responses to web requests at https://api.adoptopenjdk.net.

Architecture Diagrams

The diagrams have been created using the C4 model with http://diagrams.net/ .

Context

context

Container

container

Updater

The updater periodically polls for new data, and stores that data into the DB. There are 2 types of refresh:

  1. Full
    • A full refresh of all data from the various sources
    • Performed when the updater starts and then every 24 hours there-after.
  2. Incremental
    • A refresh of only newly added or modified files.
    • Performed every 15 min

The sources of data for the api are:

Github

The binary repositories such as:

Each of these repos contains a number of releases, inside each release are a number of assets in general for each asset there is:

  • Binary archive (i.e OpenJDK11U-jdk_x64_linux_hotspot_2020-05-02-13-03.tar.gz)
  • Metadata file (i.e OpenJDK11U-jdk_x64_linux_hotspot_2020-05-02-13-03.tar.gz.json)
  • Checksum (i.e OpenJDK11U-jdk_x64_linux_hotspot_2020-05-02-13-03.tar.gz.sha256.txt)

The updater interacts with the Github api using the V4 GraphQL interface. Once we have obtained the data through the Github api the Upstream (for the upstream OpenJDK project) and Adopt mappers which map the Github data into Adopt API models. It does this by iterating through the list of repos and releases, for each binary asset download its metadata or checksum and parse their contents in order to extract the data. If metadata is not available then we attempt to extract the relevant data by parsing the file name.

In order to speed up access and reduce bandwidth we use Mongo as a cache for this data. When we require data such as the metadata file or checksum, that data will be provided by the cache (assuming it is present), and an asynchronous refresh of that data will be scheduled to make sure it is up to date.

DockerHub

The DockerHub repositories are only required for displaying stats. We pull data from the DockerHub API inside DockerStatsInterface.

Running

To run the updater tool:

  • generate the artifacts by running mvnw clean install.
  • cd into the adoptopenjdk-api-v3-updater directory
  • run java -jar ./target/adoptopenjdk-api-v3-updater-3.0.0-SNAPSHOT-jar-with-dependencies.jar

Database

The database stores 3 main types of data:

  1. Release - The raw binary data extracted from Github
  2. Stats - Download statistics. Updated at the end of a full refresh.
    • DockerStats - Broken down into each docker repository
    • GitHubStats - Broken down into each feature version
  3. Web-Cache - Cached data used to speed up requests

Front-end

The front-end is a Quarkus application that uses OpenAPI for documentation. Data is polled from the database into memory and requests are then serviced from that dataset.

Examples

Fetch binaries and installers:

Raw asset data:

Release info:

Download statistics:

A full list of endpoints and each of the parameters can be found at https://api.adoptopenjdk.net/swagger-ui/

Running

To run the front-end quarkus tool, cd into the adoptopenjdk-api-v3-frontend directory and run ../../mvnw quarkus:dev. This will then run the tool on port 8080. NOTE: You will need to have let the Updater run a full cycle before any data is shown.