The AdoptOpenJDK API has 2 main components:
- Updater - Pulls data from a number of sources such as Github and Docker, parses the data and stores it to the DB.
- Front-end - Serve up responses to web requests at https://api.adoptopenjdk.net.
The updater periodically polls for new data, and stores that data into the DB. There are 2 types of refresh:
- A full refresh of all data from the various sources
- Performed when the updater starts and then every 24 hours there-after.
- A refresh of only newly added or modified files.
- Performed every 15 min
The sources of data for the api are:
The binary repositories such as:
Each of these repos contains a number of releases, inside each release are a number of assets in general for each asset there is:
- Binary archive (i.e OpenJDK11U-jdk_x64_linux_hotspot_2020-05-02-13-03.tar.gz)
- Metadata file (i.e OpenJDK11U-jdk_x64_linux_hotspot_2020-05-02-13-03.tar.gz.json)
- Checksum (i.e OpenJDK11U-jdk_x64_linux_hotspot_2020-05-02-13-03.tar.gz.sha256.txt)
The updater interacts with the Github api using the V4 GraphQL interface. Once we have obtained the data through the Github api the Upstream (for the upstream OpenJDK project) and Adopt mappers which map the Github data into Adopt API models. It does this by iterating through the list of repos and releases, for each binary asset download its metadata or checksum and parse their contents in order to extract the data. If metadata is not available then we attempt to extract the relevant data by parsing the file name.
In order to speed up access and reduce bandwidth we use Mongo as a cache for this data. When we require data such as the metadata file or checksum, that data will be provided by the cache (assuming it is present), and an asynchronous refresh of that data will be scheduled to make sure it is up to date.
The DockerHub repositories are only required for displaying stats. We pull data from the DockerHub API inside DockerStatsInterface.
To run the updater tool:
- generate the artifacts by running
mvnw clean install.
java -jar ./target/adoptopenjdk-api-v3-updater-3.0.0-SNAPSHOT-jar-with-dependencies.jar
The database stores 3 main types of data:
- Release - The raw binary data extracted from Github
- Stats - Download statistics. Updated at the end of a full refresh.
- DockerStats - Broken down into each docker repository
- GitHubStats - Broken down into each feature version
- Web-Cache - Cached data used to speed up requests
The front-end is a Quarkus application that uses OpenAPI for documentation. Data is polled from the database into memory and requests are then serviced from that dataset.
Fetch binaries and installers:
Raw asset data:
A full list of endpoints and each of the parameters can be found at https://api.adoptopenjdk.net/swagger-ui/
To run the front-end quarkus tool,
cd into the
adoptopenjdk-api-v3-frontend directory and run
../../mvnw quarkus:dev. This will then run the tool on port 8080.
NOTE: You will need to have let the Updater run a full cycle before any data is shown.