Azul + PodResourceProfiles

samples/Readme.md

@@ -3,6 +3,7 @@ Source code for examples used in blog posts hosted at [https://www.kedify.io/blo
 
 | Directory                                  | Description                          | Used KEDA scalers                   |
 | ------------------------------------------ | ------------------------------------ | ----------------------------------- |
+| [azul-prp](./azul-prp)                     | This example demonstrates how to vertically scale a Java app running on Azul JVM | Kedify Vertical Scaling |
 | [envoy-http-scaler](./envoy-http-scaler)   | This example demonstrates how to use the already exisiting envoy to scale a deployment based on the request rate  | Kedify Envoy HTTP |
 | [grpc-responder](./grpc-responder)         | This application can be scaled by incoming gRPC traffic, including scale to zero  | Kedify HTTP |
 | [http-server](./http-server)               | This application can be scaled by incoming HTTP (or HTTPS) traffic, including scale to zero  | Kedify HTTP |

samples/azul-prp/.gitignore

@@ -0,0 +1 @@
+renaissance-mit-*.jar

samples/azul-prp/Makefile

@@ -0,0 +1,56 @@
+###############################
+#		CONSTANTS
+###############################
+IMAGE ?= ghcr.io/kedify
+VERSION	?= main
+RENAISSANCE_VERSION=0.16.0
+GIT_COMMIT ?= $(shell git rev-list -1 HEAD)
+ARCH ?= $(shell uname -m)
+ifeq ($(ARCH), x86_64)
+	ARCH=amd64
+endif
+
+
+###############################
+#		TARGETS
+###############################
+all: help
+
+##@ Build
+
+.PHONY: build-base-image
+build-base-image: ## Builds the container (base) image for current arch.
+	@$(call say,Build base container image)
+	docker build . -f base.Dockerfile -t $(IMAGE)/azul-prime:21
+
+renaissance-mit-$(RENAISSANCE_VERSION).jar:
+	wget https://github.com/renaissance-benchmarks/renaissance/releases/download/v$(RENAISSANCE_VERSION)/renaissance-mit-$(RENAISSANCE_VERSION).jar
+
+.PHONY: build-image
+build-image: build-base-image renaissance-mit-$(RENAISSANCE_VERSION).jar ## Builds the container (app) image for current arch.
+	@$(call say,Build app container image)
+	docker build . -f app.Dockerfile -t $(IMAGE)/azul-app:$(VERSION)
+
+.PHONY: build-images-multiarch
+build-images-multiarch: ## Builds the container images for amd and arm arch and pushes them to container registry.
+	@$(call say,Build container images (multiarch))
+	docker buildx build . -f base.Dockerfile --push --platform linux/amd64,linux/arm64 -t $(IMAGE)/azul-prime:21
+	docker buildx build . -f app.Dockerfile --push --platform linux/amd64,linux/arm64 -t $(IMAGE)/azul-app:$(VERSION)
+
+.PHONY: help
+help: ## Display this help.
+	@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n  make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf "  \033[36m%-24s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
+
+###############################
+#		HELPERS
+###############################
+
+ifndef NO_COLOR
+YELLOW=\033[0;33m
+# no color
+NC=\033[0m
+endif
+
+define say
+echo "\n$(shell echo "$1  " | sed s/./=/g)\n $(YELLOW)$1$(NC)\n$(shell echo "$1  " | sed s/./=/g)"
+endef

samples/azul-prp/README.md

@@ -0,0 +1,54 @@
+## Azul JVM & PodResourceProfiles
+
+This example shows how PodResourceProfiles (vertical scaling) can help with resource intensive workloads during startup. Azul JVM - Zing runs JIT compilation during application warmup that dynamically 
+optimize certain (hot) paths of code into machine code. This compilation process requires more CPU than normal mode of the Java application. At the same time, we would like to make sure,
+the users get the best experience so that we should allow the incoming traffic to the application only after it has been heated and it is performant enough.
+
+### Architecture
+
+Azul JVM can expose the information about its compilation queue using JMX. With a simple Python script we can read this number and consider the workload ready only after it is bellow some configurable
+threshold. Startup probes in Kubernetes are great fit for this use-case. They allow to check certain criteria more often during the startup and only after the startup is done, the classical readiness & liveness probes can kick in and start doing their periodic checks.
+
+To demonstrate a Java application that does some serious heavy lifting, we choose to use the [Renaissance](https://renaissance.dev/) benchmarking suite from MIT. Namely the `finagle-http` benchmark. This particular benchmark 
+sends many small Finagle HTTP requests to a Finagle HTTP server and waits for the responses. Once the benchmark run to completion, we run a sleep command.
+
+> [!IMPORTANT]
+> This feature is possible only with Kubernetes In-Place Pod Resource Updates. This feature is enabled by default since 1.33 (for older version it needs to be enabled using a feature flag).
+
+### Demo
+
+> [!TIP]
+> For trying this on k3d, create the cluster using:
+> ```bash
+> k3d cluster create in-place-updates --no-lb --k3s-arg "--disable=traefik,servicelb@server:*" --k3s-arg "--kube-apiserver-arg=feature-gates=InPlacePodVerticalScaling=true@server:*"
+> ```
+
+1. Install Kedify in K8s cluster - https://docs.kedify.io/installation/helm
+2. Deploy example application:
+
+```bash
+kubectl apply -f k8s/
+```
+
+3. Keep checking its CPU resources:
+
+```bash
+kubectl get po -lapp=heavy-workload -ojsonpath="{.items[*].spec.containers[?(.name=='main')].resources}" | jq
+```
+
+We should be able to see that after some time, it drops from `1` CPU to `0.2`.
+
+In order to check the length of the compilation Q, one can run:
+```bash
+kubectl exec -ti $(kubectl get po -lapp=heavy-workload -ojsonpath="{.items[0].metadata.name}") -- /ready.py
+JMX_HOST=127.0.0.1
+JMX_PORT=9010
+OUTSTANDING_COMPILES_THRESHOLD=500
+786
+TotalOutstandingCompiles still above threshold: 786 >= 500
+command terminated with exit code 2
+```
+
+## Conclusion
+
+By asking for right amount of compute power at right times, we allow for more effective bin-packing algorithm in Kubernetes and, if used together with tools like Karpenter, this boils down to real cost savings.

samples/azul-prp/app.Dockerfile

@@ -0,0 +1,17 @@
+# Use Azul Prime JDK 21 as the base image
+ARG BASE_IMAGE=ghcr.io/kedify/azul-prime:21
+ARG RENAISSANCE_VERSION=0.16.0
+FROM --platform=$TARGETARCH ${BASE_IMAGE}
+
+ARG RENAISSANCE_VERSION
+ENV RENAISSANCE_VERSION=${RENAISSANCE_VERSION}
+COPY renaissance-mit-*.jar /
+CMD [ \
+  "bash", "-c", \
+  "java -version && \
+  rm -f done && \
+  java -XX:+UseZingMXBeans -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Djava.rmi.server.hostname=localhost \
+  -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false \
+  -jar /renaissance-mit-${RENAISSANCE_VERSION}.jar finagle-http && \
+  touch done && echo 'I have done the benchmark, now taking a nap..' && sleep infinity" \
+]

samples/azul-prp/base.Dockerfile

@@ -0,0 +1,13 @@
+# docker buildx build . -f base.Dockerfile --push --platform linux/amd64,linux/arm64 -t ghcr.io/kedify/azul-prime:21
+# Use Azul Prime JDK 21 as the base image
+FROM --platform=$TARGETARCH azul/prime:21
+# Install Python 3 and pip
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends python3 python3-pip && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+# Install jmxquery via pip
+RUN pip3 install --no-cache-dir jmxquery
+# Verify installations
+RUN python3 --version && pip3 show jmxquery
+COPY --chmod=0755 ready.py /

samples/azul-prp/k8s/deployment.yaml

@@ -0,0 +1,45 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: heavy-workload
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: heavy-workload
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        app: heavy-workload
+      annotations:
+        prp.kedify.io/reconcile: enabled
+    spec:
+      containers:
+        - name: main
+          image: ghcr.io/kedify/azul-app:main
+          # initial resources for the workload (lifted)
+          resources:
+            requests:
+              cpu: 1
+            limits:
+              cpu: 1
+          startupProbe:
+            exec:
+              command: 
+              - sh
+              - -c
+              - "[ -f done ] || /ready.py"
+            initialDelaySeconds: 0
+            # read the compilation Q every 2 seconds
+            periodSeconds: 2
+            # python script should respond within 5 seconds
+            timeoutSeconds: 10
+            failureThreshold: 1
+            successThreshold: 1
+          readinessProbe:
+            exec:
+              command: ["test", "-f", "done"]
+            initialDelaySeconds: 10
+            periodSeconds: 10

samples/azul-prp/k8s/prp.yaml

@@ -0,0 +1,18 @@
+apiVersion: keda.kedify.io/v1alpha1
+kind: PodResourceProfile
+metadata:
+  name: heavy-workload
+spec:
+  target:
+    kind: deployment
+    name: heavy-workload
+  containerName: main
+  trigger:
+    after: containerReady
+    delay: 0s
+  # these resources will be applied for the workload, once the compilation Q is low enough or all the work has been done
+  newResources:
+    requests:
+      cpu: "0.2"
+    limits:
+      cpu: "0.2"

samples/azul-prp/ready.py

@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+import sys
+import os
+from jmxquery import JMXConnection, JMXQuery
+
+host = os.environ.get("JMX_HOST", "127.0.0.1")
+port = int(os.environ.get("JMX_PORT", "9010"))
+threshold = int(os.environ.get("OUTSTANDING_COMPILES_THRESHOLD", "500"))
+print(f"JMX_HOST={host}")
+print(f"JMX_PORT={port}")
+print(f"OUTSTANDING_COMPILES_THRESHOLD={threshold}")
+service_url = f"service:jmx:rmi:///jndi/rmi://{host}:{port}/jmxrmi"
+bean = "com.azul.zing:type=Compilation"
+attribute = "TotalOutstandingCompiles"
+
+try:
+    conn = JMXConnection(service_url)
+    queries = [JMXQuery(f"{bean}", attribute=attribute)]
+
+    metrics = conn.query(queries)
+    # Expect exactly one result
+    if not metrics:
+        print("No value returned", file=sys.stderr)
+        sys.exit(4)
+
+    value = metrics[0].value
+    print(value)
+    if value < threshold:
+      sys.exit(0)
+    else:
+      print(f"TotalOutstandingCompiles still above threshold: {value} >= {threshold}", file=sys.stderr)
+      sys.exit(2)
+except Exception as e:
+    print(f"Error: {e}", file=sys.stderr)
+    sys.exit(3)