fix metrics for k8s setup (#2473)

* fix metrics for k8s setup Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix lint error --------- Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
pytorch · Aug 24, 2023 · b37296d · b37296d
1 parent 921f423
commit b37296d
Show file tree

Hide file tree

Showing 5 changed files with 26 additions and 13 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -12,6 +12,7 @@ repos:
       - id: check-json
       - id: check-toml
       - id: check-yaml
+        args: [--allow-multiple-documents, --unsafe]
       - id: end-of-file-fixer
       - id: mixed-line-ending
       - id: trailing-whitespace

diff --git a/kubernetes/EKS/config.properties b/kubernetes/EKS/config.properties
@@ -1,6 +1,8 @@
 inference_address=http://0.0.0.0:8080
 management_address=http://0.0.0.0:8081
 metrics_address=http://0.0.0.0:8082
+enable_metrics_api=true
+metrics_mode=prometheus
 NUM_WORKERS=1
 number_of_gpu=1
 number_of_netty_threads=32

diff --git a/kubernetes/GKE/config.properties b/kubernetes/GKE/config.properties
@@ -1,6 +1,8 @@
 inference_address=http://0.0.0.0:8080
 management_address=http://0.0.0.0:8081
 metrics_address=http://0.0.0.0:8082
+enable_metrics_api=true
+metrics_mode=prometheus
 NUM_WORKERS=1
 number_of_gpu=1
 number_of_netty_threads=32

diff --git a/kubernetes/Helm/templates/torchserve.yaml b/kubernetes/Helm/templates/torchserve.yaml
@@ -6,13 +6,14 @@ metadata:
   labels:
     app: torchserve
   annotations:
-    prometheus.io/scrape: 'true'
+    prometheus.io/scrape: "true"
+    prometheus.io/path: /metrics
     prometheus.io/port: '8082'
 spec:
   ports:
   - name: preds
     port: {{ .Values.torchserve.inference_port }}
-    targetPort: ts 
+    targetPort: ts
   - name: mdl
     port: {{ .Values.torchserve.management_port }}
     targetPort: ts-management

diff --git a/kubernetes/README.md b/kubernetes/README.md
@@ -1,5 +1,5 @@
 # Torchserve on Kubernetes
-  
+
 ## Overview
 
 This page demonstrates a Torchserve deployment in Kubernetes using Helm Charts. It uses the DockerHub Torchserve Image for the pods and a PersistentVolume for storing config / model files.
@@ -66,7 +66,7 @@ persitant_volume:
   size: 1Gi
 ```
 
-To install Torchserve run ```helm install ts .```  
+To install Torchserve run ```helm install ts .```
 
 ```bash
 ubuntu@ip-172-31-50-36:~/serve/kubernetes/Helm$ helm install ts .
@@ -207,12 +207,6 @@ helm repo add grafana https://grafana.github.io/helm-charts
 helm install grafana grafana/grafana
 ```
 
-Get admin user password by running:
-
-```bash
-kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
-```
-
 ## Add prometheus as data source in grafana
 
 ```bash
@@ -233,8 +227,6 @@ kubectl get pod prometheus-server-f8677599b-xmjbt -o jsonpath='{.status.podIPs[0
 192.168.52.141
 ```
 
-![Add data source](images/grafana_datasource.png)
-
 ## Expose grafana with loadbalancer
 
 ```bash
@@ -243,9 +235,24 @@ kubectl patch service grafana -p '{"spec": {"type": "LoadBalancer"}}'
 kubectl get svc grafana -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
 ```
 
+Get admin user password by running:
+
+```bash
+kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
+```
+
 ## Login to grafana
 
-`<http://your.grafana.loadbalancer.address:3000>`
+<b>Username:</b> admin
+<b>Password:</b> <--The password got from previous step-->
+
+Open Grafana in browser with the url - `<http://your.grafana.loadbalancer.address:3000>`
+
+Add Prometheus data source
+
+![Add data source](images/grafana_datasource.png)
+
+The TS metrics will be available in Prometheus for Grafana dashboards.
 
 ## Logging