[Prometheus] unexpected end of JSON input

[ch9hn]

Hello,
we are using the Prometheus Integration to scrape Metrics from an endpoint which is working quite well with Metricbeat, but the same endpoint is not working with the Prometheus integration.

Metricsbeat Config:

- module: prometheus
  period: 10s
  environment: ["mysql-production", "xx-xx-xx"]
  tags: ["mysql-production", "mysql", "scw-metrics", "scw-mysql"]
  metricsets: ["query"]
  hosts: ["https://metrics.prd.obs.fr-par.scw.cloud/api/prom"]
  headers:
    x-token: ${SCW_PROM_TOKEN}
  queries:
    - name: "mysql_cpu"
      path: "/api/v1/query"
      params:
        query: '100 - avg by (instance) (irate(rdb_instance_mysql_node_cpu_seconds_total{resource_id="xx-xx-xx-xx", mode="idle"}[5m])) * 100'

We implemented the same config in the Prometheus Integration Query field:

Collect Prometheus Metrics => ON
HTTP config options: headers
x-token: <token>
Prometheus query metrics => ON
Host: https://metrics.prd.obs.fr-par.scw.cloud/api/prom

Queries:

- name: "mysql_cpu"
  path: "/api/v1/query"
  params:
    query: '100 - avg by (instance) (irate(rdb_instance_mysql_node_cpu_seconds_total{resource_id="xx-xx-xx-xx", mode="idle"}[5m])) * 100'

Leader election: ON

Error:

error.message:
error parsing response from: https://metrics.prd.obs.fr-par.scw.cloud/api/prom/api/v1/query?query=%28irate%28rdb_instance_mysql_node_cpu_seconds_total%7Bresource_id%3D%xx-xx-xx-xx%22%2C+mode%3D%22idle%22%7D%5B5m%5D%29%29: Failed to parse api response: unexpected end of JSON input

Ive inserted the exact same path into Postman and got a response with success:

{
    "status": "success",
    "data": {
        "resultType": "vector",
        "result": [
            {
                "metric": {
                    "cluster_type": "cluster",
                    "cpu": "0",
                    "instance": "cluster main xxx",
                    "mode": "idle",
                    "node_role": "main",
                    "resource_id": "xx-xx-xx-xx",
                    "resource_name": "mysql-name"
                },
                "value": [
                    1675452321.505,
                    "0.9931666666719442"
                ]
            }, 
            ....

[botelastic]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!