Use only queries.yml

[potzkovge]

Hi,
we have one postgres server with multiple databases in it, if you run SELECT * FROM pg_stat_user_tables;, each database will give you a different result. Since i can configure only one DATA_SOURCE_URI i have to use multiple instances of the exporter. However this will lead to duplication in other metrics like pg_database_size which i need only once.

The question is:
Is there a feature to only use the metrics defined in the queries.yml and ignore the rest?

Regards,
Georgi

[wrouesnel]

This feature is on the agenda in the libraryize_mk2 branch. All the queries will be moving into YAML files which are pre-shipped, and there'll be a flag to disable them.

[potzkovge]

Thank you that's good news! 🎉

[pedroj84]

Hi wrouesnel,
Is there any news about this feature? I am struggling with queries.yaml file trying to only parse what I really need, but I get errors, and not sure if this feature is already implemented.

I share with you my queries.yaml (not sure if this is the correct way to build one, as I did not get so much info about this)

pg_database:
query: " SELECT pg_database.datname, pg_database_size(pg_database.datname) as size FROM pg_database"
metrics:
- datname:
usage: "LABEL"
description: "Name of the database"
- size:
usage: "GAUGE"
description: "Disk space used by the database"
node_filesystem_size:
query: " SELECT * from metrics_labels WHERE metric_name='node_filesystem_size'"
metrics:
- job:
usage: "LABEL"
description: "Name of the job"
- name:
usage: "LABEL"
description: "Instance name"
- device:
usage: "LABEL"
description: "Device name"
- fstype:
usage: "LABEL"
description: "FileSystem name"
- status:
usage: "LABEL"
description: "Status"
- project:
usage: "LABEL"
description: "Project name"
- instance:
usage: "LABEL"
description: "Instance name"
- subsystem:
usage: "LABEL"
description: "Subsystem name"
- mountpoint:
usage: "GAUGE"
description: "Path mountpoint"
- environment:
usage: "LABEL"
description: "Environment"

And I also share the error retrieved:
postgres_exporter.sh: time="2018-06-21T10:22:06Z" level=info msg="Could not parse []byte: strconv.ParseFloat: parsing "{\"job\": \"node pre\", \"name\": \"XCUAMIDAM001\", \"device\": \"/dev/mapper/dadesvg-loglv\", \"fstype\": \"ext4\", \"status\": \"stopped\", \"project\": \"XXXX\", \"instance\": \"XXXXX\", \"mountpoint\": \"/log\", \"environment\": \"pre\"}": invalid syntax" source="postgres_exporter.go:627"
Jun 21 10:22:07 XXXXXXX postgres_exporter.sh: time="2018-06-21T10:22:06Z" level=info msg="Could not parse string: strconv.ParseFloat: parsing "node_filesystem_size": invalid syntax" source="postgres_exporter.go:634"

Here also my postgres_exporter version:
postgres_exporter v0.4.6-2-g3f90528-dirty (built with go1.10.2)

Best Regards

[wrouesnel]

@pedroj84 I haven't had much time to work on this lately, but your current problem isn't related to that it looks like - you have a gauge type defined for mountpoint which is a string, so it's erroring because of that I would suspect.

[pedroj84]

Many thanks for your answer:
I expected somehow this kind of answer, many thanks for giving me the light.

On the other hand I am facing a very funny thing which is when I run the bin running out default provided system metrics and pointing to a default queries.yaml file already provided it still passes the same metrics. ;)

Thanks

[wrouesnel]

As it is now, most of the postgres metrics are hardcoded. queries.yml lets you add extra metrics, but you always get the base set if they're available.

[pedroj84]

Cool!! many thanks! This clarifies my situation right now.
Many thanks mate ;)

Cheers

[itmecho]

Also interested in this feature as the current version exposes pg_settings values and the server version. It would be great if the disable-default-metrics flag disabled all pg metrics so that only the metrics specified in queries.yml are used, avoiding any unwanted exposed data.

[wrouesnel]

This is now possible, so closing this issue.