[question] How do I ingest Prometheus metric txt files into VictoriaMetrics

[anjakefala]

The documentation lead to me to believe that it should be possible to backfill with metrics from Prometheus exporters:

Supports metrics’ scraping, ingestion and backfilling via the following protocols:
Metrics from Prometheus exporters such as node_exporter. See these docs for details.

We have txt files which contain the raw Prometheus metrics in Prometheus metric format. i.e.

up{instance="foo:1337", job="node", origin="foo"} 1.000000 1594285791544

However, clicking on the linked documentation did not really help me figure out how to ingest these files. I see documentation for scraping node exporter targets, but I am confused from reading it how I ingest metrics these txt files that already exist. /import does not work because it only accepts json files. It cannot parse these.

[valyala]

VictoriaMetrics doesn't have a handler for importing data in Prometheus format. You need to scrape it by exposing it via Prometheus exporter - see https://victoriametrics.github.io/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details. Note that you should set honor_timestamps: true in scrape_config in order to honor the given timestamps.

See also backfilling docs about importing historical data into VictoriaMetrics.

[anjakefala]

You need to scrape it by exposing it via Prometheus exporter - see https://victoriametrics.github.io/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details.

Thanks for the swift response! I saw those documents, but had trouble understanding them. The confusion is that I do not understand what is meant by the words scrape and exposing.

[valyala]

scraping means collecting metrics from Prometheus exporters such as node_exporter. VictoriaMetrics can be configured to collect such metrics if -promscrape.config command-lime arg points to Prometheus config with scrape targets (i.e. addresses of Prometheus exporters to collect metrics from).
exposing means returning metrics in Prometheus format be http server when /metrics page is requested.

BTW, the next version of VictoriaMetrics will support data ingestion in Prometheus format via /api/v1/import/prometheus. See more details at https://victoriametrics.github.io/#how-to-import-data-in-prometheus-exposition-format . You can try this feature by building VictoriaMetrics from sources. See https://victoriametrics.github.io/#how-to-build-from-sources

[anjakefala]

Thanks, @valyala! I will give it a try.

By the way, I really like that in the docs you include a "verifying the ingestion worked" section.

[anjakefala]

Great news, @valyala! It totally works. Thank you so much for this feature.

[anjakefala]

Hi @valyala, as an open source maintainer, I feel like the scum of the Earth asking this question.

Do you know which version of VM this feature is slated for? I see that you have a very rapid/frequent release process. =)

[valyala]

Do you know which version of VM this feature is slated for?

This will be included in v1.38.1 release. I'll post a message here when the release will be published (until the end of this week).

Re-opening this issue until v1.38.1 is released.

[valyala]

@anjakefala , VictoriaMetrics v1.38.1 has been released. It supports /api/v1/import/prometheus handler for importing data in Prometheus exposition format. Closing this issue as done.

[anjakefala]

👏 🎉