Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using timestamp from api as timestamp in promehteous metric #80

Closed
mostafashr opened this issue Feb 12, 2021 · 5 comments
Closed

Using timestamp from api as timestamp in promehteous metric #80

mostafashr opened this issue Feb 12, 2021 · 5 comments

Comments

@mostafashr
Copy link

Is there a way to use timestamp from the api data as timestamp in output result? in promehteous documentation in instrumentation part there is :

metric_name [
  "{" label_name "=" `"` label_value `"` { "," label_name "=" `"` label_value `"` } [ "," ] "}"
] value [ timestamp ]

I have this data from my api:

{
  "data": {
    "object1": {
      "label": "test1",
      "value": 25551.671688,
      "timestamp": 1613115501
    },
    "object2": {
      "label": "test1"
      "value": 20630.985927,
      "timestamp": 1613115603
    }
}

is it possible to get output like this?

example_metrics{label="test1"}   20630.985927   1613115603
example_metrics{label="test1"}   25551.671688   1613115501
@SuperQ
Copy link
Contributor

SuperQ commented Feb 12, 2021

It's possible, but usually not what you want. Timestamps are meant for TSDB federation. They disable staleness handling and can break monitoring if you're not careful.

@mostafashr
Copy link
Author

mostafashr commented Feb 13, 2021

I couldn't find an example config to make it happen. would you please help me?
my example config is :

---
metrics:
- name: example
  path: "{ .data }"
  help: Example 
  labels:
    label: '{.label}' 
  values:
    metrics: '{.value}'

How should I add the timestamp to the config?

@rustycl0ck
Copy link
Member

I think what @SuperQ was trying to convey was that Prometheus supports the concept of creating metrics with explicit timestamp.

This exporter however, does not currently support providing external timestamps with the scrapes.

fatpat added a commit to fatpat/json_exporter that referenced this issue May 5, 2021
fatpat added a commit to fatpat/json_exporter that referenced this issue May 5, 2021
This fixes prometheus-community#80

Signed-off-by: Jérôme LOYET <822436+fatpat@users.noreply.github.com>
fatpat added a commit to fatpat/json_exporter that referenced this issue May 5, 2021
This fixes prometheus-community#80

Signed-off-by: Jérôme LOYET <822436+fatpat@users.noreply.github.com>
fatpat added a commit to fatpat/json_exporter that referenced this issue May 5, 2021
This fixes prometheus-community#80

Signed-off-by: Jérôme LOYET <822436+fatpat@users.noreply.github.com>
@fatpat
Copy link

fatpat commented May 5, 2021

See the PR #97 which adds the option to fetch the timestamp from the json:

---
metrics:
- name: example
  path: "{ .data }"
  timestamp: "{ .date }"
  help: Example 
  labels:
    label: '{.label}' 
  values:
    metrics: '{.value}'

fatpat added a commit to fatpat/json_exporter that referenced this issue May 5, 2021
This fixes prometheus-community#80

Signed-off-by: Jérôme LOYET <822436+fatpat@users.noreply.github.com>
@rustycl0ck
Copy link
Member

From the discussion on #97, a possible way to do this can be to export the timestamp as a metric. If you can send a PR for this, it can be reviewed.

You can also try to use the feature from #69 if your server supports providing metrics for a given time period (for example, scrape every minute and get the metrics in the time period: from now - 1m to now).

Modifying the timestamps can break monitoring in case of inaccurate data, therefore it is not an option.

janphkre added a commit to janphkre/json_exporter that referenced this issue Jul 11, 2022
based on prometheus-community#97 and prometheus-community#80 this provides the posibility to use a metric that has a unix style timestamp as the timestamp of the scraped metric
janphkre added a commit to janphkre/json_exporter that referenced this issue Jul 11, 2022
based on prometheus-community#97 and prometheus-community#80 this provides the posibility to use a metric that has a unix style timestamp as the timestamp of the scraped metric

When deserializing objects we need to take the key json path into account as well like we would do for all the values as well. This allows collections to be defined still with each entry having a separate timestamp (e.g. list of time-stamped log messages).

Update examples for timestamp

Update Readme about staleness for custom timestamps
janphkre added a commit to janphkre/json_exporter that referenced this issue Jul 11, 2022
based on prometheus-community#97 and prometheus-community#80 this provides the posibility to use a metric that has a unix style timestamp as the timestamp of the scraped metric

When deserializing objects we need to take the key json path into account as well like we would do for all the values as well. This allows collections to be defined still with each entry having a separate timestamp (e.g. list of time-stamped log messages).

Update examples for timestamp

Update Readme about staleness for custom timestamps

Signed-off-by: Jan Phillip Kretzschmar <janphkre@gmx.de>
janphkre added a commit to janphkre/json_exporter that referenced this issue Jul 11, 2022
based on prometheus-community#97 and prometheus-community#80 this provides the posibility to use a metric that has a unix style timestamp as the timestamp of the scraped metric

When deserializing objects we need to take the key json path into account as well like we would do for all the values as well. This allows collections to be defined still with each entry having a separate timestamp (e.g. list of time-stamped log messages).

Update examples for timestamp

Update Readme about staleness for custom timestamps

Signed-off-by: Jan Phillip Kretzschmar <janphkre@gmx.de>
janphkre added a commit to janphkre/json_exporter that referenced this issue Jul 11, 2022
based on prometheus-community#97 and prometheus-community#80 this provides the posibility to use a metric that has a unix style timestamp as the timestamp of the scraped metric

When deserializing objects we need to take the key json path into account as well like we would do for all the values as well. This allows collections to be defined still with each entry having a separate timestamp (e.g. list of time-stamped log messages).

Update examples for timestamp

Update Readme about staleness for custom timestamps

Signed-off-by: Jan Phillip Kretzschmar <janphkre@gmx.de>
SuperQ added a commit that referenced this issue Aug 28, 2022
* Pick timestamp from metric

based on #97 and #80 this provides the posibility to use a metric that has a unix style timestamp as the timestamp of the scraped metric

When deserializing objects we need to take the key json path into account as well like we would do for all the values as well. This allows collections to be defined still with each entry having a separate timestamp (e.g. list of time-stamped log messages).

Update examples for timestamp

Update Readme about staleness for custom timestamps

Signed-off-by: Jan Phillip Kretzschmar <janphkre@gmx.de>
Signed-off-by: Ben Kochie <superq@gmail.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants