Request for merge for change that ignores data points that utilize undefined tag values, tag keys, or metric names as opposed to throwing an exception (optional based on query string param) #243

Open
wants to merge 3 commits into
from

Conversation

Projects
None yet
2 participants

Overview

This change adds a query string parameter, 'silent'. If present then any query that contains a tag key, tag value, or metric name that has not been registered in TSDB is ignored (dropped) as opposed to throwing an exception. Thus an entire query will not fail if a member of it has one of these values that does not exist.

Justification

A query that specifies tag information or a metric name does not need to fail and should instead return no data as this is essentially the same as specifying a filter that does not match any targets and thus should be treated as that way.

This is very helpful when you are using OpenTSDB to collect data and at time of query you can not determine if a given metric or tag pair has been added (--auto-metric).

This is also helpful when a single query contains multiple data points some of which are valid (i.e. have valid tag / metric information) and some don't. Instead of the whole query failing, valid data is returned for those data points for which the filter matches targets.

Backward Compatibility

This capability is activated by the presence of a 'silent' option on the end of the query string, i.e. '&silent'. Thus if this option is not include the existing behavior (exception) will be the default.

davidkbainbridge and others added some commits Oct 4, 2013

Initial checkin of code to ignore those parts of the query if tag inf…
…ormation or metric have not yet been created in TSDB
add a query param option to use as a switch to ignore bad queries, wh…
…ere bad is defined as queries that contain tag value, tag keys, or metric names that have not been registered

@davidkbainbridge davidkbainbridge referenced this pull request in zenoss/query Oct 4, 2013

Merged

Review please feature/ignorebad #42

Owner

manolama commented Nov 21, 2013

Sorry for the delay. So the use case would be if a user executes something like "?start=1d-ago&m=sum:foo{host=web01|web02}" but "web02" has not been assigned a UID yet, instead of throwing the exception "no UID found for web02" we should just return the data for "host=web01"? That sounds reasonable as an optional config like you implemented. Would you mind changing the flag from "silent" to "ignore_missing" or "ignore_nsun" for "no such unique name"?

Changing the flag is perfectly reasonable. I am wondering if something like
ignore_unknown_tags might work. I personally prefer a flag that is a little
more descriptive. I might not get to this right away as I have changed
companies and OpenTSDB is not something I am actively working with at the
moment.

On Thu, Nov 21, 2013 at 1:38 PM, Chris Larsen notifications@github.comwrote:

Sorry for the delay. So the use case would be if a user executes something
like "?start=1d-ago&m=sum:foo{host=web01|web02}" but "web02" has not been
assigned a UID yet, instead of throwing the exception "no UID found for
web02" we should just return the data for "host=web01"? That sounds
reasonable as an optional config like you implemented. Would you mind
changing the flag from "silent" to "ignore_missing" or "ignore_nsun" for
"no such unique name"?


Reply to this email directly or view it on GitHubhttps://github.com/OpenTSDB/opentsdb/pull/243#issuecomment-29026979
.

manolama added a commit to manolama/opentsdb that referenced this pull request Aug 26, 2014

Add the tsd.query.skip_unresolved_tagvs command line option and code …
…that allows

queries to complete even if an explicitly declared tag value does not exist such
as a query "m=sum:sys.cpu.user{host=web01|web02}" where "web02" has not been
assigned a UID yet.
This solves the most common use case from #243. Metrics and tag names should
still throw an exception as those are assigned much less frequently than values.
Additional flags can be given to handle these situations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment