Tagged demo

[mary3000]

This demonstrates how tagged series can be inserted in CH with carbon-clickhouse.
Also with tags I need to create more complex queries for Grafana, like

SELECT 
    $timeSeries AS t, 
    avg(Value) 
FROM graphite 
ALL INNER JOIN 
( 
    SELECT Path
    FROM graphite_tagged
    WHERE 
        Tag1 = 'a=1'  OR Tag1 = 'c=5' 
    GROUP BY (Path, Tags) 
    HAVING count(Tag1) == 2 
) USING Path 
WHERE $timeFilter 
GROUP BY t 
ORDER BY t

for selecting all series with both tag a=1 and b=5. This query selects tagged metric paths from special table (graphite_tagged) and joins it with the main (graphite) table.

Pros:

• Simple to use
• ...

Cons:

• Extra memory on storing tagged data in graphite_tagged table
• Join is not a good approach, as far as I know. It is slow.
• ...

[code-of-kpp]

How tags are exposed / used in grafana?

[code-of-kpp]

Who manages graphite_tagged table and how?

[mary3000]

Who manages graphite_tagged table and how?

It is managed by clickhouse-carbon, see this.

It is parsed here: all keys in the string are sorted and repeated keys are deleted. You can see in the tests examples of resulting string. It will be then added to graphite_tagged and graphite tables with this string as Path.

[code-of-kpp]

so basically old tags are never deleted?

[mary3000]

so basically old tags are never deleted?

If I'm not mistaken, yes. It can be a problem since metrics can have a huge amount of different tags? Maybe we can drop too old partitions periodically through some demon (delete is now supported in CH)

[code-of-kpp]

Things like tags exploding happen: when something is marked as tag by error. Just imagine more or less random data (strings and numbers) going directly to 2003 port. Suddenly you can end up with 100500+ different things labeled as tags with no semantics.
Usually it is not the case though: even if you have lots of virtual machines and/or containers popping out of nowhere with random hostnames, the tag space is likely to be small enough keep all the values forever and if lookups are cached somehow that should be fine.

[mary3000]

How tags are exposed / used in grafana?

What do you mean? Grafana has no knowledge about tags in CH, I just write some query that that us using some tags for filtering.

[code-of-kpp]

Usually people use grafana template variables for this. With template variable of type "Query" you can get a list of possible tag values. Then you can use this variable in query.

For example you can have template variable for hostname: with drop-down menu (or using GET parameters) user specifies one or more hostnames and grafana shows only data for selected hostnames.

i.e. tag values query SELECT DISTINCT tag_value FROM tags WHERE tag_key = 'hostname' AND...
and templated query SELECT hostname, time_slot, value FROM metrics WHERE match(hostname, '${hostname:regex}') AND ... GROUP BY (hostname, time_slot)

[mary3000]

If these template variables rely on query, would they be automatically updated if I will insert some data that changes the result of this query (for example new tags), or I should update these templates manually after it?

[code-of-kpp]

Grafana can update them, yes

[code-of-kpp]

is it WIP?

[mary3000]

is it WIP?

No, I added template variables.

[code-of-kpp]

Could you please rebase?

[mary3000]

rebased