Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add InfluxDB exporter to Actuator #5688

Closed
jmkgreen opened this issue Apr 14, 2016 · 17 comments

Comments

Projects
None yet
8 participants
@jmkgreen
Copy link

commented Apr 14, 2016

I am being pointed at InfluxDB by various people to record statistics, it would seem sensible to provide a MetricsWriter implementation for InfluxDB.

The InfluxDB side of things seems very well documented by them. Feels like this should be easy enough...

@philwebb

This comment has been minimized.

Copy link
Member

commented Apr 15, 2016

It would indeed be a nice addition but I'm not sure if we'll get to it for 1.4 without some help. I've tagged as ideal-for-contribution, let's see if anyone can contribute something.

@jmkgreen

This comment has been minimized.

Copy link
Author

commented Apr 15, 2016

https://github.com/jmkgreen/spring-boot-actuator-influxdb is the raw code - copied almost verbatim from the OpenTSDB writer.

Interestingly for InfluxDB it has both tags and fields. So where Metric is essentially a String identifier and a Long value, Influx asks that the value be associated with a field name. This is on top of any tags that might be included.

To start with the value is written to a field named simply value. I am unsure whether the existing metrics code base has enough semantics to map into field=value pairs or not..?

@dsyer

This comment has been minimized.

Copy link
Member

commented Apr 16, 2016

See also #5701

@jmkgreen

This comment has been minimized.

Copy link
Author

commented Apr 19, 2016

The code has been updated. I now have a demo application that is successfully writing into InfluxDB.

@dsyer dsyer self-assigned this Apr 27, 2016

@dsyer dsyer added this to the 1.4.0.M3 milestone Apr 27, 2016

@dsyer

This comment has been minimized.

Copy link
Member

commented Apr 28, 2016

@jmkgreen how does influxdb deal with counters? I only see a GaugeWriter in your implementation. It's not necessarily an issue, but I'm interested to know how you aggregate your counters across processes.

@jmkgreen

This comment has been minimized.

Copy link
Author

commented Apr 28, 2016

If your question is "does influx accept a relative value" in order to increment a counter by some amount, it doesn't. It just records field=value pairs against a named metric. The line protocol takes a little getting used to due to metric names, fields, and tags. But it's the field values that rule out counters at the InfluxDB level.

Influx does have derivative() in the query interface. I would suggest posting gauges in with a suitable tag value of the current process id for consumers to spot changes over time of individual processes.

Looks like there are examples where metric names are checked for timer or gauge and considered to be a gauge, versus counter as a actual counter? There may be options here for interpretation.

@philwebb philwebb modified the milestones: 1.4.0.RC1, 1.4.0.M3 May 16, 2016

@philwebb philwebb removed this from the 1.4.0.RC1 milestone May 25, 2016

mklimasz added a commit to mklimasz/spring-boot that referenced this issue Nov 8, 2016

Add InfluxDB exporter to Actuator.
Configurable implementation of gauge writer to export metrics into InfluxDB.

Fixes spring-projectsgh-5688

mklimasz added a commit to mklimasz/spring-boot that referenced this issue Nov 8, 2016

Add InfluxDB exporter to Actuator.
Configurable implementation of gauge writer to export metrics into InfluxDB.

Fixes spring-projectsgh-5688

mklimasz added a commit to mklimasz/spring-boot that referenced this issue Nov 8, 2016

Add InfluxDB exporter to Actuator.
Configurable implementation of gauge writer to export metrics into InfluxDB.

Fixes spring-projectsgh-5688

@philwebb philwebb added this to the 1.5.0 M1 milestone Nov 15, 2016

philwebb added a commit to philwebb/spring-boot that referenced this issue Nov 16, 2016

Add InfluxDB exporter to Actuator.
Configurable implementation of gauge writer to export metrics into InfluxDB.

Fixes spring-projectsgh-5688
@Smoker21

This comment has been minimized.

Copy link

commented Nov 23, 2016

I test the implementation. It write gauge and counter only, and cannot set "retention policy" for InfluxDBWriter. Is there any plan to write metrics to influxDb?

@jiffle

This comment has been minimized.

Copy link

commented Nov 25, 2016

The above implementation has an issue with it that caused it to fail to send certain metrics. In InfluxDBMetricWriter the line
this.influxDB.write(this.databaseName, value.getName(), point);
is incorrect.
I found that using null instead of value.getName() for the retention policy works (presumably uses default):
this.influxDB.write( this.databaseName, null, point);

Secondly, am I right in thinking that this won't work with Dropwizard metrics? Does it need some adaptor to work, or does it just need a different configuration (I'm using @ExportMetricsWriter)?

@jmkgreen

This comment has been minimized.

Copy link
Author

commented Nov 25, 2016

I will try to resurrect this. I've been working hard (commercially) with getting reporting of Spring Boot applications into data repositories such as DataDog and it is quite hard going when you uncover the range of issues at detail. We may well end up going InfluxDB for mass metric storage, possibly with New Relic to tell us what is wrong with performance.

@jiffle

This comment has been minimized.

Copy link

commented Nov 25, 2016

Is there anything I can help you with to get Influx integration into 1.5? I've been looking at the TICK Stack and it's looking pretty good.

@philwebb philwebb removed this from the 1.5.0 M1 milestone Dec 20, 2016

@philwebb

This comment has been minimized.

Copy link
Member

commented Dec 20, 2016

Unfortunately I need to de-scope some issues targeted for Spring Boot 1.5. I'd like to push this back since we want to review metrics in general.

@jiffle

This comment has been minimized.

Copy link

commented Dec 20, 2016

What's are the issues here? As far as I can see all it needs to be ready for 1.5 are autoconfig, standardised properties and an alternate configuration for Dropwizard. Am I missing something?

I can understand reviewing metrics in general, but surely Influx support would follow the same patterns and changes as all the other metrics integrations?

@jiffle

This comment has been minimized.

Copy link

commented Dec 20, 2016

Looking at DB-Engines website's rankings of Time-series DBs, Influx is comfortably the most popular.
http://db-engines.com/en/ranking/time+series+dbms
Why therefore is it such a low priority?

@dsyer

This comment has been minimized.

Copy link
Member

commented Dec 20, 2016

There is a PR, but the author seems to have become demotivated. Maybe fixing that or providing an alternative would help move things along?

@jiffle

This comment has been minimized.

Copy link

commented Dec 20, 2016

I got the code from the PR working (and wrote up a simplified non-Dropwizard example as a blog post).
I'd be happy to work on autoconfig and properties, and if someone could point me in the direction of what changes would be required to also support the DW config, that would be everything we would need for a PR for complete support, would it not?

@JulienChampseix

This comment has been minimized.

Copy link

commented Apr 5, 2017

@philwebb is it planned to be added soon ? @jiffle thanks for your blog post !

@wilkinsona

This comment has been minimized.

Copy link
Member

commented Aug 11, 2017

This has been superseded by the planned move to Micrometer-based metrics (#9970)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.