Logfmt parser #4539
Logfmt parser #4539
Conversation
…between Parse and ParseLine.
plugins/parsers/logfmt/parser.go
Outdated
|
|
||
| //add default tags | ||
| metrics = append(metrics, m) | ||
| p.applyDefaultTags(metrics) |
There was a problem hiding this comment.
Call only once at end of function, since this applies the tags to all metrics.
plugins/parsers/logfmt/parser.go
Outdated
| //attempt type conversions | ||
| value := string(decoder.Value()) | ||
| if iValue, err := strconv.Atoi(value); err == nil { | ||
| //log.Printf("Print Atoi Value Here:", iValue) |
There was a problem hiding this comment.
Clean up (delete) log messages.
plugins/parsers/logfmt/parser.go
Outdated
| @@ -0,0 +1,115 @@ | |||
| // Package logfmt converts logfmt data into metrics. New comment | |||
There was a problem hiding this comment.
Take a pass through the file and make sure comments are all useful and well written.
plugins/parsers/logfmt/parser.go
Outdated
| "strconv" | ||
| "time" | ||
|
|
||
| glogfmt "github.com/go-logfmt/logfmt" |
There was a problem hiding this comment.
No reason to alias the import path since there is no conflict.
plugins/parsers/logfmt/parser.go
Outdated
|
|
||
| //attempt type conversions | ||
| value := string(decoder.Value()) | ||
| if iValue, err := strconv.Atoi(value); err == nil { |
There was a problem hiding this comment.
Use strconv.ParseInt here because it handles 64-bit ints correctly.
docs/DATA_FORMATS_INPUT.md
Outdated
| # Logfmt | ||
| This parser implements the logfmt format by extracting key-value pairs from log text in the form `<key>=<value>`. | ||
| At the moment, the plugin will produce one metric per line and all keys | ||
| are added as fields. Values are left as strings (for now). |
There was a problem hiding this comment.
Values are left as strings
The code below doesn't do this.
docs/DATA_FORMATS_INPUT.md
Outdated
| A typical log | ||
| ``` | ||
| method=GET host=influxdata.org ts=2018-07-24T19:43:40.275Z | ||
| connect=4ms service=8ms status=200 bytes=1653 |
There was a problem hiding this comment.
This example should be a single line
docs/DATA_FORMATS_INPUT.md
Outdated
| "connect": "4ms", | ||
| "service": "8ms", | ||
| "status": 200, | ||
| "bytes": 1653, |
There was a problem hiding this comment.
Show the converted version as line protocol.
| name: "Log parser only returns metrics from first string", | ||
| now: func() time.Time { return time.Unix(0, 0) }, | ||
| measurement: "testlog", | ||
| s: `ts=2018-07-24T19:43:35.207268Z lvl=5 msg="Write failed" log_id=09R4e4Rl000 "/n" |
There was a problem hiding this comment.
Fix the newline test here \n, like up on line 89
docs/DATA_FORMATS_INPUT.md
Outdated
| # Logfmt | ||
| This parser implements the logfmt format by extracting key-value pairs from log text in the form `<key>=<value>`. | ||
| At the moment, the plugin will produce one metric per line and all keys | ||
| are added as fields. Values are left as strings (for now). |
There was a problem hiding this comment.
Values are left as strings (for now).
This isn't true, we are doing auto conversion to other types.
| // NewParser creates a parser. | ||
| func NewParser(metricName string, defaultTags map[string]string) *Parser { | ||
| return &Parser{ | ||
| MetricName: metricName, |
There was a problem hiding this comment.
What happens if this is not set?
There was a problem hiding this comment.
By default, doesn't telegraf use the plugin name of the input/processor that uses it?
docs/DATA_FORMATS_INPUT.md
Outdated
| ``` | ||
| will be converted into | ||
| ``` | ||
| method=GET, host=influxdata.org, ts=2018-07-24T19:43:40.275Z, connect=4ms, service=8ms, status=200, bytes=1653, |
There was a problem hiding this comment.
Turn this into valid line protocol:
logfmt method="GET",host="influxdata.org",ts="2018-07-24T19:43:40.275Z",connect="4ms",service="8ms",status=200i,bytes=1653i
Required for all PRs:
closes #4427