Add csv support to stats_over_http#5023
Add csv support to stats_over_http#5023zrhoffman merged 5 commits intoapache:masterfrom ezelkow1:tm-csv-stats
Conversation
This uses the same parsing method as astats csv and provides almost double the performance with less than half the allocations of the json version. Also add a null check for dsStats. Its possible to do a request to ATS to a DS that does not exist, ATS will then insert stats for that request into the remap stats list. When it gets to TM though it will not be in the list of known DSs so if we try to index by it it will end up null
zrhoffman
left a comment
There was a problem hiding this comment.
Our docs do not mention that, in order for Traffic Monitor to successfully poll servers using stats_over_http output, the experimental system_stats ATS plugin must also been enabled.
Otherwise, you get an unhelpful error like REPORTED - Error parsing loadavg for cache 'edge': Data was missing 'plugin.system_stats.loadavg.one'; eth0: not found in polled data.
So, this is a good time to make sure our documentation includes that info.
* Remove my dstat nil check since this has been merged in already
…SoH and that it can be used to provide system info for any custom extensions
Thanks Updated changelog, removed nil check since it went in to the codebase before this went through, and put info in the docs about adding system_stats |
zrhoffman
left a comment
There was a problem hiding this comment.
Looks good!
- TM successfully reports cache health when when accepting text/csv polling responses. See the CDN-in-a-Box CI result from https://github.com/zrhoffman/trafficcontrol/commits/tm-csv-stats, which uses ATS 9 and stats_over_http with CSV polling
- Code, documentation, and changelog look good
- Tests and benchmarks pass and have good coverage
What does this PR (Pull Request) do?
This uses the same parsing method as astats csv and provides almost double the performance with less than half the allocations of the json version.
Also add a null check for dsStats. Its possible to do a request to ATS to a DS that does not exist, ATS will then insert stats for that request into the remap stats list. When it gets to TM though it will not be in the list of known DSs so if we try to index by it it will end up null
Which Traffic Control components are affected by this PR?
What is the best way to verify this PR?
If this is a bug fix, what versions of Traffic Control are affected?
The following criteria are ALL met by this PR
Additional Information
Benchmarks comparing all stats parsing, both astats and stats over http with json and csv. In both CSV instances they are ~100% faster with less than half as many allocations
BenchmarkAstatsJson-8 5276 202209 ns/op 134453 B/op 2028 allocs/op
BenchmarkAstatsCSV-8 9963 111798 ns/op 110001 B/op 798 allocs/op
BenchmarkStatsJson-8 5512 227104 ns/op 134991 B/op 2082 allocs/op
BenchmarkStatsCSV-8 10000 108320 ns/op 111217 B/op 837 allocs/op