Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

system metrics reported by collector always have a length of 10 #33

Closed
3 of 4 tasks
at15 opened this issue May 15, 2017 · 1 comment
Closed
3 of 4 tasks

system metrics reported by collector always have a length of 10 #33

at15 opened this issue May 15, 2017 · 1 comment
Assignees
Milestone

Comments

@at15
Copy link
Member

at15 commented May 15, 2017

Related #21

When using xkc to report system metrics, the series length is always 10. And the collector buffer 10 points for each series.

DEBU[0038] mem:data create new entry cpu.6.user in map pkg=k.s.mem 
DEBU[0038] mem:data create new entry cpu.total.nice in map pkg=k.s.mem 
DEBU[0038] mem:data create new entry cpu.2.system in map pkg=k.s.mem 
DEBU[0113] mem:data merge with existing series cpu.0.iowait pkg=k.s.mem 
DEBU[0113] length 10 pkg=k.s.mem 
DEBU[0113] mem:data merge with existing series cpu.5.irq pkg=k.s.mem 
DEBU[0113] length 10 pkg=k.s.mem 

But test in series_store_test.go shows it correctly handled merge data, one explanation could be the collector is using same time stamp

TODO

  • write a test to reproduce this problem
    • reduce number of series
  • every time you restart the collector, the length can grow, if you restart collector, the length will become 20, restart collector again, 30.
  • the collector never reset the points for each series
  • the serializer has problem, it is not reset? Yes
    • but the server does not report any error?

Conclusion

  • the serializer is not reset after each send, and server drop the invalid part silently when parsing JSON
@at15 at15 added the bug label May 15, 2017
@at15 at15 added this to the 0.1.0 milestone May 15, 2017
@at15 at15 self-assigned this May 15, 2017
@at15 at15 added this to DOING in Metrics Collector May 15, 2017
@at15
Copy link
Member Author

at15 commented May 15, 2017

[{"name":"mem.total","tags":{"host":"at15-PC4073"},"points":[[1494832196000,32858548],[1494832197000,32858548],[1494832198000,32858548],[1494832199000,32858548],[1494832200000,32858548],[1494832201000,32858548],[1494832202000,32858548],[1494832203000,32858548],[1494832204000,32858548],[1494832205000,32858548]]},{"name":"mem.free","tags":{"host":"at15-PC4073"},"points":[[1494832196000,20316588],[1494832197000,20317844],[1494832198000,20344596],[1494832199000,20345128],[1494832200000,20339624],[1494832201000,20338972],[1494832202000,20338108],[1494832203000,20338100],[1494832204000,20338960],[1494832205000,20338804]]}]
[{"name":"mem.total","tags":{"host":"at15-PC4073"},"points":[[1494832196000,32858548],[1494832197000,32858548],[1494832198000,32858548],[1494832199000,32858548],[1494832200000,32858548],[1494832201000,32858548],[1494832202000,32858548],[1494832203000,32858548],[1494832204000,32858548],[1494832205000,32858548]]},{"name":"mem.free","tags":{"host":"at15-PC4073"},"points":[[1494832196000,20316588],[1494832197000,20317844],[1494832198000,20344596],[1494832199000,20345128],[1494832200000,20339624],[1494832201000,20338972],[1494832202000,20338108],[1494832203000,20338100],[1494832204000,20338960],[1494832205000,20338804]]}][{"name":"mem.total","tags":{"host":"at15-PC4073"},"points":[[1494832206000,32858548],[1494832207000,32858548],[1494832208000,32858548],[1494832209000,32858548],[1494832210000,32858548],[1494832211000,32858548],[1494832212000,32858548],[1494832213000,32858548],[1494832214000,32858548],[1494832215000,32858548],[1494832216000,32858548]]},{"name":"mem.free","tags":{"host":"at15-PC4073"},"points":[[1494832206000,20338488],[1494832207000,20338372],[1494832208000,20337504],[1494832209000,20337380],[1494832210000,20337248],[1494832211000,20338064],[1494832212000,20337968],[1494832213000,20336976],[1494832214000,20336680],[1494832215000,20336580],[1494832216000,20336208]]}]

@at15 at15 added the backlog label May 15, 2017
at15 added a commit that referenced this issue May 15, 2017
- series are not reset, so its points keep growing, but on the server
side, after merge, it is always 10 points
- time stamp on collector seems fine
@at15 at15 closed this as completed in 01d3c18 May 15, 2017
@at15 at15 moved this from DOING to BACKLOG in Metrics Collector May 15, 2017
@at15 at15 mentioned this issue May 15, 2017
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Metrics Collector
  
BACKLOG
Development

No branches or pull requests

1 participant