Minutes from 2020 06 22

Attendees: Nikolay, Octo, Matthias, sunku, Jabir

• Note: Might have missed to capture all attendees, if you (as the reader) were part of this call, please update your name

Derivative metrics via Collectd Plugins

• Calculating derivative metrics via collectd plugins is easier to do within read plugin, if it can be done
• Dealing with metrics after we submit to daemon would be trickier
• Ex: aggregation plugin provides some reference for label based metrics, we would need to invest some time to improve aggregation plugin & related code. Need to look into how best aggregation plugins would need to behave
• For example consider 10 CPUs for which the metrics need to be read from, read plugins don’t register the metrics per CPU basis, when 2 of CPUs go offline we don’t know which metrics we are supposed to have. Do we want graph showing data not available or show only the ones available? Need to consider various aggregation models
• Prometheus is really good in aggregating, assuming view that data presented to Prometheus has been updated atomically, you can essentially do math on these metrics for consistent result.
• Need to consider various models available out there to see how best Collectd can build on it
• With version 6 coming, we need to revisit fundamental way of how we deal with issues

Python plugin repo structure & contributions

• Licensing questions have been there from the beginning of Collectd with bunch of GPL code in the daemon with argument that if we link plugins, export of the symbols of daemon & plugins should also be GPL. So daemon was changed to MIT as its not copy left. Since no plugin bits are linked into daemon so there is no derived work at direct compile time
• Python plugins are one more step removed from the daemon, as it essentially starts python interpreter and it loads python plugins. So its fine to have MIT for daemon & python plugins. Its also ok to have python plugins to have GPL
• GPL spreads into everything it touches (includes dynamic linking), just to be aware
• If you want to add GPL python plugins, ideally a README should be added about licenses if its different than the repo license
• For ISC - it falls under one of permissive licenses, it shields authors from any liabilities/damages. Its saying do whatever with code as long as you keep corporate notice.
• Octo is open to changing it to MIT, but believes it wont make any difference
• Plugins could have any license as long as they provide their own
• Directory structure:
  • A sperate directory for each plugin would be good
  • As we might have different licenses for different plugins having separate directories would be great
  • There was a bug with function names clashed in python plugins, keep in mind going ahead: https://github.com/collectd/collectd/issues/3240
    • Probably due to global symbol resolution
    • Authors of python plugins be aware of this
    • If anyone wants to improve python plugin frameowkr, octo could help contact SVEN/SET
    • Thanks Matthias for sharing this
• Going ahead provide a readme about the plugin & any assumptions
• Happy to discuss as more questions as they arise

Collectd website migration - assigned to Octo

• Had some troubles with Octo's servers, but resolved now
• Will continue to work on it now

Go-collectd framework updates

• Ashley has been very helpful
• For Go based plugins that rely on C libraries, they should use Cgo to reach out to these libraries written in C (https://blog.golang.org/cgo).
  • The tricky part is about building everything. Typically need to set environment variables towards collectd header files for things to work, there is no automation for this - we have to point proper sources, install appropriate deb packages, etc.
  • There is no config script, build system, etc. but use go command line utility to build. So far we know there is no need for changes in packages provided in go-collectd framework

Rename master branch to main branch

• Term 'master branch' being insenstive towards people is raised in github
• Florian to create main branch in Collectd repo and move CI/CD towards it
• As we get more confidence with main branch that its working as intended, master branch will be deleted
• Still not sure how this affects existing open issues, looking to github to provide some guidance there. So Octo to hold off moving/deleted for now

Open issue grooming

• https://github.com/collectd/collectd/issues/203 -> very old, closing it
• https://github.com/collectd/collectd/issues/211
  • Complaint mechanism in collectd (error msg with rate limiter). If something is breaking for every iteration it would only be printed for every couple of seconds instead of 100s of times
  • It might help to tweak the error msg
  • But no one bothered to give even a +1
  • It would help to fix it
• https://github.com/collectd/collectd/issues/217
  • Would help to fix it
  • Would probably needs to replace default memory allocator to solve this
  • One thing considered, if we could make it configurable wether plugin is have it loaded from main daemon or fork off a worker that is dealing with one plugin, then we could probably look at memory usage of sepearte processes