Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Proposal: package checks using Wheels #634

Open
wants to merge 2 commits into
from

Conversation

Projects
None yet
4 participants
Member

masci commented Jul 26, 2017

This is a proposal to change the way we build and ship Agent integrations (or checks). I tried to detail the background and the reasoning of this as much as I could, so please forgive the wall of text. The document contains a few details about the internals of the new Agent but the same concepts apply to the current version.

The part about what happens when the Agent is upgraded is the weakest I have to admit, so any feedback on that side would be much appreciated.

Please notice the spirit of the proposal is about making contributors' life (myself included) a little bit easier but this can't come at the price of decreasing Agent's reliability, that's why we think your opinion matters here.

@masci masci checks as wheel proposal
3e34c0c

@masci masci added the docs label Jul 26, 2017

+check, the archives would be platform independent, meaning one could build the
+check on a Mac and copy and install the artifact in a Linux box.
+
+#### Versioning
@hush-hush

hush-hush Jul 26, 2017 edited

Member

The versioning part is very nice indeed. When we look at both dd-agent and integrations-core repos it's very difficult to know which version was shipped (even more when looking only at the history of a specific integration).

+We could also provide a dummy Agent wheel package integrations would depend on:
+this would enforce compatibility with the core agent at install time (enforced
+by pip) instead of at runtime (enforced by the gent itself after reading check
+metadata). This would also make manifest files unnecessary since we would use
@hush-hush

hush-hush Jul 26, 2017

Member

Since much more tools exist to handle setup.py files it would also make scripting integrations-core easier than right now.

+ pip.
+
+Cons:
+ * Users wouldn’t be able to pin specific check versions (this could be a non
@hush-hush

hush-hush Jul 26, 2017

Member

I don't think that it's a requirement. I don't see customers wanting a new agent with an old check. Customers usually want newer integration with an old agent to avoid a big update. When they do upgrade the agent their goal to have all the latest features. Plus we almost never remove a feature from an integration.

I think we can assume customers are only moving forward in their upgrade (and we should encourage them to do so).

@masci masci added the help wanted label Jul 27, 2017

@masci masci added proposal and removed docs labels Aug 8, 2017

ross commented Aug 11, 2017

Will the ability to drop stand-alone <whatever>.py files into a directory like /etc/dd-agent/checks.d be preserved?

For a lot of really simple cases it's nice to throw together a 30 line python file and drop it on a host with puppet/chef/salt/whatever. It'd be really nice to avoid having to deal with the full setup.py/wheel stuff for things like that. There definitely are cases where the package would useful as well.

I guess reading/skimming this it wasn't 100% clear whether this was just going to be a thing for the existing integrations, required of all integrations, or optional? The latter obviously being my preferred answer 😀

Member

gmmeyer commented Aug 11, 2017 edited

@ross yes! This is just for integrations that are shipped as upgrades to existing ones or independent from the agent. For example, all of the integrations here are available as independent packages currently (but as debs, rpms or msis).

@masci masci point out that custom checks won't be affected by the proposal
b978ba8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment