Make external API a separate module

[danielsdeleo]

🔩 Description: What code changed, and why?

Make the api/external directory a separate golang module. When using golang modules, when you depend on any package within a module, you depend on that module and therefore also all of that module's dependencies. Currently all of Automate is one big module which is nice for our internal dependency management but for external users it means you have to take on a lot of deps even if you only intend to use a few structs from api/external directory.

At present the goal is to support a project that is integrating some Chef Workstation functionality with Chef Automate, but this would be helpful to anyone who wishes to write code using Automate's API in go.

⛓️ Related Resources

👍 Definition of Done

👟 How to Build and Test the Change

✅ Checklist

• I have read the CONTRIBUTING document.
• Tests added/updated?
• Docs added/updated?
• All commits have been signed-off for the Developer Certification of Origin.

📷 Screenshots, if applicable

[netlify]

Deploy preview for chef-automate ready!

Built with commit 0f190e3

https://deploy-preview-4298--chef-automate.netlify.app

[stevendanna]

I wonder if this is a time to re-look at our vendoring in this repository. Having to re-vendor our own code anytime we make a change to it feels like it will trip people up regularly.

[srenatus]

@stevendanna you're suggesting that we stop vendoring, and rely completely on go modules instead? It has worked fine for the other project, I'd say. 👍

[srenatus]

🤔 I wonder how this got dragged in

[stevendanna]

you're suggesting that we stop vendoring, and rely completely on go modules instead?

I think we should consider it, yes. We started vendoring when using dep which was substantially slower without vendoring.

In my views, if we stop vendoring, the cons are:

• Updating dependencies no longer provides an easy way to review what is changing in those dependencies.
• We'll likely have to give up on rebuilding all affected components when a dependency changes. This means occasionally we could have build-breaks caused by a previous dependency update. (we might be able to do something here with changes to our existing tooling).
• More network resources required during the build (likely can be mitigated with some caching)

The pros are:

• Fewer situations where people need to regenerate bldr.toml.
• Likely makes it easier to split out more modules over time if we care to.
• Less work when updating a dependency

[srenatus]

Updating dependencies no longer provides an easy way to review what is changing in those dependencies.

Yeah, it's still in the go.{sum,mod} files, but you won't see the code right away.

I agree with your pros and cons there, thanks for summing this up. :)

[danielsdeleo]

Update: in #4314 we have removed vendoring, so now this change is mostly transparent for an Automate developer. I checked that you can update a proto in api/external, compile it, and then use it right away (no need to revendor or git commit or anything). Updating deps that appear in both modules may be confusing, and I'm not sure what I could do about that (or rather, I would like to have some practical experience of the problem before attempting to fix). So I think this is ready to go now.

[lancewf]

Everything worked the same for me.