Request: Support for missing API Groups from ExternalSecrets

[colstrom]

Hi!

First off, thanks for all the work that went into this. In particular, the tooling and automated releases workflows! I've taken a swing at something similar before, and I know there's a lot of effort involved.

I'm not sure if this is intentional or not, but the kcr_external_secrets_io crate only has support for a handful of the API Groups currently included with ExternalSecrets (as of v0.16.1). I suspect that some of these were just added after the generator was written, and might need to be added to the catalog here?

apiGroup	included in kcr_external_secrets_io crate?
acraccesstokens.generators.external-secrets.io	❌
clusterexternalsecrets.external-secrets.io	✅
clustergenerators.generators.external-secrets.io	❌
clusterpushsecrets.external-secrets.io	❌
clustersecretstores.external-secrets.io	✅
ecrauthorizationtokens.generators.external-secrets.io	❌
externalsecrets.external-secrets.io	✅
fakes.generators.external-secrets.io	❌
gcraccesstokens.generators.external-secrets.io	❌
generatorstates.generators.external-secrets.io	❌
githubaccesstokens.generators.external-secrets.io	❌
grafanas.generators.external-secrets.io	❌
passwords.generators.external-secrets.io	❌
pushsecrets.external-secrets.io	❌
quayaccesstokens.generators.external-secrets.io	❌
secretstores.external-secrets.io	✅
stssessiontokens.generators.external-secrets.io	❌
uuids.generators.external-secrets.io	❌
vaultdynamicsecrets.generators.external-secrets.io	❌
webhooks.generators.external-secrets.io	❌

I'm not sure if these belong in the kcr_external_secrets_io crate, if they should be enabled by default, or behind a feature flag, or if they should be a separate crate or something else.

I'd be willing to contribute to this if it would help, I'd just need a nudge in the right direction.

Thanks! ❤️

[sebhoss]

Sure!

If you want to give it a shot, do the following in your fork of this repository:

1. Add all missing CRDs to the catalog at https://github.com/metio/kube-custom-resources-rs/blob/main/code-generator/src/catalog.rs#L1303-L1313
2. Execute ./code-generator/generate.sh external-secrets/external-secrets in your shell. You must have rust/cargo/yq installed
3. Execute ./code-generator/fix-cargo-warnings.sh <GROUP> for each group that is part of the external-secrets project. From a quick look, those should be generators_external_secrets_io and external_secrets_io
4. Commit all changed files & open a PR

Happy to help if something is not working!

[sebhoss]

I've added that in #393 and added yourself as co-author. Thanks again for bringing this to my attention!

[colstrom]

Thank you very much for adding this. Can you help me understand this file?

It looks like that resource is being ignored due to this error, yes?

cannot find type `PushSecretStatusSyncedPushSecrets` in this scope

I encountered the same error when building locally, which I was able to resolve by adding the following to pushsecrets.rs:

pub type PushSecretStatusSyncedPushSecrets = BTreeMap<String, PushSecretData>;

What I don't understand is how to preserve that when the code generator regenerates the file.

Is there a hook somewhere to just like... add extra code like that to the generated output?

[sebhoss]

Yes exactly! You can specify .ignore files for each custom resource and optionally write a reason why the resource is ignored into it. After that, simply re-run the code generator and it will automatically clean up those resources. I've written a little about this in the contrib guidelines but named it weirdly 😅

Adding custom code is currently not supported and I'm not sure if we should actually do that in this project. I think a better approach would be to open a PR in kopium to add support for cases like these, so that everyone and not just users of this project can use kopium to generate working code for all custom resources out there.

Just thinking out loud: That hook idea could actually be a nice feature for kopium itself?! WDYT @clux?

[clux]

Hey, if I understand correctly, kopium already supports this via the --elide flag. It might be one of those things that's better served with a config format perhaps though.

[sebhoss]

I think in this case, we want to add additional code that was written by hand instead of removing kopium-generated code. Something like kopium ... --add-type 'PushSecretStatusSyncedPushSecrets' 'BTreeMap<String, PushSecretData>' (or even kopium ... --add-code 'pub type PushSecretStatusSyncedPushSecrets = BTreeMap<String, PushSecretData>;') would allow to generate the PushSecrets CRD.

[clux]

I normally prefer having a static additions type module for things like this as then you can have the generated stuff in one mod podcrd, and another mod additions with these things handwritten (or if simple with echo 'pub MyCode' > additions.rs). Mostly because you never know what you need and writing anything more than a single line thing like that gets awkward as a CLI arg.

[sebhoss]

ok ok - then I'm added some sort of hook here to add these code additions per custom resource.

[sebhoss]

Added this as part of #395 which now includes the PushSecrets CRD as well - thanks for the feedback here guys!

[colstrom]

Fabulous! Thank you for this!