Add non-CRD yaml file

[rittneje]

Currently the GitHub releases include two assets (among others): cert-manager.crds.yaml and cert-manager.yaml. The former only includes the CRD specs, while the latter is everything (CRDs, deployments, service accounts, etc.).

I would like to suggest that another asset be added for the non-CRD resources (i.e., the difference between the two). This is because sometimes we have to make tweaks to these specs to make it work properly, and it would be a lot easier to have a single file we can use as a base every upgrade instead of having to copy just the tail end of the full cert-manager.yaml file. But we don't make modifications to the (enormous) CRD specs.

[inteon]

Hi @rittneje,

Can you please give some more information on what tweaks you are making to the yaml files? Are you using any tools for that, eg. kustomize? Cert-manager supports tuning these yaml manifests using Helm's templating engine. You can use Helm to install cert-manager directly: https://cert-manager.io/docs/installation/kubernetes/#installing-with-helm, or you can use Helm to generate the static yaml manifests that can be applied using kubectl apply -f .... (see https://helm.sh/docs/helm/helm_template/ for more information)

This documentation contains more info on all supported helm parameters:
https://artifacthub.io/packages/helm/cert-manager/cert-manager

[rittneje]

We do not use Helm. We just have a copy to the manifest with the changes to it in our repo.
We made the following tweaks:

1. Added the eks.amazonaws.com/role-arn annotation to the cert-manager ServiceAccount. This is needed so it can interact with Route53 for DNS ACME challenges.
2. Set securityContext.fsGroup in the cert-manager Deployment. This is needed so that uid 1000 can access the mounted token file (i.e., WEB_IDENTITY_TOKEN_FILE).
3. Explicitly passed the --tls-min-version flag to the cert-manager-webhook container.

[inteon]

The following commands should generate the requested static yaml manifest:

$ helm repo add jetstack https://charts.jetstack.io
$ helm repo update
$ helm template \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v1.4.0 \
 > cert-manager.no-crds.yaml

I'll check if it would be possible to also add this manifest to upcoming releases.
The next release will certainly include a kubectl plugin that can generate these manifests (#4138), this should be easier than using Helm.

[wallrj]

@inteon I discovered that you can also use label selectors in kubectl create / apply so these commands might also help:

# All non-crds (CRDs don't have a component label)
kubectl create \
    -f https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.yaml \
    --dry-run=client \
    -o yaml \
    -l 'app.kubernetes.io/component'

# Just the webhook and controller components
kubectl create \
    -f https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.yaml \
    --dry-run=client \
    -o yaml \
    -l 'app.kubernetes.io/component in (webhook,controller)' 

But perhaps we should consider whether to provide manifests suitable for a multi-stage install, similar to that documented by linkerd

• https://linkerd.io/2.10/tasks/install/#multi-stage-install

Although, I'd have thought it should be split as follows:

• API (CRDs + Webhooks + Cainjector + related services and RBAC)
• Control-plane (controller manager + related services and RBAC)

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[rittneje]

/remove-lifecycle stale

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[rittneje]

/remove-lifecycle stale

[kriswuollett]

I usually use kubernetes_manifest in Terraform using for_each on filesets for CRDs/Namespaces, and then on "Services" -- very similar to the kbst/kustomize pattern when I don't need to use kustomize itself. This allows for an explicit depends_on relationship to make sure CRDs are in place before anything that may require them. I'd agree that it makes sense for the contents of cert-manager.crds.yaml to not be in cert-manager.yaml. Furthermore, I'd recommend that a separate file cert-manager.webhooks.yaml is made as well for the same reasons.

[kriswuollett]

A possible way to very flexible for various install methods, is to keep cert-manager.yaml as the all in one basic install, but also provide a cert-manager-yaml.tar.gz archive that contains a yaml file for each of the object types, like cert-manager-[object type].yaml, so downstream can easily consume/repackage as needed, e.g., how prometheus-operator/kube-prometheus/manifests is distributed.

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[rittneje]

/remove-lifecycle stale

[WoodyWoodsta]

On a similar topic - helm doesn't wait for CRDs to install before applying resources which depend on those CRDs. This means that technically, installCRDs is not helpful unless the rest of your chart doesn't include any of said resources. A better design IMO is to do what eck have done, which is to have two separate charts, one for CRDs and one for the controller.

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[rittneje]

/remove-lifecycle stale

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[rittneje]

/remove-lifecycle stale

[btrepp]

It would be great to have things like this to make Kustomize easier to use.

You can reference the compiled yaml, but it doesn't compose nicely. I was trying to have a 'system' setup, which puts thinks like ingress, cert-manager, metal-lb in a system namespace.

With metal-lb I can grab parts, but with cert-manager it includes a namespace definition, and then that conflicts with other tools that have a namespace definition, and as it's only as a one file, I might have to manually edit that by hand to then install it. Kustomizes philosophy is fairly additive, e.g you add in more parts, but I don't think it has 'take this base, and remove these components', so the big bundle does cause friction with it.

It would be great if the various components where published in parts too, that way you can pick which ones and use overlays afterwards to tweak the setup.

It would also be great for managing CRDs, as they are 'super admin scoped' anyway, you might want to have 'crd' maintenance and 'cert-manager' maintainer be different roles, but as the yaml contains the CRD there's a bit of an overlap

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[rittneje]

/remove-lifecycle stale

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[wallrj]

/remove-lifecycle stale

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[rittneje]

/remove-lifecycle stale

[cert-manager-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
/lifecycle stale

[rittneje]

/remove-lifecycle stale