feat: enable site clusters to run Nautobot Celery workers on site clusters

[haseebsyed12]

Nautobot currently runs entirely on the global cluster, including its Celery workers. Sites that generate heavy background task load have no way to offload that processing closer to where the work originates, and a single global worker pool becomes a bottleneck as sites scale.

This adds a site-scoped ArgoCD Application that deploys only the Celery worker portion of the Nautobot helm chart. The web server, Redis, and PostgreSQL are all disabled because they remain on the global cluster — site workers connect back to those shared services.

This lets operators scale worker capacity per-site independently, run queue-specific workers closer to the hardware they manage, and reduce cross-cluster task latency for site-driven automation.

• ArgoCD Application template (application-nautobot-worker.yaml) gated behind site.nautobot_worker.enabled
• It deploys only the Celery worker portion of the Nautobot helm chart into the nautobot namespace on the site cluster.

[skrobul]

left few comments inline

[skrobul]

This looks almost ready, added few comments in another review

[skrobul]

This expects very specific type of key/certificate and can be fragile.
Consider switching from a freeform text field to a JSON.

For example, if you store the credentials as JSON:

{
"tls.crt":"LS0tLS1CRUdJTiBDRVJ...",
"tls.key":"LS0tLS1CRUdJTiBFQyB...."
}

then you can use simpler and more robust template that does not need regular expressions:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: nautobot-mtls-client-marektest
  namespace: nautobot
spec:
  dataFrom:
  - extract:
      key: "602777"
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: pwsafe
  target:
    creationPolicy: Owner
    deletionPolicy: Retain
    template:
      data:
        tls.crt: '{{ index (.password | fromJson) "tls.crt" | b64dec }}'
        tls.key: '{{ index (.password | fromJson) "tls.key" | b64dec }}'
      engineVersion: v2
      metadata: {}
      type: kubernetes.io/tls

Alternatively, if you really want to store those as plaintext, please consider using PEM block filters provided by ESO instead of regexes.

[haseebsyed12]

done. using PEM block filters

[skrobul]

Nice, thanks. If you want to eliminate the regex and simplify the dataFrom section, sourcing data from multiple credentials like here is possible too, just need to assign secretKey for each remoteRef.

It's okay to leave it as is - your call.