Minor update to make Dapr pubsub components compliant

[aggr0cr4g]

Gist

Two Dapr pubsub component names in libs/common/common/queues.py use underscores (document_conversion, workflow_monitor) which are invalid for Kubernetes resource metadata.name fields. This silently breaks the document conversion and workflow monitoring pipelines when trying to deploy to Kubernetes or any native K8s Dapr setup.

Main Issue

Kubernetes resource names must follow https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-subdomain-names. lowercase alphanumeric characters or hyphens only. No underscores.

The current pubsub names in libs/common/common/queues.py:

DOCUMENT_CONVERSION_PUBSUB = "document_conversion"
WORKFLOW_MONITOR_PUBSUB = "workflow_monitor" 

The other five pubsub names (alerting, files, dotnet, dpapi, noseyparker) are single words with no separators and work fine in both Docker Compose and Kubernetes.

Sanity Check

Why does this work with Docker Compose??

Dapr runs in self-hosted mode and loads component from infra/dapr/components/pubsub/. It appears that the self-hosted Dapr does not enforce component name requirement, so document_conversion works. However, in K8s, Dapr components appear to be deployed as CRDs. Kubernetes validates metadata.name rejects underscores. So the CRD must use a compliant name likedocument-conversion (hyphen), but the application code looks for document_conversion (underscore) so Dapr returns:

pubsub document_conversion is not found

[leechristensen]

Thank you! Made a couple of mods to hopefully address some other issues you may have too. Let us know if running it in k8s works alright for you. Definitely interested!

[leechristensen]

In case you want to test it out (it's under dev still, but may work for you), we're experimenting with getting k8s deployment working in another branch. Testing with k3d right now, but going to test k3s as well. Any feedback welcomed!