Skip to content

AMQP producers: emit message_annotations from manifest (needed for x-opt-partition-key on Service Bus / Event Hubs) #294

Description

@clemensv

Summary

The Python AMQP producer template (xrcg/templates/py/amqpproducer) currently reads protocoloptions.properties (AMQP message-properties section) and protocoloptions.application_properties (AMQP application-properties section), but does not emit anything for the AMQP message-annotations section.

Result: manifests cannot declare partition keys that reach the broker.

Why this matters

For Azure Service Bus and Azure Event Hubs, partition routing is driven entirely by the annotation b"x-opt-partition-key" in the AMQP message-annotations section. Confirmed in the Microsoft Python SDKs:

  • azure/servicebus/_common/constants.py:119 -> _X_OPT_PARTITION_KEY = b"x-opt-partition-key"
  • azure/servicebus/_common/message.py:331 -> self._set_message_annotations(_X_OPT_PARTITION_KEY, value) (string, max 128 chars)
  • azure/eventhub/_constants.py:10 -> PROP_PARTITION_KEY = b"x-opt-partition-key"
  • azure/eventhub/_transport/_pyamqp_transport.py:393 -> annotations[PROP_PARTITION_KEY] = partition_key

Without xrcg emitting this annotation, downstream feeders either lose partition ordering or have to monkey-patch the generated _sender.send from the bridge.

Requested change

Extend the AMQP producer template to read a new section protocoloptions.message_annotations (peer to properties and application_properties):

  • For each entry whose value is a uritemplate, accept the corresponding _<placeholder> kwarg on send_* / send_*_batch.
  • Emit ann[b"<name>"] = "<expanded>" and assign to amqp_msg.annotations. Symbol-typed keys must be encoded as bytes per the AMQP 1.0 spec - Service Bus / Event Hubs reject string keys.
  • For x-opt-partition-key specifically: enforce string value, truncate to 128 chars on encode (Service Bus MESSAGE_PROPERTY_MAX_LENGTH).

Same change should land in the C#, Java, and TypeScript AMQP producer templates for parity.

Repo impact

Blocks the amqp-feeder skill default partition-key recipe in clemensv/real-time-sources. Pegelonline currently ships a bridge-side monkey-patch workaround documented in stream-bridge-implementation; we want to drop it as soon as the template supports annotations natively.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions