[ServiceBus] message.dead_letter doesn't set any properties of message which is consistent with the .NET Sdk

[yunhaoling]

Customer report:

When you are receiving a message and explicitly try to deadletter it (message.dead_letter(description)), the description (string) doesn't show up in the properties of the message even when it is specified. I tried it with the regular dead_letter and also async dead_letter in Python, both returned these properties of the message:
"properties": {},
When I run the same test using the .NET SDK for Service Bus, it works as expected. Here are the message properties from .NET
"properties": {
"DeadLetterReason": "bad"
}

---

Repro:

    queue_client = client.get_queue(standard_queue)
    with queue_client.get_receiver(idle_timeout=5, mode=ReceiveSettleMode.PeekLock, prefetch=10) as receiver:

        with queue_client.get_sender() as sender:
            for i in range(10):
                message = Message("Dead lettered message no. {}".format(i))
                sender.send(message)

        count = 0
        messages = receiver.fetch_next()
        while messages:
            for message in messages:
                print_message(message)
                message.dead_letter(description="Testing queue deadletter")
                count += 1
            messages = receiver.fetch_next()

    assert count == 10

    with queue_client.get_deadletter_receiver(idle_timeout=5, mode=ReceiveSettleMode.PeekLock) as receiver:
        count = 0
        for message in receiver:
            print_message(message)
            assert message.user_properties
            assert message.user_properties[b"DeadLetterReason"]
            message.complete()
            count += 1
    assert count == 10

[MichaelWeedon]

Is there an ETA when this issue might be addressed?

[KieranBrantnerMagee]

Hey @MichaelWeedon, my apologies for the slow reply, am digging into this issue and will loop back with ETA once we get a sense for the resolution.

Thanks in advance for all your patience thus far.

[yunhaoling]

Hi @MichaelWeedon , thanks for your patience and sorry for letting you wait for such a long time.

I have created a patch wheel for the issue, can you test whether it solves the problem on your side?
dead-letter-description-patch.zip

(Code change could be found here if you're interested in implementation detail #10246)

[MichaelWeedon]

@yunhaoling
installed the patch - getting an exception in dead_letter().
Suspect I am missing something.

File "/Users/xxxx/projects/venv/lib/python3.6/site-packages/azure/servicebus/aio/async_message.py", line 108, in dead_letter
raise MessageSettleFailed("reject", e)
azure.servicebus.common.errors.MessageSettleFailed: Failed to reject message. Error: Management request failed: Management request returned status code: 500. Description: b'Unsupported type 0x40 for array. TrackingId:2d0f849e-1528-4642-a8cc-ee6fc813dcd1_G4, SystemTracker:NoSystemTracker, Timestamp:2020-03-11T14:38:23', Data: None

[yunhaoling]

@MichaelWeedon,

Which version of uamqp library are you using, can you try installing the the latest uamqp library by pip install uamqp --upgrade? -- that should install uamqp v1.2.6 for you.

We're aware of the issue Error: Management request failed: Management request returned status code: 500. Description: b'Unsupported type 0x40 for array. and made a fix in the uamqp library.

[MichaelWeedon]

@yunhaoling
Already at 1.2.6 of uamqp.

$ pip list | grep uamqp
uamqp 1.2.6
$ pip install uamqp --upgrade
Requirement already up-to-date: uamqp in /Users/xxx/projects/venv/lib/python3.6/site-packages (1.2.6)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /Users/xxx/projects/venv/lib/python3.6/site-packages (from uamqp) (2019.9.11)
Requirement already satisfied, skipping upgrade: six~=1.0 in /Users/xxx/projects/venv/lib/python3.6/site-packages (from uamqp) (1.11.0)

[yunhaoling]

@MichaelWeedon
That's interesting, when I test the functionality it works for me.

• Which platform are you using, windows/linux/mac?
• Are you doing some long-period operation on a single message? It would be great if you could share me with the simplified code to reproduce the b'Unsupported type 0x40 for array. issue.

Thanks

[MichaelWeedon]

@yunhaoling
I am on Mac OS.
I pulled the latest sdk samples and they have been reworked and I no longer get the 0x40.
Using example_session_send_receive_with_pool_async.py in async_samples
replace line #32 with the following (sending all message to dead_leter)
await message.dead_letter("Bad Message")
but still no dead letter attributes

[yunhaoling]

@MichaelWeedon thanks for the additional context, I would try to reproduce the issue.
Will keep you updated.

[yunhaoling]

@MichaelWeedon sorry for not getting back to you sooner.

It took me some time to figure out the difference between session/non-session entity when handling dead letter. I've made another patch, this one works well for both type of entity on my side. Can you help verify the patch solves your problem or not?

azure_servicebus-0.50.3b2-py2.py3-none-any.zip

[MichaelWeedon]

@yunhaoling, installed latest patch and do not see a difference in behavior in the samples or my program - meaning no attributes in dead letter. Do you have code that illustrates this properly sending the attributes to the dead letter queue?

[yunhaoling]

@MichaelWeedon Here is the code I used, please replace CONN_STR , QUEUE_NAME and SESSION_ID with your own configs.

from azure.servicebus.aio import ServiceBusClient, Message
from azure.servicebus import ReceiveSettleMode
import asyncio

import logging
logging.basicConfig(level=logging.INFO)

_logger = logging.getLogger(__name__)


CONN_STR = "<YOUR CONNECTION STR>"
QUEUE_NAME = "<YOUR QUEUE NAME>"
SESSION_ID = "<YOUR SESSION ID>"


def print_message(message):
    _logger.info("Receiving: {}".format(message))
    _logger.debug("Time to live: {}".format(message.time_to_live))
    _logger.debug("Sequence number: {}".format(message.sequence_number))
    _logger.debug("Enqueue Sequence number: {}".format(message.enqueue_sequence_number))
    _logger.debug("Partition ID: {}".format(message.partition_id))
    _logger.debug("Partition Key: {}".format(message.partition_key))
    _logger.info("User Properties: {}".format(message.user_properties))
    _logger.debug("Annotations: {}".format(message.annotations))
    _logger.debug("Delivery count: {}".format(message.header.delivery_count))
    try:
        _logger.debug("Locked until: {}".format(message.locked_until))
        _logger.debug("Lock Token: {}".format(message.lock_token))
    except TypeError:
        pass
    _logger.debug("Enqueued time: {}".format(message.enqueued_time))


async def main():
    client = ServiceBusClient.from_connection_string(conn_str=CONN_STR)

    queue_client = client.get_queue(QUEUE_NAME)
    async with queue_client.get_receiver(session=SESSION_ID, idle_timeout=10, mode=ReceiveSettleMode.PeekLock, prefetch=10) as receiver:
        async with queue_client.get_sender() as sender:
            for i in range(10):
                message = Message("Dead lettered message no. {}".format(i))
                message.session_id = SESSION_ID
                await sender.send(message)

        count = 0
        messages = await receiver.fetch_next()
        while messages:
            for message in messages:
                print_message(message)
                await message.dead_letter(description="Testing queue deadletter")
                count += 1
            messages = await receiver.fetch_next()

    assert count == 10

    async with queue_client.get_deadletter_receiver(idle_timeout=10, mode=ReceiveSettleMode.PeekLock) as receiver:
        count = 0
        async for message in receiver:
            print_message(message)
            assert message.user_properties[b'DeadLetterReason'] == b'Testing queue deadletter'
            assert message.user_properties[b'DeadLetterErrorDescription'] == b'Testing queue deadletter'
            await message.complete()
            count += 1
    assert count == 10

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

[MichaelWeedon]

@yunhaoling, Thanks. Able to get it to work. One caveat is I cannot get the reason (i.e description) to be set if I process the dead_letter within a handle exception. Meaning if while processing the message an exception occurs, I want to dead_letter the message using the exception as the reason. In this scenario, no reason is set - thoughts?

[yunhaoling]

Hi @MichaelWeedon , can you share me with code that reproduce the issue? -- dead_letter message when an exceptions occurs and use the exception as the reason within a handle exception.

[MichaelWeedon]

@yunhaoling Upon further investigation, it appears the problem is corrected for all cases. For some reason my pycharm environment was not picking up the newer sdk you supplied.
That said, when can this me made generally available?

[yunhaoling]

Hi @MichaelWeedon , thanks for the feedback, glad it works for you.

Currently we're triaging some other ServiceBus issues and try to get them resolved.
I can't guarantee the time for the next release but it's definitely in our plan.

We'll keep you updated about the release plan, thanks for your patience in advance.

[yunhaoling]

Hi @MichaelWeedon , apologize for the late response. We've released azure-servicebus v0.50.3 today which should have the issue addressed. Please update the library.

---

The important thing worth your attention here is that v0.50.3 would be our final release for v0.50 -- it is about to be deprecated.

We're shipping a brand-new azure-servicebus v7.0.0 (which is currently in b2) to build a more user friendly and productive library following our SDK design guidelines. We decided not to maintain the old preview (v0.50) library which will never go GA.

We've provided migration guide on v0.50 to v7 to help get you onboard.

Apologize for the inconvenience we bring to you and thanks for your understanding and patience.

We're looking forward to you trying our latest preview library and feedback is most welcome.

[MichaelWeedon]

My apologies for the previous response (deleted).
This is working correctly in 0.50.3 - Thanks!