eventhub: the program crashes if producer not use with with

[davidt99]

I'm testing the new version of the SDK. If I'm not surrounding the producer with with statement, after the producer is clean up by the python interpreter, the program just crashes with double free or corruption (fasttop) message.
I do understand that the with statement is required, but this poses a couple of problems:

1. Python's memory management is done by the interpreter, the developer usually doesn't need to bother when the memory is free, the with statement is there to make sure all of the resources are clean up at the right time but not using it should never cause the program to crash.
2. My scenario consists of sending single event at a time. I can't reuse the producer when I'm forced to use it with a with statement. Creating new producer takes about 2 seconds so that not an option as well.

[kaerm]

@davidt99 thanks for reporting the issue - our team will looking into this as well //cc: @yunhaoling

[YijunXieMS]

Hi @davidt99, thanks for reporting the issue.
In your case, you can close the producer by calling producer.close() when you don't need that producer any more. The "with" statement actually calls this "close" method.
We'll look into cleaning up if no close method is called.
Are you using sync or async EventHubProducerClient?

[YijunXieMS]

@davidt99 were you trying with 5.0.0b4 or 5.0.0b5? We happened to release 5.0.0b5 last night around 9pm PST. So not sure you're talking about 5.0.0b4 or b5. Both are preview versions though.

[davidt99]

Not using the async, I was just doing a simple test. I was using 5.0.0b4.
At the moment I don't want to keep the producer around without the with statement because of the crash issue in case it gets freed by the GC.

[YijunXieMS]

@davidt99 Then please call producer.close() just before you don't need the producer variable any more.

[davidt99]

@YijunXieMS, yes I understand. As I said, I'll wait for the program crash fix before implementing this in production.

[lmazuel]

After investigation, that's a uamqp issue (azure-eventhub is not doing any crazy things), fix in progress:
Azure/azure-uamqp-python#116

[yunhaoling]

Hello @davidt99 ,

we've released the uamqp v1.2.5 including the fix: https://pypi.org/project/uamqp/1.2.5
can you try installing the latest uamqp library to see the issue still occurs on your side?

[yunhaoling]

Hello @davidt99 , is there any updates for this issue?

[davidt99]

Didn't have the time to test it, will report back next week.

…

On Tue, 17 Dec 2019, 11:47 Adam Ling (MSFT), ***@***.***> wrote: Hello @davidt99 <https://github.com/davidt99> , is there any updates for this issue? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8404?email_source=notifications&email_token=AB7JFPDQNCJYGCHDJO625NTQZA4VJA5CNFSM4JJDVZB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHA6MYI#issuecomment-566355553>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7JFPA6BHHWNVP6KLB6VK3QZA4VJANCNFSM4JJDVZBQ> .

[yunhaoling]

thanks @davidt99 , looking forwarding to your reply!

[davidt99]

@yunhaoling, using 5.0.0b6, the crash no longer happens, although "Can't call on_state_changed during garbage collection, plase be sure to close or use a context manager" is shown but I guess it makes sense (besides the typo 😉).

[lmazuel]

I take ownership for the typo :p
Azure/azure-uamqp-python#136

Yes, I think this warning if fair. If the client is not closed properly, the message sender/receiver are garbage collected while still open, and because the callback system is define recursively on the class itself, calling a callback on a garbage collected object is not possible.