Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are closed links allowed to fire error events? #205

Closed
ramya-rao-a opened this issue Mar 11, 2019 · 16 comments
Closed

Are closed links allowed to fire error events? #205

ramya-rao-a opened this issue Mar 11, 2019 · 16 comments

Comments

@ramya-rao-a
Copy link
Contributor

Below is the sequence of events relevant to this issue

  1. A receiver link is set up and is receiving messages
  2. Network connection is lost, the rhea connection object fires the disconnected event, the receiver link is now in a "closed" state. Our application hasn't called close on it.
  3. Network connection is back, and we want to resume receiving messages.
  4. We set up hand shake with our server for authentication, this invalidates the auth tokens for the previous connection. At this point the server sends out an "Unauthorized Access" error for all links that were on the previous connection
  5. So now, even though the original receiver link (from step 1) is in a closed state, it fires the error event.

It's hard for us to give a code sample to reproduce this because we would need the server to send the error. Hopefully, the above gives enough details about our situation.

My question:

  • When the link is in a "closed" state, why is the server able to send a error for it?
  • Can we expect rhea to completely "close" and clean up the link in this scenario?
  • Or is it the responsibility of the application, to call remove on the link when it gets the disconnected event on the connection?

cc @grs, @amarzavery

@grs
Copy link
Member

grs commented Mar 11, 2019

In 4. is it a connection error? I assume it can't be a link error in the AMQP sense if the link is not open. I think this is the question you ask 'When the link is in a "closed" state, why is the server able to send a error for it?' and the answer is that it can't. If the link has not been reopened after disconnection, the server cannot send a link error only a connection error (and rhea does not emit these on the link). Is it possible that rhea-promise is adding a listener for connection events to a receiver object and that is what is firing?

At present rhea does not remove listeners from any object at any state. While we could change that, my instinct would only be to do so in the case where the link.session/connection is fully closed (i.e. both sides have closed it) or at the very least where the local application has closed it. If I understand correctly, step 2. involves the implicit closing of a link due to loss of a connection and in that case I think the application should take some specific action.

So to 'Can we expect rhea to completely "close" and clean up the link in this scenario?' I would say no, and implicitly closed link with no reconnect will remain implicitly closed and I don't think rhea should do anything further without some explicit action by the application to indicate it no longer cares about that object, e.g. as you suggest by removing the link.

@ramya-rao-a
Copy link
Contributor Author

ramya-rao-a commented Mar 11, 2019

In 4. is it a connection error? I assume it can't be a link error in the AMQP sense if the link is not open.

It is not a connection error. It is an error on the original receiver link. We added the link name in the log that prints event being fired from rhea and it clearly shows that the error is coming from the original receiver link.

If the link has not been reopened after disconnection, the server cannot send a link error only a connection error (and rhea does not emit these on the link)

We also noticed a receiver_open event being fired on this receiver link just before the receiver_error event was fired. But I have no idea how this could happen, we arent opening this receiver from our side. Do you? Is it possible for a closed receiver link to re-open once the connection is back up?

Below are the relevant logs. DEBUG was set to rhea:events*. We have updated the Link got event log message to include the name of the link so that we can differentiate the various link events

rhea:events [connection-1] Connection got event: connection_open +0ms
  rhea:events [connection-1] Session got event: session_open +85ms
  rhea:events [connection-1] Link got event: sender_open Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +82ms
  rhea:events [connection-1] Link got event: sender_flow Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +2ms
  rhea:events [connection-1] Link got event: sendable Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +1ms
  rhea:events [connection-1] Link got event: receiver_open Link Name : cbs-beb64401-e24d-864a-ad87-1206edd7ba8c +79ms
  rhea:events [connection-1] Received disposition for outgoing transfers +204ms
  rhea:events [connection-1] Link got event: accepted Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +2ms
  rhea:events [connection-1] Link got event: settled Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +1ms
  rhea:events [connection-1] Link got event: message Link Name : cbs-beb64401-e24d-864a-ad87-1206edd7ba8c +2ms
  rhea:events [connection-1] Session got event: session_open +198ms
  rhea:events [connection-1] Link got event: sender_open Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +79ms
  rhea:events [connection-1] Link got event: sender_flow Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +1ms
  rhea:events [connection-1] Link got event: sendable Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +1ms
  rhea:events [connection-1] Link got event: receiver_open Link Name : e56ccd28-5d53-48ba-b95c-aceb49d2e113 +78ms
  rhea:events [connection-1] Received disposition for outgoing transfers +200ms
  rhea:events [connection-1] Link got event: accepted Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +2ms
  rhea:events [connection-1] Link got event: settled Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +0ms
  rhea:events [connection-1] Link got event: message Link Name : e56ccd28-5d53-48ba-b95c-aceb49d2e113 +2ms
  rhea:events [connection-1] Received disposition for outgoing transfers +200ms
  rhea:events [connection-1] Link got event: accepted Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +1ms
  rhea:events [connection-1] Link got event: settled Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +1ms
  rhea:events [connection-1] Link got event: message Link Name : cbs-beb64401-e24d-864a-ad87-1206edd7ba8c +1ms
  rhea:events [connection-1] Session got event: session_open +200ms
  rhea:events [connection-1] Link got event: receiver_open Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +80ms
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +79ms
### Actual message: Hello foo 1
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +5ms
### Actual message: Hello foo 2
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +2ms
### Actual message: Hello foo 3
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +1ms
### Actual message: Hello foo 4
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +2ms
### Actual message: Hello foo 5
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +1ms
### Actual message: Hello foo 6
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +1ms
### Actual message: Hello foo 7
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +2ms
### Actual message: Hello foo 8
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +1ms
### Actual message: Hello foo 9
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +1ms
### Actual message: Hello foo 10
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +1ms
.
.
.
.
. 
.
### Actual message: Hello foo 570
  rhea:events [connection-1] Link got event: message Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +2ms
### Actual message: Hello foo 571
  rhea:events [connection-1] Connection got event: disconnected +4ms
  rhea:events [connection-1] Connection got event: disconnected +324ms
  rhea:events [connection-1] Connection got event: connection_open +16s
  rhea:events [connection-1] Session got event: session_open +7ms
  rhea:events [connection-1] Link got event: sender_open Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +1ms
  rhea:events [connection-1] Link got event: sender_flow Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +1ms
  rhea:events [connection-1] Link got event: sendable Link Name : 218be28e-3b8d-4e4a-89e1-69031007126d +1ms
  rhea:events [connection-1] Link got event: receiver_open Link Name : cbs-beb64401-e24d-864a-ad87-1206edd7ba8c +1ms
  rhea:events [connection-1] Session got event: session_open +1ms
  rhea:events [connection-1] Link got event: sender_open Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +1ms
  rhea:events [connection-1] Link got event: sender_flow Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +1ms
  rhea:events [connection-1] Link got event: sendable Link Name : c0558a4d-b8eb-dc4d-8c7f-5f24fef974bc +1ms
  rhea:events [connection-1] Link got event: receiver_open Link Name : e56ccd28-5d53-48ba-b95c-aceb49d2e113 +0ms
  rhea:events [connection-1] Session got event: session_open +1ms
  rhea:events [connection-1] Session got event: session_open +63ms
  rhea:events [connection-1] Link got event: receiver_open Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +180ms
  rhea:events [connection-1] Link got event: receiver_error Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +7ms
Receiver error handler ---- receiver.isOpen() : false
      receiver.isRemoteOpen() : false
      receiver.isClosed() :false
      isItselfClosed():  false
      receiver.isSessionClosed(): false
>>>>> Error occurred:  { UnauthorizedError: Unauthorized access. 'Listen' claim(s) are required to perform this operation....
  name: 'UnauthorizedError',
  translated: true,
  retryable: false,
  info: null,
  condition: 'amqp:unauthorized-access' }
  rhea:events [connection-1] Link got event: receiver_close Link Name : 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 +61ms

Things to note:

  • 724d1fc5-29ea-4dc6-8b64-810ac6fe1b74 is the name of our receiver
  • The sender, receiver links you see other than this one are part of the handshake with our server for auth
  • This receiver fires a receiver_open event followed by the receiver_error event when the Connection comes back up.
  • At this point, we are printing the isOpen(), isRemoteOpen(), isClosed(), isItselfClosed() and isSessionClosed() for the receiver. All turn out to be false

@grs
Copy link
Member

grs commented Mar 11, 2019

So the link is not closed at the point the error is sent by the server. The link was reopened and then closed by server with an error.

@ramya-rao-a
Copy link
Contributor Author

What should we look for in the logs to confirm that the server has indeed re-opened the link?

@grs
Copy link
Member

grs commented Mar 12, 2019

The receiver_open event shows that. You can also turn on frame tracing and you should see the attach. I believe the attach will be triggered by your client (either reconnect, or if that is off then the application or rhea-promise library).

@ramya-rao-a
Copy link
Contributor Author

@grs It looks like rhea is sending the request to open the link. Below is the logs with rhea frames. If you look at each attach after the disconnected event and before the connection_open event, you will see 3 outgoing attach logs for each of the 3 links we have.

Once the connection_open event is fired, we have the incoming attach logs for these 3 links

Correct me if I am wrong, but this looks like rhea is trying to reconnect the links.

FYI: When creating the connection we have set reconnect to false in the Connection Options


  rhea:frames [connection-1] -> { protocol_id: 3, major: 1, minor: 0, revision: 0 } +0ms
  rhea:frames [connection-1] <- { protocol_id: 3, major: 1, minor: 0, revision: 0 } +85ms
  rhea:raw [connection-1] RECV: 63 0000003f02010000005340c03201e02f04b3000000074d53534243425300000005504c41494e00000009414e4f4e594d4f55530000000845585445524e414c +0ms
  rhea:frames [connection-1]:0 <- sasl_mechanisms#40 {"sasl_server_mechanisms":["MSSBCBS","PLAIN","ANONYMOUS","EXTERNAL"]}  +7ms
  rhea:frames [connection-1]:0 -> sasl_init#41 {"mechanism":"ANONYMOUS","initial_response":{"type":"Buffer","data":[0,82,111,111,116,77,97,110,97,103,101,83,104,97,114,101,100,65,99,99,101,115,115,75,101,121]},"hostname":"testeventhubssample.servicebus.windows.net"}  +4ms
  rhea:raw [connection-1] SENT: 103 0000006702010000005341d00000005700000003a309414e4f4e594d4f5553a01a00526f6f744d616e6167655368617265644163636573734b6579a12a746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e6574 +9ms
  rhea:raw [connection-1] RECV: 26 0000001a02010000005344c00d025000a00857656c636f6d6521 +75ms
  rhea:frames [connection-1]:0 <- sasl_outcome#44 {"additional_data":{"type":"Buffer","data":[87,101,108,99,111,109,101,33]}}  +77ms
  rhea:frames [connection-1] -> { protocol_id: 0, major: 1, minor: 0, revision: 0 } +1ms
  rhea:frames [connection-1]:0 -> open#10 {"transport":"tls","host":"testeventhubssample.servicebus.windows.net","username":"RootManageSharedAccessKey","port":5671,"operationTimeoutInSeconds":60,"servername":"testeventhubssample.servicebus.windows.net","id":"connection-1","container_id":"af44bc38-0ebc-5549-83fc-1a993e584b54","hostname":"testeventhubssample.servicebus.windows.net","properties":{"product":"MSJSClient","version":"1.0.5","user-agent":"/js-event-hubs","platform":"(x64-Windows_NT-10.0.17763)","framework":"Node/v10.14.1"}}  +1ms
  rhea:raw [connection-1] SENT: 248 000000f802000000005310d0000000e80000000aa12461663434626333382d306562632d353534392d383366632d316139393365353834623534a12a746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e657440404040404040d1000000860000000aa30770726f64756374a10a4d534a53436c69656e74a30776657273696f6ea105312e302e35a30a757365722d6167656e74a10e2f6a732d6576656e742d68756273a308706c6174666f726da11b287836342d57696e646f77735f4e542d31302e302e313737363329a3096672616d65776f726ba10d4e6f64652f7631302e31342e31 +5ms
  rhea:frames [connection-1] <- { protocol_id: 0, major: 1, minor: 0, revision: 0 } +76ms
  rhea:raw [connection-1] RECV: 71 0000004702000000005310c03a0aa12437376234656266613639303234656239623039383862633630656462323832305f473135407000010000601387700003a9804040404040 +148ms
  rhea:frames [connection-1]:0 <- open#10 {"container_id":"77b4ebfa69024eb9b0988bc60edb2820_G15","max_frame_size":65536,"channel_max":4999,"idle_time_out":240000}  +76ms
  rhea:events [connection-1] Connection got event: connection_open +0ms
  rhea:frames [connection-1]:0 -> begin#11 {"incoming_window":2048,"outgoing_window":4294967295}  +6ms
  rhea:raw [connection-1] SENT: 32 0000002002000000005311d000000010000000044043700000080070ffffffff +11ms
  rhea:raw [connection-1] RECV: 34 0000002202000000005311c0150860000052017000001388700000080052ff404040 +74ms
  rhea:frames [connection-1]:0 <- begin#11 {"next_outgoing_id":1,"incoming_window":5000,"outgoing_window":2048,"handle_max":255}  +77ms
  rhea:events [connection-1] Session got event: session_open +84ms
  rhea:frames [connection-1]:0 -> attach#12 {"name":"1629e36b-a2d1-e340-a4a4-8eee0039b5e1","source":[],"target":["$cbs"]}  +6ms
  rhea:raw [connection-1] SENT: 87 0000005702000000005312d0000000470000000aa12431363239653336622d613264312d653334302d613461342d3865656530303339623565314342404000532845005329d00000000a00000001a10424636273404043 +7ms
  rhea:raw [connection-1] RECV: 106 0000006a02000000005312c05d0ea12431363239653336622d613264312d653334302d613461342d38656565303033396235653143414040005328c00c0b4040404040404040404040005329c00d07a1042463627340404040404040404080ffffffffffffffff404040 +75ms
  rhea:frames [connection-1]:0 <- attach#12 {"name":"1629e36b-a2d1-e340-a4a4-8eee0039b5e1","role":true,"source":[null,null,null,null,null,null,null,null,null,null,null],"target":["$cbs",null,null,null,null,null,null],"max_message_size":{"type":"Buffer","data":[255,255,255,255,255,255,255,255]}}  +78ms
  rhea:events [connection-1] Link got event: sender_open +84ms
  rhea:raw [connection-1] RECV: 35 0000002302000000005313c0160b437000001388520170000008004343526443404240 +6ms
  rhea:frames [connection-1]:0 <- flow#13 {"incoming_window":5000,"next_outgoing_id":1,"outgoing_window":2048,"link_credit":100}  +4ms
  rhea:events [connection-1] Link got event: sender_flow +3ms
  rhea:events [connection-1] Link got event: sendable +1ms
  rhea:frames [connection-1]:0 -> attach#12 {"name":"cbs-fcf01ba6-0dcc-5542-8288-d5095f9ee0a4","handle":1,"role":true,"source":["$cbs"],"target":[]}  +6ms
  rhea:raw [connection-1] SENT: 89 0000005902000000005312d00000004900000007a1286362732d66636630316261362d306463632d353534322d383238382d6435303935663965653061345201414040005328d00000000a00000001a1042463627300532945 +9ms
  rhea:raw [connection-1] RECV: 111 0000006f02000000005312c0620ea1286362732d66636630316261362d306463632d353534322d383238382d6435303935663965653061345201424040005328c0110ba1042463627340404040404040404040005329c008074040404040404040404380ffffffffffffffff404040 +73ms
  rhea:frames [connection-1]:0 <- attach#12 {"name":"cbs-fcf01ba6-0dcc-5542-8288-d5095f9ee0a4","handle":1,"source":["$cbs",null,null,null,null,null,null,null,null,null,null],"target":[null,null,null,null,null,null,null],"max_message_size":{"type":"Buffer","data":[255,255,255,255,255,255,255,255]}}  +77ms
  rhea:events [connection-1] Link got event: receiver_open +82ms
  rhea:frames [connection-1]:0 -> flow#13 {"incoming_window":2048,"outgoing_window":4294967295,"handle":1,"link_credit":1000}  +3ms
  rhea:raw [connection-1] SENT: 40 0000002802000000005313d000000018000000074370000008004370ffffffff52014370000003e8 +5ms
  rhea:message Encoding section 1 of 4: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 112 } } +0ms
  rhea:message Encoding section 2 of 4: Typed { type: TypeDesc { name: 'List32', typecode: 208, width: 4, category: 3, create: { [Function] typecode: 208 } }, value: [ Typed { type: [TypeDesc], value: '0181260a-8f4e-c142-be84-930a1a3562c1' }, <1 empty item>, Typed { type: [TypeDesc], value: '$cbs' }, <1 empty item>, Typed { type: [TypeDesc], value: 'cbs-fcf01ba6-0dcc-5542-8288-d5095f9ee0a4' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } +4ms
  rhea:message Encoding section 3 of 4: Typed { type: TypeDesc { name: 'Map32', typecode: 209, width: 4, category: 3, create: { [Function] typecode: 209 } }, value: [ Typed { type: [TypeDesc], value: 'operation' }, Typed { type: [TypeDesc], value: 'put-token' }, Typed { type: [TypeDesc], value: 'name' }, Typed { type: [TypeDesc], value: 'sb://testeventhubssample.servicebus.windows.net/myeventhub/$management' }, Typed { type: [TypeDesc], value: 'type' }, Typed { type: [TypeDesc], value: 'servicebus.windows.net:sastoken' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 116 } } +5ms
  rhea:message Encoding section 4 of 4: Typed { type: TypeDesc { name: 'Str8', typecode: 161, width: 1, category: 2, encoding: 'utf8', create: { [Function] typecode: 161 } }, value: 'SharedAccessSignature sr=sb%3A%2F%2Ftesteventhubssample.servicebus.windows.net%2Fmyeventhub%2F%24management&sig=dcLIZhxJmUsu3VS3h%2BLC9jSprULzqWOG%2FXNl8mnoh48%3D&se=1552434018&skn=RootManageSharedAccessKey', descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 119 } } +4ms
  rhea:message encoded 466 bytes +3ms
  rhea:frames [connection-1]:0 -> transfer#14 {"delivery_tag":{"type":"Buffer","data":[48]}} <Buffer 00 53 70 45 00 53 73 d0 00 00 00 5c 00 00 00 05 a1 24 30 31 38 31 32 36 30 61 2d 38 66 34 65 2d 63 31 34 32 2d 62 65 38 34 2d 39 33 30 61 31 61 33 35 ... > +31ms
  rhea:raw [connection-1] SENT: 494 000001ee02000000005314d00000000c000000064343a0013043424200537045005373d00000005c00000005a12430313831323630612d386634652d633134322d626538342d39333061316133353632633140a1042463627340a1286362732d66636630316261362d306463632d353534322d383238382d643530393566396565306134005374d10000008f00000006a1096f7065726174696f6ea1097075742d746f6b656ea1046e616d65a14673623a2f2f746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e65742f6d796576656e746875622f246d616e6167656d656e74a10474797065a11f736572766963656275732e77696e646f77732e6e65743a736173746f6b656e005377a1ce5368617265644163636573735369676e61747572652073723d7362253341253246253246746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e65742532466d796576656e746875622532462532346d616e6167656d656e74267369673d64634c495a68784a6d55737533565333682532424c43396a537072554c7a71574f47253246584e6c386d6e6f6834382533442673653d3135353234333430313826736b6e3d526f6f744d616e6167655368617265644163636573734b6579 +31ms
  rhea:raw [connection-1] RECV: 23 0000001702000000005315c00a06414340410053244540 +157ms
  rhea:frames [connection-1]:0 <- disposition#15 {"role":true,"settled":true,"state":[]}  +159ms
  rhea:events [connection-1] Received disposition for outgoing transfers +193ms
  rhea:events [connection-1] Link got event: accepted +1ms
  rhea:events [connection-1] Link got event: settled +1ms
  rhea:raw [connection-1] RECV: 141 0000008d02000000005314c0120b520143a004010000004340424040404042005373c0330d4040404040a12430313831323630612d386634652d633134322d626538342d39333061316133353632633140404040404040005374c13104a10b7374617475732d636f646571000000caa1127374617475732d6465736372697074696f6ea1084163636570746564 +5ms
  rhea:frames [connection-1]:0 <- transfer#14 {"handle":1,"delivery_tag":{"type":"Buffer","data":[1,0,0,0]}} <Buffer 00 53 73 c0 33 0d 40 40 40 40 40 a1 24 30 31 38 31 32 36 30 61 2d 38 66 34 65 2d 63 31 34 32 2d 62 65 38 34 2d 39 33 30 61 31 61 33 35 36 32 63 31 40 ... > +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List8', typecode: 192, width: 1, category: 3, create: { [Function] typecode: 192 } }, value: [ Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: '0181260a-8f4e-c142-be84-930a1a3562c1' }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +169ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'status-code' }, Typed { type: [TypeDesc], value: 202 }, Typed { type: [TypeDesc], value: 'status-description' }, Typed { type: [TypeDesc], value: 'Accepted' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 116 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 116 } +19ms
  rhea:events [connection-1] Link got event: message +33ms
  rhea:frames [connection-1]:0 -> disposition#15 {"role":true,"settled":true,"state":[]}  +34ms
  rhea:raw [connection-1] SENT: 28 0000001c02000000005315d00000000c000000054143434100532445 +38ms
  rhea:frames [connection-1]:1 -> begin#11 {"incoming_window":2048,"outgoing_window":4294967295}  +5ms
  rhea:raw [connection-1] SENT: 32 0000002002000001005311d000000010000000044043700000080070ffffffff +5ms
  rhea:raw [connection-1] RECV: 34 0000002202000001005311c0150860000152017000001388700000080052ff404040 +215ms
  rhea:frames [connection-1]:1 <- begin#11 {"remote_channel":1,"next_outgoing_id":1,"incoming_window":5000,"outgoing_window":2048,"handle_max":255}  +218ms
  rhea:events [connection-1] Session got event: session_open +230ms
  rhea:frames [connection-1]:1 -> attach#12 {"name":"5b1323d0-43de-794f-809e-567279b770a8","source":[],"target":["$management"]}  +5ms
  rhea:raw [connection-1] SENT: 94 0000005e02000001005312d00000004e0000000aa12435623133323364302d343364652d373934662d383039652d3536373237396237373061384342404000532845005329d00000001100000001a10b246d616e6167656d656e74404043 +11ms
  rhea:raw [connection-1] RECV: 113 0000007102000001005312c0640ea12435623133323364302d343364652d373934662d383039652d35363732373962373730613843414040005328c00c0b4040404040404040404040005329c01407a10b246d616e6167656d656e7440404040404040404080ffffffffffffffff404040 +76ms
  rhea:frames [connection-1]:1 <- attach#12 {"name":"5b1323d0-43de-794f-809e-567279b770a8","role":true,"source":[null,null,null,null,null,null,null,null,null,null,null],"target":["$management",null,null,null,null,null,null],"max_message_size":{"type":"Buffer","data":[255,255,255,255,255,255,255,255]}}  +92ms
  rhea:events [connection-1] Link got event: sender_open +99ms
  rhea:raw [connection-1] RECV: 35 0000002302000001005313c0160b437000001388520170000008004343526443404240 +65ms
  rhea:frames [connection-1]:1 <- flow#13 {"incoming_window":5000,"next_outgoing_id":1,"outgoing_window":2048,"link_credit":100}  +59ms
  rhea:events [connection-1] Link got event: sender_flow +56ms
  rhea:events [connection-1] Link got event: sendable +2ms
  rhea:frames [connection-1]:1 -> attach#12 {"name":"3d06f57d-f3db-4b82-89ba-11d203d93ab7","handle":1,"role":true,"source":["$management"],"target":["3d06f57d-f3db-4b82-89ba-11d203d93ab7"]}  +7ms
  rhea:raw [connection-1] SENT: 138 0000008a02000001005312d00000007a00000007a12433643036663537642d663364622d346238322d383962612d3131643230336439336162375201414040005328d00000001100000001a10b246d616e6167656d656e74005329d00000002a00000001a12433643036663537642d663364622d346238322d383962612d313164323033643933616237 +14ms
  rhea:raw [connection-1] RECV: 151 0000009702000001005312c08a0ea12433643036663537642d663364622d346238322d383962612d3131643230336439336162375201424040005328c0180ba10b246d616e6167656d656e7440404040404040404040005329c02d07a12433643036663537642d663364622d346238322d383962612d31316432303364393361623740404040404040404380ffffffffffffffff404040 +79ms
  rhea:frames [connection-1]:1 <- attach#12 {"name":"3d06f57d-f3db-4b82-89ba-11d203d93ab7","handle":1,"source":["$management",null,null,null,null,null,null,null,null,null,null],"target":["3d06f57d-f3db-4b82-89ba-11d203d93ab7",null,null,null,null,null,null],"max_message_size":{"type":"Buffer","data":[255,255,255,255,255,255,255,255]}}  +84ms
  rhea:events [connection-1] Link got event: receiver_open +89ms
  rhea:frames [connection-1]:1 -> flow#13 {"incoming_window":2048,"outgoing_window":4294967295,"handle":1,"link_credit":1000}  +5ms
  rhea:raw [connection-1] SENT: 40 0000002802000001005313d000000018000000074370000008004370ffffffff52014370000003e8 +15ms
  rhea:message Encoding section 1 of 4: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 112 } } +499ms
  rhea:message Encoding section 2 of 4: Typed { type: TypeDesc { name: 'List32', typecode: 208, width: 4, category: 3, create: { [Function] typecode: 208 } }, value: [ Typed { type: [TypeDesc], value: '1e69a047-6705-415c-b749-1a5c8123b8e6' }, <3 empty items>, Typed { type: [TypeDesc], value: '3d06f57d-f3db-4b82-89ba-11d203d93ab7' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } +4ms
  rhea:message Encoding section 3 of 4: Typed { type: TypeDesc { name: 'Map32', typecode: 209, width: 4, category: 3, create: { [Function] typecode: 209 } }, value: [ Typed { type: [TypeDesc], value: 'operation' }, Typed { type: [TypeDesc], value: 'READ' }, Typed { type: [TypeDesc], value: 'name' }, Typed { type: [TypeDesc], value: 'myeventhub' }, Typed { type: [TypeDesc], value: 'type' }, Typed { type: [TypeDesc], value: 'com.microsoft:eventhub' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 116 } } +7ms
  rhea:message Encoding section 4 of 4: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 5b 5d>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 119 } } +11ms
  rhea:message encoded 179 bytes +4ms
  rhea:frames [connection-1]:1 -> transfer#14 {"delivery_tag":{"type":"Buffer","data":[48]}} <Buffer 00 53 70 45 00 53 73 d0 00 00 00 53 00 00 00 05 a1 24 31 65 36 39 61 30 34 37 2d 36 37 30 35 2d 34 31 35 63 2d 62 37 34 39 2d 31 61 35 63 38 31 32 33 ... > +46ms
  rhea:raw [connection-1] SENT: 207 000000cf02000001005314d00000000c000000064343a0013043424200537045005373d00000005300000005a12431653639613034372d363730352d343135632d623734392d316135633831323362386536404040a12433643036663537642d663364622d346238322d383962612d313164323033643933616237005374d10000004500000006a1096f7065726174696f6ea10452454144a1046e616d65a10a6d796576656e74687562a10474797065a116636f6d2e6d6963726f736f66743a6576656e74687562005377a0025b5d +42ms
  rhea:raw [connection-1] RECV: 23 0000001702000001005315c00a06414340410053244540 +160ms
  rhea:frames [connection-1]:1 <- disposition#15 {"role":true,"settled":true,"state":[]}  +165ms
  rhea:events [connection-1] Received disposition for outgoing transfers +216ms
  rhea:events [connection-1] Link got event: accepted +1ms
  rhea:events [connection-1] Link got event: settled +2ms
  rhea:raw [connection-1] RECV: 251 000000fb02000001005314c0120b520143a004010000004340424040404042005373c0330d4040404040a12431653639613034372d363730352d343135632d623734392d31613563383132336238653640404040404040005374c12404a1096f7065726174696f6ea10452454144a10b7374617475732d636f646571000000c8005377c1760aa1046e616d65a10a6d796576656e74687562a10474797065a116636f6d2e6d6963726f736f66743a6576656e74687562a10a637265617465645f61748300000168cf6a7f67a10f706172746974696f6e5f636f756e745402a10d706172746974696f6e5f696473e00c02b100000001300000000131 +18ms
  rhea:frames [connection-1]:1 <- transfer#14 {"handle":1,"delivery_tag":{"type":"Buffer","data":[1,0,0,0]}} <Buffer 00 53 73 c0 33 0d 40 40 40 40 40 a1 24 31 65 36 39 61 30 34 37 2d 36 37 30 35 2d 34 31 35 63 2d 62 37 34 39 2d 31 61 35 63 38 31 32 33 62 38 65 36 40 ... > +19ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List8', typecode: 192, width: 1, category: 3, create: { [Function] typecode: 192 } }, value: [ Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: '1e69a047-6705-415c-b749-1a5c8123b8e6' }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +194ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'operation' }, Typed { type: [TypeDesc], value: 'READ' }, Typed { type: [TypeDesc], value: 'status-code' }, Typed { type: [TypeDesc], value: 200 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 116 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 116 } +12ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'name' }, Typed { type: [TypeDesc], value: 'myeventhub' }, Typed { type: [TypeDesc], value: 'type' }, Typed { type: [TypeDesc], value: 'com.microsoft:eventhub' }, Typed { type: [TypeDesc], value: 'created_at' }, Typed { type: [TypeDesc], value: 1549668089703 }, Typed { type: [TypeDesc], value: 'partition_count' }, Typed { type: [TypeDesc], value: 2 }, Typed { type: [TypeDesc], value: 'partition_ids' }, Typed { type: [TypeDesc], value: [Array], array_constructor: [Object] } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 119 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 119 } +6ms
  rhea:events [connection-1] Link got event: message +48ms
  rhea:frames [connection-1]:1 -> disposition#15 {"role":true,"settled":true,"state":[]}  +36ms
  rhea:raw [connection-1] SENT: 28 0000001c02000001005315d00000000c000000054143434100532445 +41ms
  rhea:message Encoding section 1 of 4: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 112 } } +27ms
  rhea:message Encoding section 2 of 4: Typed { type: TypeDesc { name: 'List32', typecode: 208, width: 4, category: 3, create: { [Function] typecode: 208 } }, value: [ Typed { type: [TypeDesc], value: '7a4e60ef-2706-cc46-89b5-e3412958f348' }, <1 empty item>, Typed { type: [TypeDesc], value: '$cbs' }, <1 empty item>, Typed { type: [TypeDesc], value: 'cbs-fcf01ba6-0dcc-5542-8288-d5095f9ee0a4' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } +3ms
  rhea:message Encoding section 3 of 4: Typed { type: TypeDesc { name: 'Map32', typecode: 209, width: 4, category: 3, create: { [Function] typecode: 209 } }, value: [ Typed { type: [TypeDesc], value: 'operation' }, Typed { type: [TypeDesc], value: 'put-token' }, Typed { type: [TypeDesc], value: 'name' }, Typed { type: [TypeDesc], value: 'sb://testeventhubssample.servicebus.windows.net/myeventhub/ConsumerGroups/$default/Partitions/0' }, Typed { type: [TypeDesc], value: 'type' }, Typed { type: [TypeDesc], value: 'servicebus.windows.net:sastoken' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 116 } } +4ms
  rhea:message Encoding section 4 of 4: Typed { type: TypeDesc { name: 'Str8', typecode: 161, width: 1, category: 2, encoding: 'utf8', create: { [Function] typecode: 161 } }, value: 'SharedAccessSignature sr=sb%3A%2F%2Ftesteventhubssample.servicebus.windows.net%2Fmyeventhub%2FConsumerGroups%2F%24default%2FPartitions%2F0&sig=XmuH4LlYwzh4HZL07rTZR7BimW9pOsFZ8zTOAlv5H5s%3D&se=1552434019&skn=RootManageSharedAccessKey', descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 119 } } +15ms
  rhea:message encoded 518 bytes +4ms
  rhea:frames [connection-1]:0 -> transfer#14 {"delivery_id":1,"delivery_tag":{"type":"Buffer","data":[49]}} <Buffer 00 53 70 45 00 53 73 d0 00 00 00 5c 00 00 00 05 a1 24 37 61 34 65 36 30 65 66 2d 32 37 30 36 2d 63 63 34 36 2d 38 39 62 35 2d 65 33 34 31 32 39 35 38 ... > +39ms
  rhea:raw [connection-1] SENT: 547 0000022302000000005314d00000000d00000006435201a0013143424200537045005373d00000005c00000005a12437613465363065662d323730362d636334362d383962352d65333431323935386633343840a1042463627340a1286362732d66636630316261362d306463632d353534322d383238382d643530393566396565306134005374d1000000a800000006a1096f7065726174696f6ea1097075742d746f6b656ea1046e616d65a15f73623a2f2f746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e65742f6d796576656e746875622f436f6e73756d657247726f7570732f2464656661756c742f506172746974696f6e732f30a10474797065a11f736572766963656275732e77696e646f77732e6e65743a736173746f6b656e005377a1e95368617265644163636573735369676e61747572652073723d7362253341253246253246746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e65742532466d796576656e74687562253246436f6e73756d657247726f75707325324625323464656661756c74253246506172746974696f6e7325324630267369673d586d7548344c6c59777a6834485a4c303772545a523742696d5739704f73465a387a544f416c76354835732533442673653d3135353234333430313926736b6e3d526f6f744d616e6167655368617265644163636573734b6579 +39ms
  rhea:raw [connection-1] RECV: 24 0000001802000000005315c00b0641520140410053244540 +153ms
  rhea:frames [connection-1]:0 <- disposition#15 {"role":true,"first":1,"settled":true,"state":[]}  +157ms
  rhea:events [connection-1] Received disposition for outgoing transfers +200ms
  rhea:events [connection-1] Link got event: accepted +2ms
  rhea:events [connection-1] Link got event: settled +6ms
  rhea:raw [connection-1] RECV: 142 0000008e02000000005314c0130b52015201a004020000004340424040404042005373c0330d4040404040a12437613465363065662d323730362d636334362d383962352d65333431323935386633343840404040404040005374c13104a10b7374617475732d636f646571000000caa1127374617475732d6465736372697074696f6ea1084163636570746564 +17ms
  rhea:frames [connection-1]:0 <- transfer#14 {"handle":1,"delivery_id":1,"delivery_tag":{"type":"Buffer","data":[2,0,0,0]}} <Buffer 00 53 73 c0 33 0d 40 40 40 40 40 a1 24 37 61 34 65 36 30 65 66 2d 32 37 30 36 2d 63 63 34 36 2d 38 39 62 35 2d 65 33 34 31 32 39 35 38 66 33 34 38 40 ... > +17ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List8', typecode: 192, width: 1, category: 3, create: { [Function] typecode: 192 } }, value: [ Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: '7a4e60ef-2706-cc46-89b5-e3412958f348' }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null }, Typed { type: [TypeDesc], value: null } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +184ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'status-code' }, Typed { type: [TypeDesc], value: 202 }, Typed { type: [TypeDesc], value: 'status-description' }, Typed { type: [TypeDesc], value: 'Accepted' } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 116 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 116 } +6ms
  rhea:events [connection-1] Link got event: message +32ms
  rhea:frames [connection-1]:0 -> disposition#15 {"role":true,"first":1,"last":1,"settled":true,"state":[]}  +28ms
  rhea:raw [connection-1] SENT: 30 0000001e02000000005315d00000000e0000000541520152014100532445 +32ms
  rhea:frames [connection-1]:2 -> begin#11 {"incoming_window":2048,"outgoing_window":4294967295}  +12ms
  rhea:raw [connection-1] SENT: 32 0000002002000002005311d000000010000000044043700000080070ffffffff +12ms
  rhea:raw [connection-1] RECV: 34 0000002202000002005311c0150860000252017000001388700000080052ff404040 +189ms
  rhea:frames [connection-1]:2 <- begin#11 {"remote_channel":2,"next_outgoing_id":1,"incoming_window":5000,"outgoing_window":2048,"handle_max":255}  +193ms
  rhea:events [connection-1] Session got event: session_open +210ms
  rhea:frames [connection-1]:2 -> attach#12 {"name":"fc087a17-7790-45e3-b413-a485f4158df0","role":true,"source":["myeventhub/ConsumerGroups/$default/Partitions/0",null,null,null,null,null,null,["apache.org:selector-filter:string","amqp.annotation.x-opt-offset > '-1'"]],"target":[],"desired_capabilities":"com.microsoft:enable-receiver-runtime-metric"}  +5ms
  rhea:raw [connection-1] SENT: 275 0000011302000002005312d0000001030000000da12466633038376131372d373739302d343565332d623431332d61343835663431353864663043414040005328d00000009600000008a12f6d796576656e746875622f436f6e73756d657247726f7570732f2464656661756c742f506172746974696f6e732f30404040404040d10000005600000002a3216170616368652e6f72673a73656c6563746f722d66696c7465723a737472696e6700800000468c00000004a123616d71702e616e6e6f746174696f6e2e782d6f70742d6f6666736574203e20272d3127005329454040404040a32c636f6d2e6d6963726f736f66743a656e61626c652d72656365697665722d72756e74696d652d6d6574726963 +9ms
  rhea:raw [connection-1] RECV: 285 0000011d02000002005312d00000010d0000000ea12466633038376131372d373739302d343565332d623431332d6134383566343135386466304342500140005328c0900ba12f6d796576656e746875622f436f6e73756d657247726f7570732f2464656661756c742f506172746974696f6e732f30404040404040c15302a3216170616368652e6f72673a73656c6563746f722d66696c7465723a737472696e6700800000468c00000004a123616d71702e616e6e6f746174696f6e2e782d6f70742d6f6666736574203e20272d3127404040005329c008074040404040404040404380ffffffffffffffff40a32c636f6d2e6d6963726f736f66743a656e61626c652d72656365697665722d72756e74696d652d6d657472696340 +81ms
  rhea:frames [connection-1]:2 <- attach#12 {"name":"fc087a17-7790-45e3-b413-a485f4158df0","snd_settle_mode":1,"source":["myeventhub/ConsumerGroups/$default/Partitions/0",null,null,null,null,null,null,["apache.org:selector-filter:string","amqp.annotation.x-opt-offset > '-1'"],null,null,null],"target":[null,null,null,null,null,null,null],"max_message_size":{"type":"Buffer","data":[255,255,255,255,255,255,255,255]},"desired_capabilities":"com.microsoft:enable-receiver-runtime-metric"}  +86ms
  rhea:events [connection-1] Link got event: receiver_open +92ms
  rhea:frames [connection-1]:2 -> flow#13 {"incoming_window":2048,"outgoing_window":4294967295,"link_credit":1000}  +5ms
  rhea:raw [connection-1] SENT: 39 0000002702000002005313d000000017000000074370000008004370ffffffff434370000003e8 +10ms
  rhea:raw [connection-1] RECV: 291 0000012302000002005314c00d0b4343a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f65005372c15606a315782d6f70742d73657175656e63652d6e756d6265728100000000000030d1a30c782d6f70742d6f6666736574a10731323038303234a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00d2248656c6c6f20666f6f203022 +81ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +85ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419813 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +403ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12497 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1208024' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 30 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +4ms
  rhea:events [connection-1] Link got event: message +111ms
### Actual message: Hello foo 0
  rhea:raw [connection-1] RECV: 292 0000012402000002005314c00e0b435201a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f65005372c15606a315782d6f70742d73657175656e63652d6e756d6265728100000000000030d2a30c782d6f70742d6f6666736574a10731323038313230a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00d2248656c6c6f20666f6f203122 +49ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_id":1,"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +49ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419813 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +33ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12498 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1208120' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +15ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 31 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +4ms
  rhea:events [connection-1] Link got event: message +58ms
### Actual message: Hello foo 1
  rhea:raw [connection-1] RECV: 292 0000012402000002005314c00e0b435202a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f65005372c15606a315782d6f70742d73657175656e63652d6e756d6265728100000000000030d3a30c782d6f70742d6f6666736574a10731323038323136a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00d2248656c6c6f20666f6f203222 +42ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_id":2,"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +43ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419813 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +17ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12499 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1208216' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +22ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 32 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +4ms
  rhea:events [connection-1] Link got event: message +57ms
### Actual message: Hello foo 2
  rhea:raw [connection-1] RECV: 292 0000012402000002005314c00e0b435203a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f65005372c15606a315782d6f70742d73657175656e63652d6e756d6265728100000000000030d4a30c782d6f70742d6f6666736574a10731323038333132a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00d2248656c6c6f20666f6f203322 +63ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_id":3,"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +63ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419813 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +31ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12500 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1208312' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +14ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 33 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +4ms
  rhea:events [connection-1] Link got event: message +47ms
### Actual message: Hello foo 3
  rhea:raw [connection-1] RECV: 292 0000012402000002005314c00e0b435204a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f65005372c15606a315782d6f70742d73657175656e63652d6e756d6265728100000000000030d5a30c782d6f70742d6f6666736574a10731323038343038a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00d2248656c6c6f20666f6f203422 +41ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_id":4,"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +40ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419813 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +28ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12501 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1208408' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 34 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +4ms
  rhea:events [connection-1] Link got event: message +44ms
### Actual message: Hello foo 4
  rhea:raw [connection-1] RECV: 292 0000012402000002005314c00e0b435205a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f65005372c15606a315782d6f70742d73657175656e63652d6e756d6265728100000000000030d6a30c782d6f70742d6f6666736574a10731323038353034a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00d2248656c6c6f20666f6f203522 +53ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_id":5,"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +54ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419813 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +27ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12502 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1208504' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +16ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 35 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +5ms
  rhea:events [connection-1] Link got event: message +53ms
### Actual message: Hello foo 5
  rhea:raw [connection-1] RECV: 292 0000012402000002005314c00e0b435206a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f65005372c15606a315782d6f70742d73657175656e63652d6e756d6265728100000000000030d7a30c782d6f70742d6f6666736574a10731323038363030a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00d2248656c6c6f20666f6f203622 +42ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_id":6,"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +42ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419813 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +15ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12503 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1208600' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +17ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +6ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 36 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +4ms
  rhea:events [connection-1] Link got event: message +43ms
.
.
.
.
.
.
.
.

### Actual message: Hello foo 150
  rhea:raw [connection-1] RECV: 294 0000012602000002005314c00e0b435297a0004341424040404041005371c19308a31d6c6173745f656e7175657565645f73657175656e63655f6e756d6265728100000000000034b9a3146c6173745f656e7175657565645f6f6666736574a10731333034303234a3166c6173745f656e7175657565645f74696d655f75746383000001697399e3bfa31f72756e74696d655f696e666f5f72657472696576616c5f74696d655f757463830000016974104f74005372c15606a315782d6f70742d73657175656e63652d6e756d626572810000000000003168a30c782d6f70742d6f6666736574a10731323232353230a313782d6f70742d656e7175657565642d74696d6583000001697399e3bf00537345005375a00f2248656c6c6f20666f6f2031353122 +35ms
  rhea:frames [connection-1]:2 <- transfer#14 {"delivery_id":151,"delivery_tag":{"type":"Buffer","data":[]},"settled":true,"batchable":true} <Buffer 00 53 71 c1 93 08 a3 1d 6c 61 73 74 5f 65 6e 71 75 65 75 65 64 5f 73 65 71 75 65 6e 63 65 5f 6e 75 6d 62 65 72 81 00 00 00 00 00 00 34 b9 a3 14 6c 61 ... > +44ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'last_enqueued_sequence_number' }, Typed { type: [TypeDesc], value: 13497 }, Typed { type: [TypeDesc], value: 'last_enqueued_offset' }, Typed { type: [TypeDesc], value: '1304024' }, Typed { type: [TypeDesc], value: 'last_enqueued_time_utc' }, Typed { type: [TypeDesc], value: 1552422659007 }, Typed { type: [TypeDesc], value: 'runtime_info_retrieval_time_utc' }, Typed { type: [TypeDesc], value: 1552430419828 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 113 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 113 } +26ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [TypeDesc], value: 'x-opt-sequence-number' }, Typed { type: [TypeDesc], value: 12648 }, Typed { type: [TypeDesc], value: 'x-opt-offset' }, Typed { type: [TypeDesc], value: '1222520' }, Typed { type: [TypeDesc], value: 'x-opt-enqueued-time' }, Typed { type: [TypeDesc], value: 1552422659007 } ], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 114 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 114 } +5ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List0', typecode: 69, width: 0, category: 1, create: { [Function] typecode: 69 } }, value: [], descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 115 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +5ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Vbin8', typecode: 160, width: 1, category: 2, create: { [Function] typecode: 160 } }, value: <Buffer 22 48 65 6c 6c 6f 20 66 6f 6f 20 31 35 31 22>, descriptor: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Function] }, value: 117 } } of type: Typed { type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 117 } +4ms
  rhea:events [connection-1] Link got event: message +39ms
### Actual message: Hello foo 151
  rhea:events [connection-1] Connection got event: disconnected +5ms
  rhea:events [connection-1] Connection got event: disconnected +316ms
  rhea:frames [connection-1] -> { protocol_id: 3, major: 1, minor: 0, revision: 0 } +16s
  rhea:frames [connection-1] <- { protocol_id: 3, major: 1, minor: 0, revision: 0 } +78ms
  rhea:raw [connection-1] RECV: 63 0000003f02010000005340c03201e02f04b3000000074d53534243425300000005504c41494e00000009414e4f4e594d4f55530000000845585445524e414c +16s
  rhea:frames [connection-1]:0 <- sasl_mechanisms#40 {"sasl_server_mechanisms":["MSSBCBS","PLAIN","ANONYMOUS","EXTERNAL"]}  +8ms
  rhea:frames [connection-1]:0 -> sasl_init#41 {"mechanism":"ANONYMOUS","initial_response":{"type":"Buffer","data":[0,82,111,111,116,77,97,110,97,103,101,83,104,97,114,101,100,65,99,99,101,115,115,75,101,121]},"hostname":"testeventhubssample.servicebus.windows.net"}  +3ms
  rhea:raw [connection-1] SENT: 103 0000006702010000005341d00000005700000003a309414e4f4e594d4f5553a01a00526f6f744d616e6167655368617265644163636573734b6579a12a746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e6574 +13ms
  rhea:raw [connection-1] RECV: 26 0000001a02010000005344c00d025000a00857656c636f6d6521 +71ms
  rhea:frames [connection-1]:0 <- sasl_outcome#44 {"additional_data":{"type":"Buffer","data":[87,101,108,99,111,109,101,33]}}  +81ms
  rhea:frames [connection-1] -> { protocol_id: 0, major: 1, minor: 0, revision: 0 } +2ms
  rhea:frames [connection-1]:0 -> open#10 {"transport":"tls","host":"testeventhubssample.servicebus.windows.net","username":"RootManageSharedAccessKey","port":5671,"operationTimeoutInSeconds":60,"servername":"testeventhubssample.servicebus.windows.net","id":"connection-1","container_id":"af44bc38-0ebc-5549-83fc-1a993e584b54","hostname":"testeventhubssample.servicebus.windows.net","properties":{"product":"MSJSClient","version":"1.0.5","user-agent":"/js-event-hubs","platform":"(x64-Windows_NT-10.0.17763)","framework":"Node/v10.14.1"}}  +1ms
  rhea:raw [connection-1] SENT: 248 000000f802000000005310d0000000e80000000aa12461663434626333382d306562632d353534392d383366632d316139393365353834623534a12a746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e657440404040404040d1000000860000000aa30770726f64756374a10a4d534a53436c69656e74a30776657273696f6ea105312e302e35a30a757365722d6167656e74a10e2f6a732d6576656e742d68756273a308706c6174666f726da11b287836342d57696e646f77735f4e542d31302e302e313737363329a3096672616d65776f726ba10d4e6f64652f7631302e31342e31 +8ms
  rhea:frames [connection-1]:1 -> begin#11 {"incoming_window":2048,"outgoing_window":4294967295}  +6ms
  rhea:raw [connection-1] SENT: 32 0000002002000001005311d000000010000000044043700000080070ffffffff +4ms
  rhea:frames [connection-1]:1 -> attach#12 {"name":"5b1323d0-43de-794f-809e-567279b770a8","source":[],"target":["$management"]}  +3ms
  rhea:raw [connection-1] SENT: 94 0000005e02000001005312d00000004e0000000aa12435623133323364302d343364652d373934662d383039652d3536373237396237373061384342404000532845005329d00000001100000001a10b246d616e6167656d656e74404043 +4ms
  rhea:frames [connection-1]:1 -> attach#12 {"name":"3d06f57d-f3db-4b82-89ba-11d203d93ab7","handle":1,"role":true,"source":["$management"],"target":["3d06f57d-f3db-4b82-89ba-11d203d93ab7"]}  +4ms
  rhea:raw [connection-1] SENT: 138 0000008a02000001005312d00000007a00000007a12433643036663537642d663364622d346238322d383962612d3131643230336439336162375201414040005328d00000001100000001a10b246d616e6167656d656e74005329d00000002a00000001a12433643036663537642d663364622d346238322d383962612d313164323033643933616237 +4ms
  rhea:frames [connection-1]:2 -> begin#11 {"incoming_window":2048,"outgoing_window":4294967295}  +13ms
  rhea:raw [connection-1] SENT: 32 0000002002000002005311d000000010000000044043700000080070ffffffff +12ms
  rhea:frames [connection-1]:2 -> attach#12 {"name":"fc087a17-7790-45e3-b413-a485f4158df0","role":true,"source":["myeventhub/ConsumerGroups/$default/Partitions/0",null,null,null,null,null,null,["apache.org:selector-filter:string","amqp.annotation.x-opt-offset > '-1'"]],"target":[],"desired_capabilities":"com.microsoft:enable-receiver-runtime-metric"}  +5ms
  rhea:raw [connection-1] SENT: 275 0000011302000002005312d0000001030000000da12466633038376131372d373739302d343565332d623431332d61343835663431353864663043414040005328d00000009600000008a12f6d796576656e746875622f436f6e73756d657247726f7570732f2464656661756c742f506172746974696f6e732f30404040404040d10000005600000002a3216170616368652e6f72673a73656c6563746f722d66696c7465723a737472696e6700800000468c00000004a123616d71702e616e6e6f746174696f6e2e782d6f70742d6f6666736574203e20272d3127005329454040404040a32c636f6d2e6d6963726f736f66743a656e61626c652d72656365697665722d72756e74696d652d6d6574726963 +6ms
  rhea:frames [connection-1] <- { protocol_id: 0, major: 1, minor: 0, revision: 0 } +44ms
  rhea:raw [connection-1] RECV: 71 0000004702000000005310c03a0aa12439626230333962383731663234366635613663383866323865306665313038665f473230407000010000601387700003a9804040404040 +118ms
  rhea:frames [connection-1]:0 <- open#10 {"container_id":"9bb039b871f246f5a6c88f28e0fe108f_G20","max_frame_size":65536,"channel_max":4999,"idle_time_out":240000}  +78ms
  rhea:events [connection-1] Connection got event: connection_open +16s
  rhea:frames [connection-1]:0 -> begin#11 {"incoming_window":2048,"outgoing_window":4294967295}  +5ms
  rhea:raw [connection-1] SENT: 32 0000002002000000005311d000000010000000044043700000080070ffffffff +9ms
  rhea:raw [connection-1] RECV: 34 0000002202000000005311c0150860000152017000001388700000080052ff404040 +11ms
  rhea:frames [connection-1]:0 <- begin#11 {"remote_channel":1,"next_outgoing_id":1,"incoming_window":5000,"outgoing_window":2048,"handle_max":255}  +14ms
  rhea:events [connection-1] Session got event: session_open +20ms
  rhea:raw [connection-1] RECV: 113 0000007102000000005312c0640ea12435623133323364302d343364652d373934662d383039652d35363732373962373730613843414040005328c00c0b4040404040404040404040005329c01407a10b246d616e6167656d656e7440404040404040404080ffffffffffffffff404040 +9ms
  rhea:frames [connection-1]:0 <- attach#12 {"name":"5b1323d0-43de-794f-809e-567279b770a8","role":true,"source":[null,null,null,null,null,null,null,null,null,null,null],"target":["$management",null,null,null,null,null,null],"max_message_size":{"type":"Buffer","data":[255,255,255,255,255,255,255,255]}}  +16ms
  rhea:events [connection-1] Link got event: sender_open +16ms
  rhea:raw [connection-1] RECV: 35 0000002302000000005313c0160b437000001388520170000008004343526443404240 +21ms
  rhea:frames [connection-1]:0 <- flow#13 {"incoming_window":5000,"next_outgoing_id":1,"outgoing_window":2048,"link_credit":100}  +14ms
  rhea:events [connection-1] Link got event: sender_flow +14ms
  rhea:events [connection-1] Link got event: sendable +1ms
  rhea:raw [connection-1] RECV: 151 0000009702000000005312c08a0ea12433643036663537642d663364622d346238322d383962612d3131643230336439336162375201424040005328c0180ba10b246d616e6167656d656e7440404040404040404040005329c02d07a12433643036663537642d663364622d346238322d383962612d31316432303364393361623740404040404040404380ffffffffffffffff404040 +11ms
  rhea:frames [connection-1]:0 <- attach#12 {"name":"3d06f57d-f3db-4b82-89ba-11d203d93ab7","handle":1,"source":["$management",null,null,null,null,null,null,null,null,null,null],"target":["3d06f57d-f3db-4b82-89ba-11d203d93ab7",null,null,null,null,null,null],"max_message_size":{"type":"Buffer","data":[255,255,255,255,255,255,255,255]}}  +12ms
  rhea:events [connection-1] Link got event: receiver_open +10ms
  rhea:frames [connection-1]:1 -> flow#13 {"incoming_window":2048,"outgoing_window":4294967295,"handle":1,"link_credit":1000}  +12ms
  rhea:raw [connection-1] SENT: 40 0000002802000001005313d000000018000000074370000008004370ffffffff52014370000003e8 +16ms
  rhea:raw [connection-1] RECV: 34 0000002202000001005311c0150860000252017000001388700000080052ff404040 +6ms
  rhea:frames [connection-1]:1 <- begin#11 {"remote_channel":2,"next_outgoing_id":1,"incoming_window":5000,"outgoing_window":2048,"handle_max":255}  +9ms
  rhea:events [connection-1] Session got event: session_open +22ms
  rhea:raw [connection-1] RECV: 34 0000002202000002005311c0150860000052017000001388700000080052ff404040 +17ms
  rhea:frames [connection-1]:2 <- begin#11 {"next_outgoing_id":1,"incoming_window":5000,"outgoing_window":2048,"handle_max":255}  +16ms
  rhea:events [connection-1] Session got event: session_open +16ms
  rhea:frames [connection-1]:0 -> attach#12 {"name":"cd35380b-17dc-9d4e-b327-93aeeb2ccd5d","source":[],"target":["$cbs"]}  +6ms
  rhea:raw [connection-1] SENT: 87 0000005702000000005312d0000000470000000aa12463643335333830622d313764632d396434652d623332372d3933616565623263636435644342404000532845005329d00000000a00000001a10424636273404043 +12ms
  rhea:raw [connection-1] RECV: 65 0000004102000001005312c0340ea12466633038376131372d373739302d343565332d623431332d61343835663431353864663043424040404040404040404040 +102ms
  rhea:frames [connection-1]:1 <- attach#12 {"name":"fc087a17-7790-45e3-b413-a485f4158df0","source":null,"target":null}  +109ms
  rhea:events [connection-1] Link got event: receiver_open +115ms
  rhea:frames [connection-1]:2 -> flow#13 {"incoming_window":2048,"outgoing_window":4294967295,"link_credit":1000}  +5ms
  rhea:raw [connection-1] SENT: 39 0000002702000002005313d000000017000000074370000008004370ffffffff434370000003e8 +8ms
  rhea:raw [connection-1] RECV: 356 0000016402000001005316d00000015400000003434100531dd00000014600000003a318616d71703a756e617574686f72697a65642d616363657373b100000122556e617574686f72697a6564206163636573732e20274c697374656e2720636c61696d2873292061726520726571756972656420746f20706572666f726d2074686973206f7065726174696f6e2e205265736f757263653a202773623a2f2f746573746576656e746875627373616d706c652e736572766963656275732e77696e646f77732e6e65742f6d796576656e746875622f636f6e73756d657267726f7570732f2464656661756c742f706172746974696f6e732f30272e20547261636b696e6749643a39626230333962383731663234366635613663383866323865306665313038665f4732302c2053797374656d547261636b65723a67617465776179352c2054696d657374616d703a323031392d30332d31325432323a34303a343140 +5ms
  rhea:frames [connection-1]:1 <- detach#16 {"closed":true,"error":{"condition":"amqp:unauthorized-access","description":"Unauthorized access. 'Listen' claim(s) are required to perform this operation. Resource: 'sb://testeventhubssample.servicebus.windows.net/myeventhub/consumergroups/$default/partitions/0'. TrackingId:9bb039b871f246f5a6c88f28e0fe108f_G20, SystemTracker:gateway5, Timestamp:2019-03-12T22:40:41"}}  +17ms
  rhea:events [connection-1] Link got event: receiver_error +23ms
>>>>> Error occurred:  { UnauthorizedError: Unauthorized access. ....

@grs
Copy link
Member

grs commented Mar 13, 2019

Yes, as suspected, the client is initiating the attach. That is expected for reconnect, either automatic where the reconnect option is on, or where a manual call to reconnect() on the connection is made. If the reconnect option is not on then even the connection would not be re-established.

@ramya-rao-a
Copy link
Contributor Author

@grs We found that we do call connection.open() in rhea-promise that calls connection.connect() in rhea

And

rhea/lib/connection.js

Lines 271 to 273 in 96a2324

for (var k in this.local_channel_map) {
this.local_channel_map[k]._reconnect();
}
is where rhea reconnects all the sessions and links in the connection even if we explicitly set reconnect option to false

Shouldnt rhea clear up local_channel_map when the connection disconnects if client has set reconnect option to false?

cc @ShivangiReja who investigated and made the above observations

@grs
Copy link
Member

grs commented Mar 13, 2019

The reconnect option is there to control whether the library automatically reconnects on disconnection. I don't think it setting it to false should cause the local_channel_map to be deleted. For one thing some users might need to know about the state of sessions and links at the time of disconnection.

If you don't want to use any of the connection state when re-establishing a connection, you can always use a new connection. If you just wish to remove certain sessions or links, you can do that.

@ramya-rao-a
Copy link
Contributor Author

ramya-rao-a commented Mar 14, 2019

That clarifies a lot of things, thanks @grs.

One last question. Say we do the following (completely hypothetical case)

  • Call close() on a receiver and its session and then the connection
  • Now call connection.connect()

rhea will now attempt to reconnect all the sessions and its links? Because I see that session.close() or link.close() dont clear the internal map that the connection or the session maintain respectively.

@grs
Copy link
Member

grs commented Mar 14, 2019

It won't reconnect closed sessions and links.

@ramya-rao-a
Copy link
Contributor Author

ramya-rao-a commented Mar 17, 2019

Thanks @grs

If you don't want to use any of the connection state when re-establishing a connection, you can always use a new connection. If you just wish to remove certain sessions or links, you can do that.

We want to remove all the sessions and links on the connection. Would you be open for a PR (#207) from us that introduces a new function remove_all_sessions on the connection which would clear the local map?

@grs
Copy link
Member

grs commented Mar 18, 2019

Sure, though at that point you could also just create a new connection(?).

@ramya-rao-a
Copy link
Contributor Author

Yes, that is a good point and we will explore that approach, but for now this remove_all_sessions unblocks us.

Thanks for the quick response on the PR @grs!

Could you please drop a note here when a new version is published?

@grs
Copy link
Member

grs commented Mar 18, 2019

1.0.2 published

@ramya-rao-a
Copy link
Contributor Author

Thanks a ton!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants