Skip to content
This repository has been archived by the owner on Nov 20, 2020. It is now read-only.

Upon killing Group Coordinator, an entire group would fatally crash 🐛 #221

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Upon killing Group Coordinator, an entire group would fatally crash 🐛 #221

wants to merge 1 commit into from

Conversation

wiredsister
Copy link
Contributor

Using Kafka 0.11.0.1:

Given I am producing, consuming, and reporting metadata on a topic managed by 3 broker nodes
And the Group Coordinator broker is killed (cleanly)
Then the producer, consumer, and reporter will all fatally crash and not recover.

The output of this metadata error (from the producer, for example):

Relevant exn:

Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
2018-04-18 22:19:57:6517|WARN|Kafunk.Resource|failed_to_open_resource|type=Socket[127.0.0.1:9092] version=2 error="System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: Connection refused
   --- End of inner exception stack trace ---
  at Kafunk.Prelude+ResultModule.throwMap[e,a,a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x0001c] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Faults+retryResultThrow@432[e,a,a].Invoke (Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.AsyncEx+AsyncModule+map@147-2[a,b].Invoke (a x) [0x00006] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@506-1[a,b].Invoke (a a) [0x00015] in <5a953878dff9fae1a74503837838955a>:0 
---> (Inner Exception #0) System.Net.Sockets.SocketException (0x80004005): Connection refused<---

---> (Inner Exception #1) System.Net.Sockets.SocketException (0x80004005): Connection refused<---
"
2018-04-18 22:19:59:6557|WARN|Kafunk.Conn|handling_broker_chan_error|node_id=-2 endpoint=localhost:9092 req=Metadata Kafunk.Protocol+MetadataRequest critical=true errors=[[chan_exn|error="System.AggregateException: One or more errors occurred. ---> Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: Connection refused
   --- End of inner exception stack trace ---
  at Kafunk.Prelude+ResultModule.throwMap[e,a,a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x0001c] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Faults+retryResultThrow@432[e,a,a].Invoke (Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.AsyncEx+AsyncModule+map@147-2[a,b].Invoke (a x) [0x00006] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@506-1[a,b].Invoke (a a) [0x00015] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
  at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
  at Kafunk.Resource+Resource+go@220-25[b,r].Invoke (System.Exception _arg8) [0x0003a] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
---> (Inner Exception #0) Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: Connection refused
   --- End of inner exception stack trace ---
  at Kafunk.Prelude+ResultModule.throwMap[e,a,a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x0001c] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Faults+retryResultThrow@432[e,a,a].Invoke (Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.AsyncEx+AsyncModule+map@147-2[a,b].Invoke (a x) [0x00006] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@506-1[a,b].Invoke (a a) [0x00015] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
  at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---
"]] conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6607|INFO|Kafunk.Conn|connecting_to_bootstrap_brokers|brokers=[kafka://localhost:9092/; kafka://localhost:9093/; kafka://localhost:9094/] attempt=1 conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6620|INFO|Kafunk.Conn|discovered_dns|host=localhost ips=[127.0.0.1] conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6638|INFO|Kafunk.Conn|discovered_dns|host=localhost ips=[127.0.0.1] conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6639|INFO|Kafunk.Chan|tcp_connecting|remote_endpoint=127.0.0.1:9093 conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6643|INFO|Kafunk.Chan|tcp_connected|remote_endpoint=127.0.0.1:9093 local_endpoint=127.0.0.1:51619 conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6734|WARN|Kafunk.Resource|failed_to_open_resource|type=ProducerState[absurd-gina1-topic] version=2 error="System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
  at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0 
   --- End of inner exception stack trace ---
  at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
---> (Inner Exception #0) Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
  at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0 
   --- End of inner exception stack trace ---
  at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---
"
2018-04-18 22:19:59:6748|ERROR|Producer.fsx|produce_error|Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
  at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0 
   --- End of inner exception stack trace ---
  at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
  at Kafunk.Resource+Resource+go@220-25[b,r].Invoke (System.Exception _arg8) [0x0003a] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
2018-04-18 22:19:59:6771|ERROR|Producer.fsx|produce_error|System.AggregateException: One or more errors occurred. ---> Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
  at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0 
   --- End of inner exception stack trace ---
  at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
  at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
  at Kafunk.Resource+Resource+go@220-25[b,r].Invoke (System.Exception _arg8) [0x0003a] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
---> (Inner Exception #0) Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
  at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0 
   --- End of inner exception stack trace ---
  at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
  at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---

2018-04-18 22:19:59:6772|ERROR|Producer.fsx|System.AggregateException: One or more errors occurred. ---> Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
  at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0 
   --- End of inner exception stack trace ---
  at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
  at FSI_0002+go@175-23.Invoke (System.Exception _arg9) [0x00030] in <40a4ede61ca94ec1939935132ae0470a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <e36d106f4822473d8910305e5d059e11>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl.commit[a] (Microsoft.FSharp.Control.AsyncBuilderImpl+AsyncImplResult`1[T] res) [0x0002c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.CancellationTokenOps.RunSynchronouslyInCurrentThread[a] (System.Threading.CancellationToken token, Microsoft.FSharp.Control.FSharpAsync`1[T] computation) [0x00028] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.CancellationTokenOps.RunSynchronously[a] (System.Threading.CancellationToken token, Microsoft.FSharp.Control.FSharpAsync`1[T] computation, Microsoft.FSharp.Core.FSharpOption`1[T] timeout) [0x00013] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T] (Microsoft.FSharp.Control.FSharpAsync`1[T] computation, Microsoft.FSharp.Core.FSharpOption`1[T] timeout, Microsoft.FSharp.Core.FSharpOption`1[T] cancellationToken) [0x00070] in <5a953878dff9fae1a74503837838955a>:0 
  at <StartupCode$FSI_0002>.$FSI_0002.main@ () [0x00392] in <40a4ede61ca94ec1939935132ae0470a>:0 
---> (Inner Exception #0) Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
  at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0 
  at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0 
   --- End of inner exception stack trace ---
  at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
  at FSI_0002+go@175-23.Invoke (System.Exception _arg9) [0x00030] in <40a4ede61ca94ec1939935132ae0470a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---

2018-04-18 22:19:59:6801|INFO|Producer.fsx|producer_run_completed|messages=21200 missing=978800 batch_size=100 message_size=10 parallelism=1 elapsed_sec=124.522873 rate_per_sec=170.249847 MB=10
2018-04-18 22:20:00:1615|INFO|Producer.fsx|last_sec=0          | avg_sec=170        | tp50_sec=200        | tp90_sec=0          | tp99_sec=0          | max_sec=200        | count=21200     
2018-04-18 22:20:00:1904|INFO|Producer.fsx|completed=21200 elapsed_sec=124.522873 MB=0 offsets=[]

Upon killing Group Coordinator, an entire group would fatally crash g…
1a9f7e8 
…iven a late enough version of the metadata.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@eulerfx eulerfx Awaiting requested review from eulerfx

@michaelliao5 michaelliao5 Awaiting requested review from michaelliao5

Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@wiredsister