Force close

[matzew]

I think I am hitting:

• https://www.reddit.com/r/golang/comments/9b5r6h/concurrent_http_requests_not_releasing_fast/
• http://craigwickesser.com/2015/01/golang-http-to-many-open-files/

when using the the client, in a (kafka) consumer, like:

...
			go func(pc cluster.PartitionConsumer) {
				for msg := range pc.Messages() {
					log.Printf("Received %s", msg.Value)
					go post(msg.Value, ce)
					consumer.MarkOffset(msg, "") // mark message as processed
				}
			}(part)
...

where post() is like:

func post(data interface{}, ce client.Client) {
	event := cloudevents.Event{
		Context: cloudevents.EventContextV02{
			Type:   "kafka-event",
			Source: *types.ParseURLRef(topic),
		}.AsV02(),
		Data: data,
	}
	if err := ce.Send(context.TODO(), event); err != nil {
		log.Printf("sending event to channel failed: %v", err)
	}
}

I see, which really high load (on a single partition), lot's of

2019/03/13 19:27:59 sending event to channel failed: Post http://localhost:7080: dial tcp [::1]:7080: socket: too many open files

with locally hacking, this is way way less .... 🤔

ping @n3wscott

[n3wscott]

Interesting. I will test it out locally

[n3wscott]

Closing in favor of #77