panic: sniff timeout: no Elasticsearch node available

[ickma]

Please use the following questions as a guideline to help me answer
your issue/question without further inquiry. Thank you.

Which version of Elastic are you using?

[ ] elastic.v2 (for Elasticsearch 1.x)
[ ] elastic.v3 (for Elasticsearch 2.x)
[x] elastic.v5 (for Elasticsearch 5.x)
[ ] elastic.v6 (for Elasticsearch 6.x)

Please describe the expected behavior

I want to make an elastic worker queue: suppose I have 1000 document to write to elastic-search at the same time, it can create 1000 client (each client works in one go-routine)to work concurrently.

Please describe the actual behavior

Now, when testing, if I create less than 100 clients at one time, everything works fine; but, if I increase the number to more than 150, the process panics with panic: sniff timeout: no Elasticsearch node available in most case.

Any steps to reproduce the behavior?

I created clients in createClient function just as the official document said:

		client, err := esClient.NewClient(
			esClient.SetURL(utils.ConstantsESPath), // path=http://localhost:9200
			esClient.SetBasicAuth(utils.ConstantsEsUser, utils.ConstantsEsPassword), //User="" Password=""
			esClient.SetHealthcheckInterval(10*time.Second),
				)

I tested this way:

for i:=0;i<1000;i++{
...
   client,err:=createClient()
   if err!=nil{
       panic(err)
}
....
}

I guess this related to process IO limit of system, but i am not very sure.

PS: I tried setting sniff to false , but it didn't work!

[olivere]

You should only create one client for your application. A Client is thread-safe and can be used by multiple goroutines concurrently. Every Client starts goroutines to watch the cluster and monitor the connections, so it's rather expensive to create. You need to stop a Client if it's no longer used.

If you want a simple throwaway client, use NewSimpleClient.

[ickma]

Thanks!