Speech: Connections stay open in ESTABLISHED or CLOSE_WAIT state

[dmarvp]

OS: CentOS Linux release 7.2.1511 (Core) 
Python 2.7.5
google-cloud-speech Version: 0.27.0

Steps to reproduce:
After using the google speech client do:
lsof -p <pid>
or:
netstat -a
and you'll notice that there are some TCP connections still in ESTABLISHED or CLOSE_WAIT state.

The foreign addresses belong to Google. These connections remain open and, if the client is used a lot, will create problems of the type "IOError: [Errno 24] Too many open files". I'm using this inside a cherrypy server, so it actually makes my server hang after a while.

I noticed that some of the connections close themselves after some minutes, but some of them seem to remain there no matter what. I'm wondering if there is a way to close the connections opened by the speech client after finishing a Recognize call.

Code example

def audio_to_text(self, audio_segment, language_code=None,
                      phrases=None, frame_rate=16000, channels=1):
        if language_code is None:
            language_code = self.DEFAULT_LANGUAGE_CODE

        # Add a second of silence
        silence = pydub.AudioSegment.silent(duration=1000)
        audio_segment = silence + audio_segment

        # Change the frame rate if needed
        if (audio_segment.frame_rate != frame_rate):
            audio_segment = audio_segment.set_frame_rate(frame_rate)

        # Change the number of channels if needed
        if (audio_segment.channels != channels):
            audio_segment = audio_segment.set_channels(channels)

        # Sanitize the phrases array
        if phrases:
            phrases = sanitize_phrases(phrases)

        client = speech.SpeechClient(
            credentials=self.google_cloud_credentials())
        audio = types.RecognitionAudio(content=audio_segment.raw_data)
        config = types.RecognitionConfig(
            encoding=enums.RecognitionConfig.AudioEncoding.LINEAR16,
            sample_rate_hertz=audio_segment.frame_rate,
            language_code=language_code,
            speech_contexts=[speech.types.SpeechContext(phrases=phrases, )],
        )

        try:
            response = client.recognize(config, audio)
        except Exception as e:
            print "failed to do audio_to_text: ", e
            return None

        return self.process_result(response, audio_segment)

[dmarvp]

Btw, this seems somewhat related to #5523 and similar to this 2-year old issue in stack overflow: https://stackoverflow.com/questions/34794516/gcloud-python-close-connection

[theacodes]

Seems this should maybe be moved to gRPC, @tseaver?

[tseaver]

@theacodes You likely know better than I. :)

[theacodes]

This issue was moved to grpc/grpc#15990

[dmarvp]

hello @theacodes, the problem persists and according to @srini100 on grpc/grpc#15990 it's not on the grpc side of things. Anything we can do about it?

[theacodes]

If you're only using one client then I'm not sure. It could be requests?

[jtromans]

I'm also experiencing the same issue, which is similar to and reported here: #5523

I get it on Windows and on Linux using Python 3.6.5. I am using requests module version '2.18.4' and I recently tried upgrading to '2.19.1'. It would appear as though I'm getting the same issue regardless of version. Regarding grpc, seems like I'm using '1.14.1'.

It seems to me as though grpc is the issue here in that "Since there is no guarantee that memory is the only resource consumed by a grpc.Channel and since some garbage collectors only collect garbage when memory is scarce, there is a liability that applications might run out of those other resources (file descriptors and so on) when they are in fact perfectly reclaimable." from here: grpc/grpc#12531

[dmarvp]

@theacodes at this point, should I create a docker image with a minimal way to reproduce it and share it with you? or would you suggest stop using the library and start hitting the HTTP endpoint directly (is that even possible?)