Request: Add support for HTTP Keep-Alive on BigQuery Inserts

[martincad]

I noticed running some sequential BigQuery inserts using the library that for every insert the library uses a new HTTP connection which requieres of course one TCP and one SSL handshake. (I checked this was indeed happening with tcpdump/Wireshark).

I just did a small tweak on the code which is passing to the request module the argument forever:true, which means use HTTP Keep-Alive if the server supports it. The change indeed was to add to the common/src/util.js file the following

forever : true

After that change, the same HTTP connection was reused. (no more than 1 SYN on the tcpdump capture)

On some basic performance tests I did, the time saved was about 30% (of course this will vary depending on the network latency), however the resources saved (not only on the client but on the server are huge).

Is there a way why the google-cloud-node lib is not reusing connections (forever=true) ? If there is not, can you add it to it ?

Please let me know any other relevant information you may need for me.

As always, thanks for you help and support.

[stephenplusplus]

This is a great and very appreciated observation! I don't see any reason not to use this, so yes, definitely we should do this. Would you be interested in sending a PR?

[ericuldall]

Just linked a PR (CI in progress). Thanks @martincad for catching this!

[martincad]

This has already been merged. Should I Close it ? Or is close it one its added into a release ?

Thanks !

[stephenplusplus]

Woops, thanks for bringing that up. I'll close this one. We should start merging and releasing early/mid-next week. Sorry for the delay on this one!

[martincad]

@stephenplusplus Sorry I keep asking but I have not seen common package as been tag, so I'm wondering if this change will be included on the next release.

We really need to have this change on the next release.

Thanks a lot.