Skip to content

Exposing #close as a public method, to help consumers avoid using too many file descriptors#46

Merged
degemer merged 3 commits intoDataDog:masterfrom
ramfjord:expose-close-method
May 17, 2017
Merged

Exposing #close as a public method, to help consumers avoid using too many file descriptors#46
degemer merged 3 commits intoDataDog:masterfrom
ramfjord:expose-close-method

Conversation

@ramfjord
Copy link

It seems like a common use case for this client would be logging events in a web server. Many of these spin up a thread per response, or delegate response to a thread/process pool. My assumption is that these different threads/processes should not share the same Datadog::Statsd client, since I didn't see any sort of locking implemented around the socket IO.

For our use case, we don't mind creating a new UDPSocket every time we log a set of metrics, so I'd like to be able to close the socket when I'm done to avoid using thousands of file descriptors before the object get garbage collected. Please let me know if this is acceptable, or if I'm perhaps missing some more common way of using this library.

@degemer degemer added this to the Next milestone Apr 26, 2017
@degemer
Copy link
Member

degemer commented Apr 26, 2017

Hi @ramfjord, thanks for your contribution!

I'm fine with exposing close as I don't have a good answer right now for your use case, we would need to re-design Datadog::Statsd to be thread-safe which would probably require quite some work.

For the retry part: could we make sure we retry only once? Even though it's unlikely to happen, I'd prefer to avoid any risk of unbounded error loop.

…o a closed socket, and that we rescue errors from connect_to_socket when we try to reconnect
@ramfjord
Copy link
Author

@degemer I'm not sure if #connect_to_socket can ever return a closed socket, but I've added logic to retry only once if it does. Also, I realized that I wasn't catching errors raised by #connect_to_socket, which can certainly throw a SocketError if the host is no longer available, so I've added a check for that as well. Let me know if there's anything else I can do to improve this, and thanks for the quick review!

@statsd.socket = FakeUDPSocket.new
end

after { @statsd.socket.clear }
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this isn't necessary because the before clause always initializes a new socket. If I can remove it, I can also remove the after clause in the #close tests, without having to mock out #clear.

…the socket after each test, since we're reinitializing it before each test in any case
@ramfjord ramfjord force-pushed the expose-close-method branch from 5b440a5 to 081b33b Compare April 27, 2017 20:23
Copy link
Member

@degemer degemer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the changes and the tests!
LGTM, it will be part of the next release (targeting end of next week).

@degemer degemer merged commit 9d3a4c0 into DataDog:master May 17, 2017
@degemer degemer modified the milestones: 3.0.0, Next May 17, 2017
@aviflax
Copy link

aviflax commented Jun 9, 2017

Um… I’m running multiple threads on JRuby, so I have “true” concurrency — no GIL. Should I be creating an instance of Datadog::Statsd for each thread? Because right now all threads are sharing a single instance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants