Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use dogstatsd single threaded mode #1576

Merged
merged 19 commits into from
Jul 9, 2021
Merged

Use dogstatsd single threaded mode #1576

merged 19 commits into from
Jul 9, 2021

Conversation

marcotc
Copy link
Member

@marcotc marcotc commented Jul 7, 2021

This PR uses the new single_thread mode for dogstatsd-ruby clients instantiated by ddtrace.

This allows us to have the benefits of batch metric submission of dogstatsd-ruby, without memory issues we've seen with the 5.x releases.

This PR adds a new test combination for the affected 5.x versions dogstatsd-ruby, to ensure we are informing users that they should use a different version of the library in order to have a more stable environment.

@marcotc
Copy link
Member Author

marcotc commented Jul 7, 2021

I've incorporated changes from #1575 into this PR, so it should have both changes.

The only thing missing is gemfile updates, which I'm doing now.

@marcotc marcotc self-assigned this Jul 7, 2021
@marcotc marcotc added the core Involves Datadog core libraries label Jul 7, 2021
Comment on lines 267 to 279
# IMPORTANT: Once you delete this, you probably want to update the `#ignored_statsd_warning` below where we
# recommend that customers add `gem 'dogstatsd-ruby', '~> 4.0'` to their Gemfile to perhaps state something
# different.
def incompatible_statsd_warning
return if Gem.loaded_specs['dogstatsd-ruby'].version < Gem::Version.new('5.0')
return if dogstatsd_version < Gem::Version.new('5.0') || dogstatsd_version >= Gem::Version.new('5.2')

INCOMPATIBLE_STATSD_ONLY_ONCE.run do
Datadog.logger.warn(
'This version of `ddtrace` is incompatible with `dogstastd-ruby` version >= 5.0 and can ' \
'cause unbounded memory usage. Please use `dogstastd-ruby` version < 5.0 instead.'
'`ddtrace` is incompatible with `dogstastd-ruby` versions 5.0.0, 5.0.1, and 5.2.0 and can ' \
'cause unbounded memory usage. Use `dogstastd-ruby` version >= 5.2 instead.'
)
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: In #1575 we had discussed and decided to remove this, and instead just make ddtrace treat 5.0.0, 5.0.1 and 5.1.0 as "incompatible" versions and outright not use them. See the discussion in the PR for details.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it wasn't a good idea to make it incompatible, but I now agree with metrics being missing for safety. Changes applied.

Gemfile Outdated Show resolved Hide resolved
@@ -30,7 +30,7 @@ gem 'redcarpet', '~> 3.4' if RUBY_PLATFORM != 'java'
gem 'rspec', '~> 3.10'
gem 'rspec-collection_matchers', '~> 1.1'
gem 'rspec_junit_formatter', '>= 0.4.1'
gem 'rspec_n', '~> 1.3' if RUBY_VERSION >= '2.3.0'
gem 'rspec_n', '~> 1.3' if RUBY_VERSION >= '2.4.0'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest version of transient dependency cri is incompatible with Ruby 2.3.

@@ -1,4 +1,4 @@
require 'bundler/stub_specification'
require 'bundler'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surprise fix for Bundler::StubSpecification access 🤷

@codecov-commenter
Copy link

codecov-commenter commented Jul 8, 2021

Codecov Report

Merging #1576 (8b4518e) into master (b196dc3) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1576      +/-   ##
==========================================
+ Coverage   98.24%   98.25%   +0.01%     
==========================================
  Files         886      886              
  Lines       42687    42685       -2     
==========================================
+ Hits        41939    41942       +3     
+ Misses        748      743       -5     
Impacted Files Coverage Δ
lib/ddtrace/metrics.rb 97.08% <100.00%> (+1.45%) ⬆️
spec/ddtrace/metrics_spec.rb 100.00% <100.00%> (+0.38%) ⬆️
spec/ddtrace_integration_spec.rb 96.20% <100.00%> (+1.14%) ⬆️
spec/support/configuration_helpers.rb 100.00% <100.00%> (ø)
spec/ddtrace/profiling/integration_spec.rb 94.63% <0.00%> (-0.08%) ⬇️
lib/ddtrace/configuration/components.rb 98.18% <0.00%> (-0.02%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b196dc3...8b4518e. Read the comment docs.

@marcotc marcotc marked this pull request as ready for review July 8, 2021 22:17
@marcotc marcotc requested a review from a team July 8, 2021 22:17
Copy link
Member

@ivoanjo ivoanjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

spec/ddtrace/metrics_spec.rb Outdated Show resolved Hide resolved
@marcotc marcotc merged commit 3c904b4 into master Jul 9, 2021
@marcotc marcotc deleted the statsd-single-thread branch July 9, 2021 19:23
@github-actions github-actions bot added this to the 0.51.0 milestone Jul 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Involves Datadog core libraries
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants