Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instrumentation for Concurrent Ruby #1682

Merged
merged 70 commits into from Jan 5, 2023
Merged

Conversation

kaylareopelle
Copy link
Contributor

@kaylareopelle kaylareopelle commented Dec 8, 2022

Overview

This PR adds automatic instrumentation for the concurrent-ruby gem to the Ruby agent.

We've instrumented Concurrent::ThreadPoolExecutor#post, which is called by many methods within concurrent-ruby including, but not limited to:

  • Concurrent::Promises.future
  • Concurrent::Future.execute

Now, what happens within the blocks passed to these methods will be visible to the agent. Concurrent::ThreadPoolExecutor#post will add a segment to an existing transaction called Concurrent/Task and create nested segments for calls to libraries the agent instruments within the passed-in block, such as Net::HTTP requests.

Furthermore, a new Supportability metric will be generated the first time the agent fields a call to Concurrent::ThreadPoolExecutor#post, helping us better understand which customers use the concurrent-ruby so that we can improve our instrumentation in the future.

Also, we've updated the Ruby/Thread instrumentation omit the Thread id from the metric name only when thread_id_enabled is set to true. It is false and private by default. We believe Thread ids in metrics are valuable for debugging the agent, so we want to have this option easily available if support tickets arise.

Closes #1601
Closes #1705

Submitter Checklist:

  • Include a link to the related GitHub issue, if applicable
  • Include a security review link, if applicable

Testing

The agent includes a suite of unit and functional tests which should be used to
verify your changes don't break existing functionality. These tests will run with
GitHub Actions when a pull request is made. More details on running the tests locally can be found
here for our unit tests,
and here for our functional tests.
For most contributions it is strongly recommended to add additional tests which
exercise your changes.

Reviewer Checklist

  • Perform code review
  • Add performance label
  • Perform appropriate level of performance testing
  • Confirm all checks passed
  • Add version label prior to acceptance

@kaylareopelle kaylareopelle changed the title Add instrumentation for Concurrent::Promises#future Add instrumentation for Concurrent::ThreadPoolExecutor#post Dec 14, 2022
@kaylareopelle
Copy link
Contributor Author

The changes I pushed on 12/14 change the following:

  • Instrumentation now points to Concurrent::ThreadPoolExecutor#post instead of Concurrent::Promises#future
  • There isn't an option to set your own name anymore. There are too many methods that feed into ThreadPoolExecutor to make this safe.

We're getting segments nested within a transaction that have the DEFAULT_NAME. Calling something else that gets instrumented within Concurrent::Promises#future, like a Net::HTTP.get call, will also be reported if instrumentation.thread.tracing is true. However, it'll appear as a fragment, not as a nested call.

Next steps:

  • Have a Net::HTTP.get call correctly nest under the Concurrent::ThreadPoolExecutor#post segment
  • Verify the segment for Concurrent::ThreadPoolExecutor#post gets generated if it's called within a Rails controller action

@kaylareopelle
Copy link
Contributor Author

It also looks like I may have some syntax that isn't compatible with Ruby 2.2: https://github.com/newrelic/newrelic-ruby-agent/actions/runs/3699041227/jobs/6265975231#step:11:3068

if use_prepend?
prepend_instrument(Concurrent::ThreadPoolExecutor, NewRelic::Agent::Instrumentation::ConcurrentRuby::Prepend)
else
chain_instrument NewRelic::Agent::Instrumentation::ConcurrentRuby::Chain
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this class has to have a ::Chain suffix to have the log messages for what was instrumented show up correctly. This method looks for the second-to-last namespace and puts that in the logs. Without the ::Chain suffix, "Instrumentation" is logged instead of "ConcurrentRuby". I'll make a separate PR to fix this in the instrumentation generator.

@kaylareopelle kaylareopelle changed the base branch from dev to concurrent_ruby December 14, 2022 22:11
@kaylareopelle kaylareopelle changed the base branch from concurrent_ruby to dev December 14, 2022 22:14
# raise future rescue $! => rescues the error, returns inspected version

def test_promises_future_captures_segment_error
skip "future doesn't raise errors, so they can't be captured"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a different problem someone could tackle. See comments above the test for background. Essentially, the return value if an error is raised is nil. You have to call a different method to get the error. What should we do to capture errors automatically within calls to Concurrent::Promises#future?

Turns out we do need to copy the current transaction over every time.
Concurrent ruby can create a thread prior to the transaction beginning
or during a different transaction, but continue to use the same thread.
This ensures the agent is always using the most accurate source
for the current transaction
Concurrent ruby will reuse threads created before a transaction
began, which causes the thread creation segment to not appear in the
expected transaction.
@kaylareopelle kaylareopelle marked this pull request as ready for review January 4, 2023 22:51
CHANGELOG.md Outdated Show resolved Hide resolved
Co-authored-by: James Bunch <fallwith@gmail.com>
end

def test_task_segment_has_correct_parent
txn = future_in_transaction { 'are you my mother?' }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a movie that has a character deliver this line and follow it up with "Am I part of you both?" but I now I can't remember it's name. Something about a garden I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting! I was thinking Dr. Seuss when I wrote this one. If you think of the title, let me know!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, the word "garden" was in the quote itself and it's from a book - not a movie.

“Are these Gardens my mother?” she asked suddenly, and he told her they were. “Am I a part of you both?” she asked, and he told her yes.

The Druid of Shannara

Now I'm wondering why I thought it was a movie. I definitely have an old (15+ years?) audio clip of the quote. Maybe there was an audio book? Weird.

fallwith
fallwith previously approved these changes Jan 5, 2023
fallwith
fallwith previously approved these changes Jan 5, 2023
@kaylareopelle kaylareopelle self-assigned this Jan 5, 2023
@github-actions
Copy link

github-actions bot commented Jan 5, 2023

SimpleCov Report

Coverage Threshold
Line 93.22% 93%
Branch 84.5% 84%

@kaylareopelle kaylareopelle merged commit 84c6249 into dev Jan 5, 2023
@kaylareopelle kaylareopelle deleted the concurrent_ruby_promises branch January 5, 2023 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants