Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Add helper to easily enable core dumps #2010

Merged
merged 6 commits into from
May 13, 2022

Conversation

ivoanjo
Copy link
Member

@ivoanjo ivoanjo commented May 10, 2022

Enabling core dumps on an application can get confusing/fiddly when the application may be running in a container and thus adding the needed ulimit -c ... configuration may be non-trivial.

Last year while debugging what turned out to be a Ruby VM bug, I created a helper that allows a Ruby app to turn core dumps on for itself.

Is this something we're interested in shipping? The advantage here is that to get a core dump, you'd only need to add require 'datadog/kit/enable_core_dumps' to an application.

Here's a simple example using the Ruby VM crash I referenced above.

Without helper (no core dump):

$ docker-compose run --rm tracer-2.6 /bin/bash --init-file .config
Creating dd-trace-rb_tracer-2.6_run ... done
root# bundle exec ruby -e "Process.detach(fork { sleep }).instance_variable_get(:@hello)"
-e:1: [BUG] Segmentation fault at 0x0000000000000008
ruby 2.6.7p197 (2021-04-05 revision 67941) [x86_64-linux]
...
Aborted

With helper:

root# bundle exec ruby -e "require 'datadog/kit/enable_core_dumps'; Process.detach(fork { sleep }).instance_variable_get(:@hello)"
[DDTRACE] Enabled core dumps. Maximum size: 18446744073709551615 Output pattern: 'core'
-e:1: [BUG] Segmentation fault at 0x0000000000000008
ruby 2.6.7p197 (2021-04-05 revision 67941) [x86_64-linux]
...
Aborted (core dumped)

Enabling core dumps on an application can get confusing/fiddly
when the application may be running in a container and thus adding the
needed `ulimit -c ...` configuration may be non-trivial.

Last year while debugging what turned out to be a Ruby VM bug, I
created a helper that allows a Ruby app to turn core dumps on for
itself.

Is this something we're interested in shipping? The advantage here
is that to get a core dump, you'd only need to add
`require 'datadog/enable_core_dumps'` to an application.

Here's a simple example using the Ruby VM crash I referenced
above.

Without helper (no core dump):

```
$ docker-compose run --rm tracer-2.6 /bin/bash --init-file .config
Creating dd-trace-rb_tracer-2.6_run ... done
root# bundle exec ruby -e "Process.detach(fork { sleep }).instance_variable_get(:@hello)"
-e:1: [BUG] Segmentation fault at 0x0000000000000008
ruby 2.6.7p197 (2021-04-05 revision 67941) [x86_64-linux]
...
Aborted
```

With helper:

```
root# bundle exec ruby -e "require 'datadog/enable_core_dumps'; Process.detach(fork { sleep }).instance_variable_get(:@hello)"
[DDTRACE] Enabled core dumps. Maximum size: 18446744073709551615 Output pattern: 'core'
-e:1: [BUG] Segmentation fault at 0x0000000000000008
ruby 2.6.7p197 (2021-04-05 revision 67941) [x86_64-linux]
...
Aborted (core dumped)
```
@ivoanjo ivoanjo requested a review from a team May 10, 2022 14:13
Copy link
Contributor

@lloeki lloeki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very useful. Probably needs an update to CI to gather the actual core dump files for us to use.

@@ -0,0 +1,40 @@
# This helper is used to enable core dumps for the current Ruby app. This is useful when debugging native-level
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has a significant side effect, so it needs # typed: ignore otherwise Sorbet will evaluate the file!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 14afd4d

lib/datadog/enable_core_dumps.rb Outdated Show resolved Hide resolved
@ivoanjo
Copy link
Member Author

ivoanjo commented May 11, 2022

I've also added specs for it; yesterday I was a bit hesitant because this may be thrown away, but if we're keeping it, specs it gets!

@ivoanjo ivoanjo requested a review from lloeki May 12, 2022 07:33
@ivoanjo ivoanjo merged commit 80e443a into master May 13, 2022
@ivoanjo ivoanjo deleted the ivoanjo/rfc-enable-core-dumps-helper branch May 13, 2022 10:34
@github-actions github-actions bot added this to the 1.1.0 milestone May 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants