DD_PROFILING_NO_EXTENSION required when building native extensions using a nix environment

[langsharpe]

As requested in the help message, here is an example of the new native extension failing to build. Perhaps the nix package doesn't include the required files?

Let me know if there is anything I can do to help.

❯ nix-shell -p ruby

❯ uname -a
Darwin index.rubygems.org 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64 x86_64 i386 MacBookPro15,2 Darwin

❯ which ruby
/nix/store/6pmrf1633z5c9b73w6a8ah4h65gxhqvb-ruby-2.7.4/bin/ruby

❯ ruby --version
ruby 2.7.4p191 (2021-07-07) [x86_64-darwin17]

❯ which gem
/nix/store/6pmrf1633z5c9b73w6a8ah4h65gxhqvb-ruby-2.7.4/bin/gem

❯ gem install ddtrace -v '0.54.0' --source 'https://rubygems.org/'
Fetching msgpack-1.4.2.gem
Fetching ddtrace-0.54.0.gem
Fetching debase-ruby_core_source-0.10.12.gem
Building native extensions. This could take a while...
Successfully installed msgpack-1.4.2
Successfully installed debase-ruby_core_source-0.10.12
Building native extensions. This could take a while...
ERROR:  Error installing ddtrace:
	ERROR: Failed to build gem native extension.

    current directory: /Users/lang.sharpe/.local/share/gem/ruby/2.7.0/gems/ddtrace-0.54.0/ext/ddtrace_profiling_native_extension
/nix/store/6pmrf1633z5c9b73w6a8ah4h65gxhqvb-ruby-2.7.4/bin/ruby -I /nix/store/6pmrf1633z5c9b73w6a8ah4h65gxhqvb-ruby-2.7.4/lib/ruby/2.7.0 -r ./siteconf20211118-29473-1wrgai5.rb extconf.rb

+------------------------------------------------------------------------------+
| **Preparing to build the ddtrace native extension...**                       |
|                                                                              |
| If you run into any failures during this step, you can set the               |
| `DD_PROFILING_NO_EXTENSION` environment variable to `true` e.g.              |
| `$ DD_PROFILING_NO_EXTENSION=true bundle install` to skip this step.         |
|                                                                              |
| Disabling the extension will lead to the ddtrace profiling features not      |
| working in future releases.                                                  |
| If you needed to use this, please tell us why on                             |
| <https://github.com/DataDog/dd-trace-rb/issues/new> so we can fix it :)      |
|                                                                              |
| Thanks for using ddtrace! You rock!                                          |
+------------------------------------------------------------------------------+

creating extconf.h
creating Makefile

current directory: /Users/lang.sharpe/.local/share/gem/ruby/2.7.0/gems/ddtrace-0.54.0/ext/ddtrace_profiling_native_extension
make DESTDIR\= clean

current directory: /Users/lang.sharpe/.local/share/gem/ruby/2.7.0/gems/ddtrace-0.54.0/ext/ddtrace_profiling_native_extension
make DESTDIR\=
compiling clock_id_from_pthread.c
compiling clock_id_noop.c
compiling private_vm_api_access.c
private_vm_api_access.c:13:10: fatal error: 'rb_mjit_min_header-2.7.4.h' file not found
#include RUBY_MJIT_HEADER
         ^~~~~~~~~~~~~~~~
./extconf.h:4:26: note: expanded from macro 'RUBY_MJIT_HEADER'
#define RUBY_MJIT_HEADER "rb_mjit_min_header-2.7.4.h"
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [Makefile:244: private_vm_api_access.o] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/lang.sharpe/.local/share/gem/ruby/2.7.0/gems/ddtrace-0.54.0 for inspection.
Results logged to /Users/lang.sharpe/.local/share/gem/ruby/2.7.0/extensions/x86_64-darwin-17/2.7.0/ddtrace-0.54.0/gem_make.out

[ivoanjo]

Not good! Thanks @langsharpe for reporting. I'll investigate and report back.

Feel free to keep using DD_PROFILING_NO_EXTENSION=true in this case, as it has very little impact on macOS (and no impact at all if you don't use profiling).

[ivoanjo]

I've been investigating this, and the issue is that the nixos Ruby binary has been built without JIT support, which the profiling native extension relies on.

I've opened #1776 to add auto-detection of this situation, and skip building the extension, so that installs are not impacted.

Thanks a lot @langsharpe for the awesome report. Just to sum up the current status:

• If you did not intend to use profiling, there is no impact to you other than needing to use DD_PROFILING_NO_EXTENSION=true while installing dd-trace-rb 0.54.0. This will go away with the next release.

• If you did intend to use profiling and also use nixos on Linux, your profiles will have wall-clock data, but no CPU time profiling data.

• If you did intend to use profiling, but on macOS, there is currently no impact to you (other than needing the env var), since the profiling native extension doesn't yet have support for extracting CPU time profiling data on macOS. But, in future releases of dd-trace-rb, it's possible that the profiler will not turn on altogether, and will just emit a warning when you try to do so.

Hope this works for you!

[langsharpe]

Thanks, @ivoanjo. For us, this was a developer experience problem and not a production problem. Thank you for the fast and knowledgeable response.

[ivoanjo]

@langsharpe thanks again for the report. The fix has been released as 0.54.1 so you'll no longer need DD_PROFILING_NO_EXTENSION=true when installing the gem 🎉