Skip to content

Conversation

@r1viollet
Copy link
Collaborator

@r1viollet r1viollet commented Dec 3, 2025

What does this PR do?

handle the case where we can't perform the double mapping

Motivation

User reported that we do not load on newer kernels.

Additional Notes

Tested on recent kernel

uname -a
Linux r1viollet-QEMU-Virtual-Machine 6.17.0-6-generic #6-Ubuntu SMP PREEMPT_DYNAMIC Tue Oct  7 14:22:06 UTC 2025 aarch64 GNU/Linux

Before

./ddprof-arm64 sleep 10
<ERROR>Dec 03 17:17:40.385527 ddprof[4618]: Could not mmap memory for watcher #0: Invalid argument. Please increase kernel limits on pinned memory (ulimit -l). OR associate the IPC_LOCK capability to this process.
<ERROR>Dec 03 17:17:40.385560 ddprof[4618]: PERFMMAP: error in mmap operations at /go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/src/pevent_lib.cc:216
<ERROR>Dec 03 17:17:40.385564 ddprof[4618]: Forward error at /go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/src/pevent_lib.cc:255 - PERFMMAP: error in mmap operations
<ERROR>Dec 03 17:17:40.392791 ddprof[4618]: Could not mmap memory for watcher #0: Invalid argument. Please increase kernel limits on pinned memory (ulimit -l). OR associate the IPC_LOCK capability to this process.
<ERROR>Dec 03 17:17:40.392814 ddprof[4618]: PERFMMAP: error in mmap operations at /go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/src/pevent_lib.cc:216
<ERROR>Dec 03 17:17:40.392818 ddprof[4618]: Forward error at /go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/src/pevent_lib.cc:255 - PERFMMAP: error in mmap operations
<ERROR>Dec 03 17:17:40.392821 ddprof[4618]: Forward error at /go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/src/pevent_lib.cc:268 - PERFMMAP: error in mmap operations
<ERROR>Dec 03 17:17:40.392823 ddprof[4618]: Forward error at /go/src/github.com/DataDog/apm-reliability/ddprof-build/ddprof/src/ddprof.cc:89 - PERFMMAP: error in mmap operations
<ERROR>Dec 03 17:17:40.392826 ddprof[4618]: Failed to initialize profiling
<ERROR>Dec 03 17:17:40.394082 ddprof-library[4616]: Unable to connect to socket: Unknown error

Now we fall back to a single mapping.

../../ddprof -l notice ./src/BadBoggleSolver_run 1000
<NOTICE>Dec 03 17:27:37.905974 ddprof[8973]: Successfully calibrated TSC from perf (mult: 87381333, shift: 21, offset: -523777351127)
<WARNING>Dec 03 17:27:37.916109 ddprof[8973]: Slow clock source detected. Current clock source: arch_sys_counter. Available clock sources: arch_sys_counter .
<WARNING>Dec 03 17:27:37.927817 ddprof[8973]: Unable to create mirrored perf ring buffer mapping, falling back to single mapping with wrap-around copies.

How to test the change?

I had to boot a more recent kernel version.

@r1viollet r1viollet force-pushed the r1viollet/fix_mirror_mmap branch from 254e710 to aa1d877 Compare December 3, 2025 10:53
@pr-commenter
Copy link

pr-commenter bot commented Dec 3, 2025

Benchmark results for collatz

Parameters

Baseline Candidate
config baseline candidate
profiler-version ddprof 0.22.1+52c3b33f.81451347 ddprof 0.22.1+f79cec9b.84860473

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean execution_time
scenario:ddprof -S bench-collatz --preset cpu_only collatz_runner.sh same

@pr-commenter
Copy link

pr-commenter bot commented Dec 3, 2025

Benchmark results for BadBoggleSolver_run

Parameters

Baseline Candidate
config baseline candidate
profiler-version ddprof 0.22.1+52c3b33f.81451347 ddprof 0.22.1+f79cec9b.84860473

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean execution_time
scenario:ddprof -S bench-bad-boggle-solver BadBoggleSolver_run work 1000 same

@r1viollet r1viollet force-pushed the r1viollet/fix_mirror_mmap branch from 9ecaf0c to aa1d877 Compare December 3, 2025 15:09
@r1viollet r1viollet marked this pull request as ready for review December 3, 2025 15:20
@r1viollet r1viollet requested a review from nsavoire as a code owner December 3, 2025 15:20
@r1viollet r1viollet changed the title Fix double mmap failures on newer kernel versions Avoid mirror mmap failures on newer kernel versions Dec 3, 2025
Manual test for ring buffer wrap logics
if (required_size == 0) {
return rb.wrap_copy;
}
size_t const aligned_size = align_up(required_size, kRingBufferAlignment);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to bother about alignment, ring buffer alignment (8) will always be satisfied by memory returned by malloc / new.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, though code should be valid regardless of kRingBufferAlignment's value

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to keep things simple, static assert that kRingBufferAlignment is less than 16 and use make_unique:

inline std::byte *RingBuffer::ensure_wrap_copy_buffer(size_t required_size) {
  static_assert(kRingBufferAlignment <= 16, "kRingBufferAlignment must be <= 16");
  if (this->buffer_capacity < required_size) {
    this->buffer = std::make_unique<std::byte[]>(required_size);
    this->buffer_capacity = required_size;
  }
  return this->buffer.get();
}

auto defer_munmap = make_defer([&]() { perfdisown(region, size_of_buffer); });

auto *ptr = static_cast<std::byte *>(region);
PerfMmapRegion perfown_sz(int fd, size_t size_of_buffer,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicating the function into two versions would make the code simpler

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I see it considering how we catch the failures.

@r1viollet r1viollet requested a review from nsavoire December 4, 2025 09:59
@r1viollet r1viollet merged commit 0a7e43a into main Dec 4, 2025
3 checks passed
@r1viollet r1viollet deleted the r1viollet/fix_mirror_mmap branch December 4, 2025 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants