Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Will RR work in WSL 2 #2506

Closed
dulinriley opened this issue Apr 14, 2020 · 14 comments
Closed

Will RR work in WSL 2 #2506

dulinriley opened this issue Apr 14, 2020 · 14 comments

Comments

@dulinriley
Copy link

dulinriley commented Apr 14, 2020

I want to try out RR on a project of mine, and the easiest way for me to try it out is through a WSL 2 Ubuntu VM on a Windows computer.

After downloading v5.3.0 using the instructions from the homepage:

wget https://github.com/mozilla/rr/releases/download/5.3.0/rr-5.3.0-Linux-$(uname -m).deb
sudo dpkg -i rr-5.3.0-Linux-$(uname -m).deb

And using it on an example binary:

rr record ./a.out

I get the following error message:

[FATAL /home/roc/rr/rr/src/PerfCounters.cc:314:start_counter() errno: ENOENT] Unable to open performance counter with 'perf_event_open'; are perf events enabled? Try 'perf record'.
=== Start rr backtrace:
rr(_ZN2rr13dump_rr_stackEv+0x3b)[0x5f229b]
rr(_ZN2rr15notifying_abortEv+0x47)[0x5f2327]
rr[0x52cbea]
rr[0x52dede]
rr(_ZN2rr12PerfCounters23default_ticks_semanticsEv+0x1a)[0x52e9ca]
rr(_ZN2rr7SessionC1Ev+0x139)[0x5bc369]
rr(_ZN2rr13RecordSessionC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt6vectorIS6_SaIS6_EESD_RKNS_20DisableCPUIDFeaturesENS0_16SyscallBufferingEiNS_7BindCPUES8_PKNS_9TraceUuidE+0x4d)[0x54069d]
rr(_ZN2rr13RecordSession6createERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EESB_RKNS_20DisableCPUIDFeaturesENS0_16SyscallBufferingEhNS_7BindCPUERKS7_PKNS_9TraceUuidE+0xba9)[0x541769]
rr(_ZN2rr13RecordCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0xe14)[0x535c24]
rr(main+0x353)[0x4ac743]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f77633761e3]
rr(_start+0x29)[0x4acb59]
=== End rr backtrace
Aborted

Given the advice, I looked into how to use perf record on WSL 2. According to this issue: microsoft/WSL#329, it looks like all I need to do is build perf from source (which is thankfully very easy) and run it.

Here's what I ran:

git clone --depth 1 --shallow-submodules https://github.com/microsoft/WSL2-Linux-Kernel.git
cd ./WSL2-Linux-Kernel/tools
make perf
./perf/perf record ~/a.out

This command seems to run fine! I can view the trace with perf report and there were no error messages.

So why is perf_event_open returning ENOENT to RR?

Some of my build info in case any of that is relevant:

Field Value
Windows Build number 19041.173
CPU Intel Core i7-7700HQ
uname -sr Linux 4.19.84-microsoft-standard
rr --version 5.3.0

EDIT: Tried on the newest Windows Insiders build as of 2020-04-13 (build number 19603), still no luck

@dulinriley
Copy link
Author

I didn't add this as a comment on #2118 because that was for WSL 1, which wasn't trying to allow tools like perf to work on them.
Now WSL 2 is more like a VM (using Hyper-V), so it might be able to work.

@rocallahan
Copy link
Collaborator

Hyper-V doesn't virtualize hardware performance counters as far as I know. If you run perf list and it shows [Hardware event] anywhere then hardware counters work, but I expect that it won't. So this is really an issue you would need to take up with Microsoft.

@dulinriley
Copy link
Author

dulinriley commented Apr 14, 2020

I don't see any [Hardware event], but I do see a [Hardware breakpoint] and [Raw hardware event descriptor].

This article seems to imply that Hyper-V does allow perf counters: https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/manage/performance-monitoring-hardware

although WSL 2 VMs do not show up in the various *-VMProcessor powershell commands.

@rocallahan
Copy link
Collaborator

rocallahan commented Apr 14, 2020

I don't see any [Hardware event], but I do see a [Hardware breakpoint] and [Raw hardware event descriptor].

[Hardware breakpoint] is not a performance event, it just gets exposed through perf because perf exposes lots of random stuff. [Raw hardware event descriptor] is just a generic line that is always printed. So your WSL environment doesn't have hardware performance counters enabled.

This article seems to imply that Hyper-V does allow perf counters: https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/manage/performance-monitoring-hardware

Great, that's good news! You need to find a way to turn that on for WSL2.

If you successfully turn it on you should see stuff like this in perf list:

  branch-instructions OR branches                    [Hardware event]
  branch-misses                                      [Hardware event]
  bus-cycles                                         [Hardware event]
  cache-misses                                       [Hardware event]
  cache-references                                   [Hardware event]
  cpu-cycles OR cycles                               [Hardware event]
  instructions                                       [Hardware event]
  ref-cycles                                         [Hardware event]

Then you can try rr. There are many more reasons why rr might not work, some which we could fix and some we couldn't, but it's definitely not going to work until you get that perf list output.

@khuey
Copy link
Collaborator

khuey commented Apr 14, 2020

I'm 99.9% sure that WSL's ptrace implementation is still not sufficient too.

@rocallahan
Copy link
Collaborator

I'm 99.9% sure that WSL's ptrace implementation is still not sufficient too.

That's WSL1. WSL2 is real Linux so ptrace should be OK.

@kruxmeier
Copy link

The performance counters can't be unlocked via normal Hyper-V settings, but you can modify vmwp.exe.

@Trass3r
Copy link

Trass3r commented Dec 8, 2021

It will work in the Store version of WSL.

@tyler274
Copy link

tyler274 commented Feb 16, 2022

It will work in the Store version of WSL.

What do you mean?

Ah I see HW perf counters was shipped with 0.50.2

edit: It seems to work in WSL2 now.

@vjpr vjpr mentioned this issue Jul 16, 2022
@J-jaeyoung
Copy link

For those who are still experiencing a perf issue, just try wsl --update command on powershell and relaunch the wsl.
After doing so, rr works perfectly fine on wsl2+win11 setting.

@Kodp
Copy link

Kodp commented May 1, 2024

@J-jaeyoung
I'm using WSL2 and Windows 11, but I'm still encountering issues.

When I run rr record ./rtarget -q -i ./raw_5_exploit.o, it saves execution to the trace directory `/home/syh/.local/share/rr/rtarget-4', but then I receive the following fatal error:

[FATAL /home/ubuntu/rr/src/PerfCounters.cc:316:start_counter()] Unable to open performance counter with 'perf_event_open'; are hardware perf events available? See https://github.com/rr-debugger/rr/wiki/Will-rr-work-on-my-system

When I follow the instructions in rr/wiki/Will-rr-work-on-my-system and execute the command:

$ perf stat -e br_inst_retired.conditional true

I encounter the following warning:

WARNING: perf not found for kernel 5.15.146.1-microsoft

  You may need to install the following packages for this specific kernel:
    linux-tools-5.15.146.1-microsoft-standard-WSL2
    linux-cloud-tools-5.15.146.1-microsoft-standard-WSL2

  You may also want to install one of the following packages to keep up to date:
    linux-tools-standard-WSL2
    linux-cloud-tools-standard-WSL2

I tried to install the mentioned Microsoft packages using apt or snap, but they couldn't be found:

$ sudo apt install linux-tools-5.15.146.1-microsoft-standard-WSL2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package linux-tools-5.15.146.1-microsoft-standard-WSL2
E: Couldn't find any package by glob 'linux-tools-5.15.146.1-microsoft-standard-WSL2'

Have you encountered these issues before?

@J-jaeyoung
Copy link

@song-yuhang

Try the following commands.

git clone --depth 1 --shallow-submodules https://github.com/microsoft/WSL2-Linux-Kernel.git
cd ./WSL2-Linux-Kernel/tools
make perf

cp perf /usr/local/bin     # if you want

@vimkim
Copy link

vimkim commented Jun 4, 2024

❯ perf stat -e br_inst_retired.conditional true
event syntax error: 'br_inst_retired.conditional'
                     \___ parser error
Run 'perf list' for a list of valid events

 Usage: perf stat [<options>] [<command>]

    -e, --event <event>   event selector. use 'perf list' to list available events

This is my result, after I did what @J-jaeyoung exactly mentioned. I ended up here.

@GitMensch
Copy link
Contributor

@vimkim The exact name depends on both the processor and the perf version - see https://github.com/rr-debugger/rr/wiki/Will-rr-work-on-my-system

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants