Attempt to set memory executable before setting up YJIT and deactivate YJIT on failure

[noahgibbs]

Using Docker and QEMU, running what looks like x86-based YJIT on an ARM64 host, @byroot encountered some problems. That makes sense, and it's a great example of the kind of platform we will probably never test regularly.

But he points out that if we did a quick capability check for setting memory as executable, we could refuse to activate YJIT if it fails. I've seen this error happen before on weird platforms, and that sounds like a very sane approach to it. Then we'd have no YJIT in those weird cases, but also few-or-no bug reports.

@XrXr - does this seem like a reasonable approach to you?

[casperisfine]

I found this bug report: https://developer.apple.com/forums/thread/672804

Seem to be a bug in macOS. Either way would be nice to handle it gracefully.

[maximecb]

Using Docker and QEMU, running what looks like x86-based YJIT on an ARM64 host

Can you provide a bit more context? It's not fully clear to me what you mean by this.

You mean running x86 YJIT inside a QEMU+Docker emulator, on an arm64 machine? (I'm also curious why we're doing this since x86 machines seem to be easier to get ahold of than arm machines?)

I know there is a thread on slack but we should also try to make GitHub issues self-contained if possible. I saw Jean posted this error message:

$ docker run -it rubylang/ruby:master-debug-nightly-focal ruby --yjit -e 'p 1'
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
ruby: [BUG] Couldn't make JIT page (0x0000ffff9033b000, 0 bytes) executable, errno: Cannot allocate memory

ruby 3.2.0dev (2022-09-26T16:21:58Z master 830b5b5c35) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0001 p:0000 s:0003 E:000310 (none) [FINISH]


-- C level backtrace information -------------------------------------------
/usr/local/lib/libruby.so.3.2(rb_print_backtrace+0x19) [0x4001ae4ef9] /usr/src/ruby/vm_dump.c:762
/usr/local/lib/libruby.so.3.2(fprintf+0x0) [0x4001ae5010] /usr/src/ruby/vm_dump.c:1057
/usr/local/lib/libruby.so.3.2(rb_vm_bugreport) /usr/src/ruby/vm_dump.c:1060
/usr/local/lib/libruby.so.3.2(rb_bug_without_die+0x68) [0x400194cfa8] /usr/src/ruby/error.c:795
/usr/local/lib/libruby.so.3.2(rb_bug+0x9f) [0x40018cc0c0] /usr/src/ruby/error.c:803
/usr/local/lib/libruby.so.3.2(rb_yjit_mark_executable+0x4d) [0x4001aebb5d] /usr/src/ruby/yjit.c:78
/usr/local/lib/libruby.so.3.2(0x4001bade6f) [0x4001bade6f]
/usr/local/lib/libruby.so.3.2(rb_yjit_init_rust) [0x4001b1680f]
/usr/local/lib/libruby.so.3.2(rb_yjit_init+0xd) [0x4001aec80d] /usr/src/ruby/yjit.c:1020
/usr/local/lib/libruby.so.3.2(process_options+0x5b5) [0x4001a56145] /usr/src/ruby/ruby.c:1870
/usr/local/lib/libruby.so.3.2(ruby_process_options+0x8d) [0x4001a56a2d] /usr/src/ruby/ruby.c:2679
/usr/local/lib/libruby.so.3.2(ruby_options+0xc3) [0x4001956c03] /usr/src/ruby/eval.c:118
/usr/local/bin/ruby(rb_main+0x33) [0x4000001283] /usr/src/ruby/main.c:38
/usr/local/bin/ruby(main+0x38) [0x4000001158] /usr/src/ruby/main.c:57

If this is purely a limitation of QEMU Docker which doesn't support JIT compilation (that's surprising and too bad), then I agree we should handle it more gracefully. I think what you are suggesting is that rb_yjit_init should be able to fail gracefully, which makes sense.

[casperisfine]

I'm also curious why we're doing this since x86 machines seem to be easier to get ahold of than arm machines?

Well, the vast majority of Docker image out there are built for x86. You can provide multi-arch images, but that's still rare.

So when you are working on a M1 Mac and trying to do Docker stuff, it's extremely frequent that you end up running an x86 image, and Docker for Mac transparently run it through Qemu. So that's how.

It's somewhat likely that people will run into this if they are debugging their production docker images on an M1 mac.

If this is purely a limitation of QEMU Docker which doesn't support JIT compilation

From the bug report I found, this seem to be a Mac specific issue of Qemu, something about a Mac security feature or something.

[k0kubun]

I agree that multi-arch images are still rare (while we do have ruby/docker-images#39) and it's sometimes convenient to just rely on the x86 emulation. However, I think the user should be aware that emulation is slow. If a user wants faster Ruby, it seems reasonable to ask them to prepare a multi-arch image and avoid emulation, whether you use YJIT or not.

Anyway, until we find a way around it, just giving up JIT on the environment seems to be the only option. And I agree with what the issue description is proposing.

[k0kubun]

https://developer.apple.com/forums/thread/672804
on 11.2, if PROT_WRITE + PROT_EXEC are specified on mmap(MAP_JIT), then any mprotect operations on the allocated page are failing with "Permission denied".

Well, does this mean it could work if we transition PROT_NONE to PROT_READ | PROT_WRITE | PROT_EXEC and then leave it, giving up W^X for such environments?

[byroot]

I agree that multi-arch images are still rare (while we do have ruby/ruby-docker-images#39)

Well, that still doesn't provide native images for M1 macs. I think it's something about linux/arm64 vs linux/arm64/v8, there something about it here: ruby/docker-images#35

[noahgibbs]

Pretty sure Jean noticed already, but worth pointing out:

The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8)

This says amd64 (that is, an x86-based image) versus arm64/v8 (ARM-based image.)

[casperisfine]

🤦 I always misread amd64 as arm64. But yeah the problem remain, it's extremely common for M1 users to use x86 docker images.

As for ruby/docker-images#39, while in theory it allows for ARM images, it seems that only x86 ones are published https://hub.docker.com/r/rubylang/ruby/tags

[tatsuya6502]

Hi. I and @sisshiki1969 looked into this issue, and found the root cause of the following error:

ruby: [BUG] Couldn't make JIT page (0x0000ffff9033b000, 0 bytes) executable, errno: Cannot allocate memory

However, after fixing it, Ruby apparently got hung somewhere before start running a Ruby script. So there must be another issue to fix.

The above error can be avoided by modifying rb_yjit_mark_executable not to call mprotect when len is 0 byte.

This was caused by a subtle behavior difference on mprotect system call between Linux kernel and qemu-x86_64. qemu-x86_64 is a user space emulator, which runs on a host Linux kernel (AArch64 for our case), emulates a target CPU (x86_64 for our case) and translates Linux system calls (from x86_64 to AArch64).

(Docker Desktop for Mac on Apple silicon runs qemu-system-aarch64. It is a system emulator and runs a VM with Arm (AArch64) Linux. Inside the VM, qemu-x86_64 is used to run x86-based YJIT in the x86-based Docker image)

On Linux kernel, mprotect system call with len 0 byte does nothing and returns 0 (success). However, on qemu-x86_64, it returns -ENOMEM (Cannot allocate memory) error, which causes YJIT to fail with the above error.

• qemu-x86_64 — guest_range_valid_untagged returns false when len is 0.
  • https://gitlab.com/qemu/qemu/blob/c8de6ec63d766ca1998c5af468483ce912fdc0c2/include/exec/cpu_ldst.h#L104
• Then the caller target_mprotect returns -TARGET_ENOMEM.
  • https://gitlab.com/qemu/qemu/blob/c8de6ec63d766ca1998c5af468483ce912fdc0c2/linux-user/mmap.c#L133

We will submit a patch to the QEMU project to make qemu-x86_64's mprotect behave like Linux kernel when len is 0.

For the another issue (Ruby got hung), I will look into it when I have time. But I have no idea if I can find the root cause and how difficult to fix it.

[maximecb]

Thank you so much for looking into this and submitting a patch for QEMU @tatsuya6502 ! :)

[tatsuya6502]

OK. We made some progress. Here is a summary:

1. The other issue (Ruby got hang) was fixed after reinstalling Docker Desktop on a Mac.
2. We found no other issues.
3. We are preparing a patch for QEMU user space emulator (qemu-x86_64).
   • a. We will also submit a patch to YJIT not to call mprotect when len is 0.
4. To verify the YJIT patch would solve the issue, we ran yjit-bench on patched YJIT and it worked well:
   • We built two Docker images with patched YJIT Linux x86_64 and AArch64 respectively (using Linux machines, not Mac).
   • We ran yjit-bench on Docker on M1 Mac and they both worked well.
   • Due to the emulation, YJIT Linux x86_64 on Docker on M1 Mac is about eight times slower than YJIT Linux AArch64 on Docker on M1 Mac. (This is expected)

Here are some details:

1. We fixed the other issue (Ruby got hang) as well

We fixed the other issue (hangs). It turned out that one of our M1 Macs had an outdated Linux VM for Docker Desktop. We do not know how it got into such a state as we have regularly updated Docker Desktop app on the Mac. But we found its Linux kernel was one year old, so we guess qemu-x86_64 on the VM was old too and causing the issue? The issue was fixed after reinstalling Docker Desktop app on the Mac.

3-a. We will also submit a patch to YJIT

For YJIT and QEMU (qemu-x86_64), we only have to deal with mprotect system call with len 0.

While we are submitting the patch to the QEMU project, we will also submit a patch (GitHub pull request) for YJIT at ruby/ruby repository. Here is why: We reviewed the spec of mprotect and found it is unclear what mprotect should do when len 0 is given. It seems it is OS dependent; Linux does nothing and returns 0, but a man page of old Solaris says it returns -INVAL. So for the maximum compatibility between different OSes, we will propose a change to YJIT not to call mprotect when len is 0.

We applied above change to YJIT on our local checkout (on a Linux x86_64 machine) and built a Docker image with YJIT x86_64. We confirmed that the image works on our M1 Macs.

4. We ran yjit-banch on patched YJIT

We ran yjit-bench using the Docker image on Docker Desktop on one of our M1 Macs. It worked well except it was slow due to the emulation.

Total time spent benchmarking: 12979s
​
end_time: 2022-10-04 16:31:19 UTC (+0000)
interp: ruby 3.2.0dev (2022-10-03T21:22:50Z master b2668248b6) [x86_64-linux]
yjit: ruby 3.2.0dev (2022-10-03T21:22:50Z master b2668248b6) +YJIT [x86_64-linux]
​
-------------  -----------  ----------  ---------  ----------  -----------  ------------
bench          interp (ms)  stddev (%)  yjit (ms)  stddev (%)  interp/yjit  yjit 1st itr
30k_ifelse     5482.9       0.2         6332.9     0.8         0.87         0.41
30k_methods    13446.8      0.1         8075.3     0.2         1.67         1.29
activerecord   813.5        0.8         702.0      0.9         1.16         0.93
binarytrees    3626.3       0.3         1797.5     0.5         2.02         1.95
cfunc_itself   910.3        0.4         315.7      0.7         2.88         2.83
chunky_png     8705.1       0.2         5675.1     0.2         1.53         1.50
erubi          2475.3       0.3         2063.8     0.4         1.20         1.18
erubi_rails    155.6        0.8         134.6      1.4         1.16         0.43
fannkuchredux  43599.2      0.1         20755.2    0.1         2.10         1.00
fib            1964.4       0.4         332.6      0.5         5.91         5.80
getivar        843.6        0.5         204.4      0.2         4.13         1.01
hexapdf        18724.9      0.6         14499.1    1.2         1.29         1.16
keyword_args   2497.0       0.2         334.8      0.7         7.46         7.37
lee            9805.0       0.2         7275.7     0.3         1.35         1.34
liquid-render  1241.2       0.5         858.4      1.0         1.45         1.20
mail           1241.2       0.3         1147.0     0.3         1.08         0.78
nbody          1290.9       0.3         649.5      0.2         1.99         1.92
optcarrot      47447.1      0.7         22512.9    0.6         2.11         2.05
psych-load     17517.0      0.1         13167.0    0.2         1.33         1.32
railsbench     13889.0      0.5         14022.7    0.5         0.99         0.90
respond_to     2704.5       0.3         204.3      0.8         13.24        12.91
rubykon        84292.0      0.4         49645.2    0.3         1.70         1.68
setivar        582.8        0.4         289.4      0.6         2.01         1.01
str_concat     686.1        0.6         356.7      0.9         1.92         1.84
-------------  -----------  ----------  ---------  ----------  -----------  ------------

For a performance comparison, we built another Docker image with YJIT AArch64 and ran yjit-bench on it. It took only about one-eighth of the time on the x86_64 image.

Total time spent benchmarking: 1641s
​
end_time: 2022-10-04 18:11:33 UTC (+0000)
interp: ruby 3.2.0dev (2022-10-03T21:22:50Z master b2668248b6) [aarch64-linux]
yjit: ruby 3.2.0dev (2022-10-03T21:22:50Z master b2668248b6) +YJIT [aarch64-linux]
​
-------------  -----------  ----------  ---------  ----------  -----------  ------------
bench          interp (ms)  stddev (%)  yjit (ms)  stddev (%)  interp/yjit  yjit 1st itr
30k_ifelse     1866.5       0.9         496.7      0.4         3.76         1.48
30k_methods    4448.5       0.4         1269.6     0.3         3.50         2.93
activerecord   80.8         1.2         47.8       1.1         1.69         1.19
binarytrees    296.0        0.9         132.9      0.3         2.23         2.12
cfunc_itself   58.2         0.9         22.1       0.9         2.63         2.52
chunky_png     639.8        0.6         372.4      0.2         1.72         1.67
erubi          172.6        1.3         149.4      1.4         1.15         1.15
erubi_rails    12.6         3.4         8.8        4.2         1.43         0.42
fannkuchredux  5364.6       0.2         1516.0     0.1         3.54         0.99
fib            226.8        0.6         39.8       0.3         5.70         5.64
getivar        114.3        0.9         35.6       1.0         3.21         0.98
hexapdf        1607.7       1.1         1136.6     2.6         1.41         1.25
keyword_args   196.9        0.7         32.6       0.3         6.03         5.98
lee            811.0        0.8         494.5      1.4         1.64         1.63
liquid-render  94.7         1.1         61.3       1.3         1.54         1.17
mail           91.1         1.1         79.1       0.6         1.15         0.74
nbody          86.5         0.9         46.3       0.4         1.87         1.79
optcarrot      4473.8       0.8         2372.0     0.7         1.89         1.84
psych-load     1434.5       0.1         1002.2     0.1         1.43         1.41
railsbench     1355.1       0.7         1040.7     1.3         1.30         1.15
respond_to     153.2        0.2         18.6       0.5         8.26         8.07
rubykon        7633.9       0.5         3922.8     0.3         1.95         2.00
setivar        65.5         0.6         26.6       0.4         2.46         1.01
str_concat     70.1         0.8         23.1       1.2         3.03         2.44
-------------  -----------  ----------  ---------  ----------  -----------  ------------

[maximecb]

Thanks a lot @tatsuya6502 !

[tatsuya6502]

@maximecb — I submitted a pull request: ruby#6563. Please review when you have time. Thanks!