[Bug] Heap-use-after-free in mrb_vm_exec involving mruby-rational / mruby-bigint

[oneafter]

Description

We discovered a Heap-use-after-free vulnerability in mruby. The crash occurs within the main VM loop (mrb_vm_exec) immediately after a rational number reduction operation.

The ASAN report indicates a READ memory access violation on a freed memory region. The region was freed during a GCD calculation inside mrb_bint_reduce, but is subsequently accessed by the VM interpreter.

Environment

• OS: Linux x86_64
• Complier: Clang
• Build Configuration: Release mode with ASan enabled.

Vulnerability Details

• Target: mruby
• Vulnerability Type: Heap-use-after-free (READ)
• Function: mrb_vm_exec
• Location: src/vm.c:1953
• Root Cause Analysis: The crash sequence involves the interaction between the VM and the rational number extension:

1. The VM executes code that likely creates a Rational number from BigInts.
2. The call stack rational_new_b -> mrb_bint_reduce -> mpz_gcd indicates that the runtime is reducing the fraction (calculating the Greatest Common Divisor).
3. During this reduction, mruby-bigint frees some temporary memory or updates the BigInt structure (free called via mrb_basic_alloc_func).
4. However, mrb_vm_exec (at stack frame #-0) seems to hold a dangling pointer to this memory and attempts to read 4 bytes from it after the return of the rational operation.

Reproduce

1. Build mruby with Release optimization and ASAN enabled.
2. Run with the crashing file:

poc

3-24/3r**433-2244r**-433;RR=>^ı0ııı;ıÙıRR=>^˙˙ıı;ı˙˙o3-433-2243-33-24/3r**433-2244r**-433;33-4244r**33-U44

./bin/mruby poc

ASAN report

==19204==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000000168 at pc 0x564adebbbf3e bp 0x7ffc60d2d090 sp 0x7ffc60d2d088
READ of size 4 at 0x503000000168 thread T0
    #0 0x564adebbbf3d in mrb_vm_exec /src/mruby/src/vm.c:1953:43
    #1 0x564adebeab09 in mrb_load_exec /src/mruby/mrbgems/mruby-compiler/core/parse.y:7786:7
    #2 0x564adeaecf36 in main /src/mruby/harness.c:37:28
    #3 0x7f73e5bc21c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #4 0x7f73e5bc228a in __libc_start_main csu/../csu/libc-start.c:360:3
    #5 0x564adea0b884 in _start (/src/mruby/harness+0xb8884) (BuildId: 962df40fb0c1af68f183b395b4c7a9dbe4b192b9)

0x503000000168 is located 8 bytes inside of 20-byte region [0x503000000160,0x503000000174)
freed by thread T0 here:
    #0 0x564adeaab41a in free (/src/mruby/harness+0x15841a) (BuildId: 962df40fb0c1af68f183b395b4c7a9dbe4b192b9)
    #1 0x564adecaeca2 in mrb_basic_alloc_func /src/mruby/src/allocf.c:30:5
    #2 0x564adec94d00 in mpz_gcd /src/mruby/mrbgems/mruby-bigint/core/bigint.c:2700:5
    #3 0x564adec9399e in mrb_bint_reduce /src/mruby/mrbgems/mruby-bigint/core/bigint.c:3837:3
    #4 0x564adec523ba in rational_new_b /src/mruby/mrbgems/mruby-rational/src/rational.c:265:3
    #5 0x564adeb9af11 in mrb_vm_exec /src/mruby/src/vm.c
    #6 0x564adebeab09 in mrb_load_exec /src/mruby/mrbgems/mruby-compiler/core/parse.y:7786:7
    #7 0x564adeaecf36 in main /src/mruby/harness.c:37:28

previously allocated by thread T0 here:
    #0 0x564adeaabad0 in realloc (/src/mruby/harness+0x158ad0) (BuildId: 962df40fb0c1af68f183b395b4c7a9dbe4b192b9)
    #1 0x564adeaf721d in mrb_realloc_simple /src/mruby/src/gc.c:202:8
    #2 0x564adeaf721d in mrb_realloc /src/mruby/src/gc.c:216:8
    #3 0x564adeaf721d in mrb_malloc /src/mruby/src/gc.c:232:10
    #4 0x564adec8ddb3 in mpz_mul_2exp /src/mruby/mrbgems/mruby-bigint/core/bigint.c:2098:7
    #5 0x564adeca882b in mpz_barrett_mu /src/mruby/mrbgems/mruby-bigint/core/bigint.c:2736:3

SUMMARY: AddressSanitizer: heap-use-after-free /src/mruby/src/vm.c:1953:43 in mrb_vm_exec
Shadow bytes around the buggy address:
  0x502ffffffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x502fffffff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x502fffffff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x503000000000: fa fa 00 00 00 fa fa fa 00 00 00 fa fa fa 00 00
  0x503000000080: 00 fa fa fa 00 00 00 fa fa fa 00 00 00 00 fa fa
=>0x503000000100: fd fd fd fa fa fa fd fd fd fa fa fa fd[fd]fd fa
  0x503000000180: fa fa fd fd fd fa fa fa fa fa fa fa fa fa fa fa
  0x503000000200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x503000000280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x503000000300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x503000000380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==19204==ABORTING