[compiler-rt][sanitizer_common] Increase min user-map/freearray round…

[PiJoules]

…ing size

About 90% of the time when running scudo tests on fuchsia+riscv with asan instrumentation, we will be unable to map the arena for scudo's primary allocator. We have enough space, but much of the existing allocations are fragmented enough such that we can't allocate that large enough contiguous space. This reduces fragmentation more by increase min allocation rounding for user-map and freearray in the primary allocator.

Note that changing these values doesn't have a functional effect on the actual size of the user-map or freearray portions of a region. Those are toggled by different constants. This will reduce the number of times mmap is called and reduce the number of fragments.

Locally, I've been able to run the scudo tests under asan-instrumentation on riscv 39-bit vma and haven't run into the SCUDO ERROR: zx_vmar_map error.

[llvmbot]

@llvm/pr-subscribers-compiler-rt-sanitizer

Author: None (PiJoules)

Changes

…ing size

About 90% of the time when running scudo tests on fuchsia+riscv with asan instrumentation, we will be unable to map the arena for scudo's primary allocator. We have enough space, but much of the existing allocations are fragmented enough such that we can't allocate that large enough contiguous space. This reduces fragmentation more by increase min allocation rounding for user-map and freearray in the primary allocator.

Note that changing these values doesn't have a functional effect on the actual size of the user-map or freearray portions of a region. Those are toggled by different constants. This will reduce the number of times mmap is called and reduce the number of fragments.

Locally, I've been able to run the scudo tests under asan-instrumentation on riscv 39-bit vma and haven't run into the SCUDO ERROR: zx_vmar_map error.

---

Full diff: https://github.com/llvm/llvm-project/pull/73600.diff

1 Files Affected:

• (modified) compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h (+2-2)

diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h b/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h
index d77bc05b780203b..34a64f26478fd5e 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h
@@ -641,11 +641,11 @@ class SizeClassAllocator64 {
   // kRegionSize must be <= 2^36, see CompactPtrT.
   COMPILER_CHECK((kRegionSize) <= (1ULL << (SANITIZER_WORDSIZE / 2 + 4)));
   // Call mmap for user memory with at least this size.
-  static const uptr kUserMapSize = 1 << 16;
+  static const uptr kUserMapSize = 1 << 18;
   // Call mmap for metadata memory with at least this size.
   static const uptr kMetaMapSize = 1 << 16;
   // Call mmap for free array memory with at least this size.
-  static const uptr kFreeArrayMapSize = 1 << 16;
+  static const uptr kFreeArrayMapSize = 1 << 18;
 
   atomic_sint32_t release_to_os_interval_ms_;
 

[PiJoules]

I also haven't seen any significant time differences when running tests.

[vitalybuka]

This has nothing to do with SCUDO

[vitalybuka]

I am confused with description, asan does not use SCUDO allocator.

[vitalybuka]

I guess you just run some subset of scudo tests with asan.

This change looks safe for the rest

[PiJoules]

I guess you just run some subset of scudo tests with asan.

This change looks safe for the rest

Yeah, sorry I could've described this better. This doesn't have anything to do with scudo. This just helps run asan-instrumented scudo tests on 39-bit vma riscv. The order of operations is:

1. We load a single binary that has all the tests in compiler-rt/lib/scudo/standalone/tests/. This binary is asan-instrumented.
2. Before entering main, we setup asan and its allocator.
3. We enter main and run some scudo tests.
4. We reach SCUDO_TYPED_TEST(ScudoCombinedTest, IsOwned) which creates the scudo allocator for different target configs.
5. Scudo's primary allocator attempts to mmap an arena, but fails with SCUDO ERROR: zx_vmar_allocate failed with size 11796480KB (ZX_ERR_NO_RESOURCES). This happens because much of the 39-bit vma is already occupied, mainly by the asan-shadow, asan's primary allocator, and individual mmaps done by asan. There's definitely enough space for scudo to do it's allocation, but there's just not enough contiguous space and reducing the number of freearray and user-map fragments helps.

Normally, we wouldn't have any programs that run both scudo and asan as the allocators. It just so happens that this particular one tests the allocator while the "default" one is asan's in a small vma.