Heap buffer overflow in RenderingServer::mesh_surface_get_lods

[nikitalita]

Tested versions

current master @ b00e1cb

System information

macOS 14.5.0, M2 Pro, Vulkan

Issue description

Related: #98884 , #97862

CC @clayjohn @BlueCube3310

The culprit this time is a 1024x1024 RGTC RG8 normal map, except this time, it seems to decompress normally when testing:

*** Loading texture res://ShakerNormal_decompressed.png
Loading resource: res://ShakerNormal_decompressed.png
*** Decompressing the image
bcdec: Decompressing mipmap 0: 1024x1024, src_ofs: 0, dst_ofs: 0, target_size: 2796202
bcdec: Decompressing mipmap 1: 512x512, src_ofs: 1048576, dst_ofs: 2097152, target_size: 2796202
bcdec: Decompressing mipmap 2: 256x256, src_ofs: 1310720, dst_ofs: 2621440, target_size: 2796202
bcdec: Decompressing mipmap 3: 128x128, src_ofs: 1376256, dst_ofs: 2752512, target_size: 2796202
bcdec: Decompressing mipmap 4: 64x64, src_ofs: 1392640, dst_ofs: 2785280, target_size: 2796202
bcdec: Decompressing mipmap 5: 32x32, src_ofs: 1396736, dst_ofs: 2793472, target_size: 2796202
bcdec: Decompressing mipmap 6: 16x16, src_ofs: 1397760, dst_ofs: 2795520, target_size: 2796202
bcdec: Decompressing mipmap 7: 8x8, src_ofs: 1398016, dst_ofs: 2796032, target_size: 2796202
bcdec: Decompressing mipmap 8: 4x4, src_ofs: 1398080, dst_ofs: 2796160, target_size: 2796202
bcdec: Decompressing mipmap 9: 2x2, src_ofs: 1398096, dst_ofs: 2796192, target_size: 2796202
bcdec: Decompressing mipmap 10: 1x1, src_ofs: 1398112, dst_ofs: 2796200, target_size: 2796202
bcdec: Decompression of a 1024x1024 RGTC RedGreen8 image with 10 mipmaps took 13 ms.
*** Image decompressed
*** Saving the image to res://ShakerNormal_decompressed-test.png

HOWEVER, when the texture is attached to a mesh as a surface material, and we attempt to save the scene as a GLB, the following occurs:

*** Loading the scene res://Shaker_test.tscn
Loading resource: res://Shaker_test.tscn
Loading resource: res://ShakerNormal_decompressed.png
*** Dependencies for res://Shaker_test.tscn:
uid://8eeijsot8t6::::res://ShakerNormal_decompressed.png
*** Instancing the scene
*** Appending the scene to the GLTF document
glTF: Converting light: Light
glTF: Converting camera: Camera
Copying sd.lods[0].index_data, size 98304, lc 49152, rptr[0] @ index_data + 0, past the end? false
Copying sd.lods[1].index_data, size 49152, lc 24576, rptr[1] @ index_data + 2, past the end? false
Copying sd.lods[2].index_data, size 24564, lc 12282, rptr[2] @ index_data + 4, past the end? false
Copying sd.lods[3].index_data, size 12288, lc 6144, rptr[3] @ index_data + 6, past the end? false
Copying sd.lods[4].index_data, size 6144, lc 3072, rptr[4] @ index_data + 8, past the end? false
Copying sd.lods[5].index_data, size 3072, lc 1536, rptr[5] @ index_data + 10, past the end? false
Copying sd.lods[6].index_data, size 1536, lc 768, rptr[6] @ index_data + 12, past the end? false
Copying sd.lods[7].index_data, size 768, lc 384, rptr[7] @ index_data + 14, past the end? false
Copying sd.lods[8].index_data, size 384, lc 192, rptr[8] @ index_data + 16, past the end? false
Copying sd.lods[9].index_data, size 192, lc 96, rptr[9] @ index_data + 18, past the end? false
Copying sd.lods[10].index_data, size 96, lc 48, rptr[10] @ index_data + 20, past the end? false
Copying sd.lods[11].index_data, size 48, lc 24, rptr[11] @ index_data + 22, past the end? false
Copying sd.lods[12].index_data, size 12, lc 6, rptr[12] @ index_data + 24, past the end? true

=================================================================
==39550==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x000159ea7d48 at pc 0x000115fc4960 bp 0x00016b0fb210 sp 0x00016b0fb208
READ of size 2 at 0x000159ea7d48 thread T0
    #0 0x115fc495c in RenderingServer::mesh_surface_get_lods(RID, int) const rendering_server.cpp:1724

The problem is here:

The uint8_t index_data.ptr() is cast to a uint16_t ptr rptr, and then it's read at rptr[i].
Thus, rptr[12] is at index_data + 24, and since sd.lods[12].index_data is only 12 bytes long, it reads past the bounds and causes the crash.

Full asan log:
thing.log

Steps to reproduce

1. Optionally, put the following in servers/rendering_server.cpp @ line 1722:

print_verbose(vformat("Copying sd.lods[%d].index_data, size %d, lc %d, rptr[%d] @ index_data + %d, past the end? %s", i, sd.lods[i].index_data.size(), lc, i, i * 2, i * 2 >= sd.lods[i].index_data.size() ? "true" : "false"));

2. Build the editor with sanitizers enabled
3. Extract the MRP somewhere.
4. Run the MRP with <editor_bin> --path <wherever_you_extracted_the_mrp_to> --verbose
5. Observe crash.

Minimal reproduction project (MRP)

bcdec-crash-mrp.zip

[nikitalita]

mesh-lod-crash-mrp.zip

Was able to trigger a crash (with sanitizers enabled) in a gameplay scenario by attaching a mesh with "bad" index_data to a soft body. It has nothing to do with bcdec or textures, this is a completely stand-alone mesh.