fs/rpmsgfs: fix potential integer overflow in readdir handler #18339
+16
−6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The rpmsg_get_tx_buffer_size() and rpmsg_get_rx_buffer_size() may return negative error codes (e.g., -EPERM which is -2003). When storing the MIN() result in an unsigned size_t variable, negative values overflow to large positive values, causing incorrect buffer size calculations. This fix: - Change the size variable type from size_t to ssize_t to properly handle negative return values. - Add explicit error checking for negative buffer sizes before use. - Add boundary check to ensure size >= len before string operations. Signed-off-by: Yongrong Wang <wangyongrong@xiaomi.com>
github-actions
bot
added
Area: File System
CV-Bowen
approved these changes
Feb 3, 2026
simbit18
requested review from
acassis,
anchao,
linguini1 and
xiaoxiang781216
linguini1
approved these changes
Feb 3, 2026
Contributor
linguini1
left a comment
linguini1
left a comment
There was a problem hiding this comment.
Looks good! In the future, can you please put the logs inside a markdown code block? It makes it much easier to read.
jerpelea
approved these changes
Feb 3, 2026
acassis
approved these changes
Feb 3, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The rpmsg_get_tx_buffer_size() and rpmsg_get_rx_buffer_size() may return negative error codes (e.g., -EPERM which is -2003). When storing the MIN() result in an unsigned size_t variable, negative values overflow to large positive values, causing incorrect buffer size calculations.
Summary
Fix integer overflow bug in rpmsgfs_readdir_handler() when rpmsg buffer size functions return error codes. The
rpmsg_get_tx_buffer_size()andrpmsg_get_rx_buffer_size()APIs return ssize_t and may return negative error codes like -EPERM (-2003) when the endpoint is not ready. Previously, the result was stored in an unsignedsize_tvariable, causing negative values to overflow into very large positive numbers. This led to incorrect buffer size calculations and potential memory issues.This fix:
Change the size variable type from size_t to ssize_t to properly handle negative return values.
Add explicit error checking for negative buffer sizes before use.
Add boundary check to ensure size >= len before string operations.
Impact
Impact on security : fixes potential buffer overflow due to incorrect size calculation
Testing
Testing log in QEMU ARMV8:
qemu-system-aarch64 -cpu cortex-a53 -nographic \ wyr@hp 20:13:29
-machine virt,virtualization=on,gic-version=3
-chardev stdio,id=con,mux=on -serial chardev:con
-object memory-backend-file,discard-data=on,id=shmmem-shmem0,mem-path=/dev/shm/my_shmem0,size=4194304,share=yes
-device ivshmem-plain,id=shmem0,memdev=shmmem-shmem0,addr=0xb
-device virtio-serial-device,bus=virtio-mmio-bus.0
-chardev socket,path=/tmp/rpmsg_port_uart_socket,server=on,wait=off,id=foo
-device virtconsole,chardev=foo
-mon chardev=con,mode=readline -kernel ./nuttx/cmake_out/v8a_server/nuttx
-gdb tcp::7775
[ 0.000000] [ 0] [ INFO] [server] pci_register_rptun_ivshmem_driver: Register ivshmem driver, id=0, cpuname=proxy, master=1
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: pci_scan_bus for bus 0
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: class = 00000600, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: 00:00 [1b36:0008]
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar0 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: class = 00000200, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: 00:08 [1af4:1000]
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar0: mask64=fffffffe 32bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar1: mask64=fffffff0 4096bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar4: mask64=fffffffffffffff0 16384bytes
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: class = 00000500, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [server] pci_scan_bus: 00:58 [1af4:1110]
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar0: mask64=fffffff0 256bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar2: mask64=fffffffffffffff0 4194304bytes
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [server] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [server] ivshmem_probe: shmem addr=0x8000400000 size=4194304 reg=0x10001000
[ 0.000000] [ 3] [ INFO] [server] rptun_ivshmem_probe: shmem addr=0x8000400000 size=4194304
NuttShell (NSH)
server>
server>
server>
server> [ 0.000000] [ 0] [ INFO] [proxy] pci_register_rptun_ivshmem_driver: Register ivshmem driver, id=0, cpuname=server, master=0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: pci_scan_bus for bus 0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000600, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:00 [1b36:0008]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000200, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:08 [1af4:1000]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0: mask64=fffffffe 32bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1: mask64=fffffff0 4096bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4: mask64=fffffffffffffff0 16384bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000500, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:58 [1af4:1110]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0: mask64=fffffff0 256bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2: mask64=fffffffffffffff0 4194304bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] ivshmem_probe: shmem addr=0x8000400000 size=4194304 reg=0x10001000
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: shmem addr=0x8000400000 size=4194304
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: Start the wdog
server> ls /dev/rpmsg
/dev/rpmsg:
proxy
proxy2
server> cat /proc/version
NuttX version 0.0.0 8c5f0e5-dirty Feb 3 2026 20:12:48 qemu-armv8a/rpserver
server>
qemu-system-aarch64 -cpu cortex-a53 -nographic \ wyr@hp 20:13:27
-machine virt,virtualization=on,gic-version=3
-chardev stdio,id=con,mux=on -serial chardev:con
-object memory-backend-file,discard-data=on,id=shmmem-shmem0,mem-path=/dev/shm/my_shmem0,size=4194304,share=yes
-device ivshmem-plain,id=shmem0,memdev=shmmem-shmem0,addr=0xb
-device virtio-serial-device,bus=virtio-mmio-bus.0
-chardev socket,path=/tmp/rpmsg_port_uart_socket,server=off,id=foo
-device virtconsole,chardev=foo
-mon chardev=con,mode=readline -kernel ./nuttx/cmake_out/v8a_proxy/nuttx
-gdb tcp::7776
[ 0.000000] [ 0] [ INFO] [proxy] pci_register_rptun_ivshmem_driver: Register ivshmem driver, id=0, cpuname=server, master=0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: pci_scan_bus for bus 0
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000600, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:00 [1b36:0008]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000200, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:08 [1af4:1000]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0: mask64=fffffffe 32bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1: mask64=fffffff0 4096bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar3 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4: mask64=fffffffffffffff0 16384bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: class = 00000500, hdr_type = 00000000
[ 0.000000] [ 3] [ INFO] [proxy] pci_scan_bus: 00:58 [1af4:1110]
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar0: mask64=fffffff0 256bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar1 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar2: mask64=fffffffffffffff0 4194304bytes
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar4 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] pci_setup_device: pbar5 set bad mask
[ 0.000000] [ 3] [ INFO] [proxy] ivshmem_probe: shmem addr=0x8000400000 size=4194304 reg=0x10001000
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: shmem addr=0x8000400000 size=4194304
[ 0.000000] [ 3] [ INFO] [proxy] rptun_ivshmem_probe: Start the wdog
NuttShell (NSH)
proxy>
proxy> ls /dev/rpmsg
/dev/rpmsg:
server
server2
proxy>
proxy> mount -t rpmsgfs -o cpu=server,fs=/proc /proc_server
proxy> cat /proc_server/version
NuttX version 0.0.0 8c5f0e5-dirty Feb 3 2026 20:12:48 qemu-armv8a/rpserver
proxy>