Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
read_fragment_table_n: Fix recommendation for stack overflow. #5
There seems to be a stack overflow and an integer overflow in read_fragment_table_4.
The first problem overflows the bytes variable, so that the allocation of fragments_bytes has an erroneous size.
int bytes = SQUASHFS_FRAGMENT_BYTES(sBlk.s.fragments); ... fragment_table = malloc(bytes);
If we fix this by making the variable size_t, we run into an unrelated problem in which the stack VLA allocation of fragment_table_index can easily exceed RLIMIT_STACK.
long long fragment_table_index[indexes];
In the case of my system, the RLIMIT_STACK is 8388608 bytes, and VLA is asking for 15728648 bytes. This is what I believe ultimately leads to the stack overflow. Plus the stack probably already has a bunch of other things, and I don't know a portable way to check how much stack space is available.
Afterwards, the disastrous call to read_fs_bytes is made, which initiates the transfer from the squashfs image to the stack. At this stage, a stack overflow appears to be in full effect.
res = read_fs_bytes(fd, sBlk.s.fragment_table_start, SQUASHFS_FRAGMENT_INDEX_BYTES(sBlk.s.fragments), fragment_table_index);
Perhaps we should avoid using VLA altogether, and allocate fragment_table_index to the heap? It's likely that the fragment_table_index can be a lot bigger than 8388608 bytes, according to the squashfs specification.
This problem is present in other read_fragment_table_N functions, and in in the original squashfs-tools.
This pull request is an example implementation of the recommended fix for unsquash-4, but I don't have enough test vectors to verify it doesn't break anything. All code that uses VLA should probably be converted to use the heap instead.
VLA Usages (In squashfs-tools)
Please let me know what you think.