Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
FDK bit buffer causes access violations #61
The FDK bit buffer implementation is broken in that it regularly reads past the provided buffer - seemingly as a performance optimization. This causes access violation errors if the bytes directly following the buffer belong to a protected page.
You can see this in action in Valgrind where you get lots of invalid access errors when decoding an AAC stream with libfdk: valgrind.txt
On Windows I get an access violation about once every 4000 files when converting AAC. Here's a dump I got using Dr. Mingw: fdk-aac-dump.txt
The code that causes the bug is basically:
I'm working around this bug in my software by copying the bytes returned from MP4ReadSample to a new buffer that is 4 bytes larger than the original one, thereby making sure FDK_get will not access protected pages:
It would be great if this could be fixed in libfdk!
It does not seem to be easy, however. The bit buffer would need to know the actual buffer size, but transportDec_FillData always initializes it with 65536 instead of passing bufferSize, because "the FDK bit buffer implementation needs a number 2^x" which seems quite obscure to me.
Thanks for reporting this issue!
The comment about the bit buffer implementation needing a power of two number of bytes sounds very strange to me as well. On a very brief look through
Allocating a slightly large buffer for the input might seem odd at first, but in practice, this would not be the only library with such a restriction. For example, libavcodec also requires input buffers to have extra zero padding at the end - see e.g. https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/avcodec.h;h=1c58fe2d689c1c3daca248208c1e2154737e7568;hb=10f4511f14a4e830c0ed471df4cd1cc2a18a481a#l3734.
Ideally, the library could of course be modified to not overread, but if that doesn't turn out to be straightforward, this issue could just be documented instead.
Thank you for your quick response! After looking a little deeper into it, I opt for documenting this behaviour instead of a real fix.
At first I thought replacing
in FDK_get would suffice, but FDK_bitstream.h is doing its own caching to do things like synchronizing and aligning to byte boundaries on top of FDK_BitBuffer and thus still reads behind the buffer.
So, fixing this would probably be a major undertaking and just documenting it properly might be the better way to go.
I guess the main question, which I can try to look into using valgrind at some point, is - is it enough to add 4 bytes of zero initialized padding, or would it still overread more than that? (If the internal apis actually don't take the size parameter into account?) That is, if you take a real AAC packet, truncate it and copy to a new allocation of that exact size + 4 bytes zeros, would it still overread past those extra padding bytes, or would that still work (failing decoding cleanly)?
FWIW, I tested decoding with truncated packets, and in that case, having allocated the input buffer 4 bytes larger than the truncated data is not enough - it can still overread in those cases. So just allocating a larger buffer doesn't seem to be enough to be safe, in case the input data is broken somehow.
Not sure how large undertaking it is to fix it properly though...
The issue with truncated packets is easy to fix. Just add a guard to FDK_get (and FDK_get32) in FDK_bitbuffer.cpp:
and change the subtraction of hBitBuf->ValidBits so it doesn't wrap around (alternatively, make ValidBits signed):
and in FDK_get32:
The +31 in the first line is needed because of the caching in FDK_bitstream.h, so this fixes issues with truncated packets, but not the original 4 bytes overflow problem.