You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Given version 1.3.1, and a file of size between 100 and 799 bytes, with a cachePercent of 1.
When running
RandomAccessFile raf = new RandomAccessFile(new File("file799bytes.txt"), "r");
new FileWordList(raf, true, 1);
Then
java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:521)
at java.nio.ByteBufferAsLongBufferB.put(ByteBufferAsLongBufferB.java:128)
at org.passay.dictionary.AbstractFileWordList$Cache.put(AbstractFileWordList.java:386)
at org.passay.dictionary.AbstractFileWordList.initialize(AbstractFileWordList.java:139)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:138)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:109)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:83)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:62)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:43)
Analysis:
For small files that use a small cache, the cache buffer that is generated can be too small. This is due to how the cache is generated. In the constructor of cache the cacheSize is computed as:
final long cacheSize = (fileSize / 100) * cachePercent;
If this cacheSize is larger than 0, then used in resize(..) to create a ByteBuffer using:
ByteBuffer.allocate((int) size)
This ByteBuffer is then converted to a longBuffer using:
ByteBuffer.allocate((int) size).asLongBuffer()
Assume that the ByteBuffer is generated (cacheSize is larger than 0), then replacing the cacheSize formula in the ByteBuffer allocation, you get this:
The error happens when the ByteBuffer generated by allocate is less than 8 bytes, as asLongBuffer will then convert it to a long buffer of size 0 (allocateSize divided by 8, integer). When trying to putting values in an empty buffer, the BufferOverflowException is thrown.
Edit: It can also fail when the buffer is resized when calling cache.put(..).
A workaround is too disable the cache (cachePercent 0), use a cachePercent of at least 8, or not use small files. A fix would probably be to check for a too small cacheSize (< 8 ), and allocate at least 8 bytes in that case.
The text was updated successfully, but these errors were encountered:
For a cache to operate properly it must be at least two bytes in size.
That allows for storage of one position and incrementing to the next position before a resize occurs.
Fixes#72.
Given version 1.3.1, and a file of size between 100 and 799 bytes, with a cachePercent of 1.
When running
Then
Analysis:
For small files that use a small cache, the cache buffer that is generated can be too small. This is due to how the cache is generated. In the constructor of cache the cacheSize is computed as:
final long cacheSize = (fileSize / 100) * cachePercent;
If this cacheSize is larger than 0, then used in
resize(..)
to create a ByteBuffer using:ByteBuffer.allocate((int) size)
This ByteBuffer is then converted to a longBuffer using:
ByteBuffer.allocate((int) size).asLongBuffer()
Assume that the ByteBuffer is generated (cacheSize is larger than 0), then replacing the cacheSize formula in the ByteBuffer allocation, you get this:
ByteBuffer.allocate((int) (fileSize / 100) * cachePercent).asLongBuffer()
Assume cachePercent is 1:
ByteBuffer.allocate((int) (fileSize / 100) * 1).asLongBuffer()
ByteBuffer.allocate((int) (fileSize / 100)).asLongBuffer()
The error happens when the ByteBuffer generated by
allocate
is less than 8 bytes, asasLongBuffer
will then convert it to a long buffer of size 0 (allocateSize divided by 8, integer). When trying to putting values in an empty buffer, the BufferOverflowException is thrown.For example, 100 bytes:
ByteBuffer.allocate((int) (100 / 100) * 1).asLongBuffer()
ByteBuffer.allocate(1).asLongBuffer()
1 size ByteBuffer -> asLongBuffer -> 0 size LongBuffer
For example, 799 bytes:
ByteBuffer.allocate((int) (799 / 100) * 1).asLongBuffer()
ByteBuffer.allocate(7).asLongBuffer()
7 size ByteBuffer -> asLongBuffer -> 0 size LongBuffer
No problem, 800 or more bytes:
ByteBuffer.allocate((int) (800 / 100) * 1).asLongBuffer()
ByteBuffer.allocate(8).asLongBuffer()
8 size ByteBuffer -> asLongBuffer -> 1 size LongBuffer
When using cachePercent of 2, it goes wrong up until 399 bytes:
ByteBuffer.allocate((int) (399 / 100) * 2).asLongBuffer()
ByteBuffer.allocate(3 * 2).asLongBuffer()
6 size ByteBuffer -> asLongBuffer -> 0 size LongBuffer
Errors (anything less than 8)
Edit: It can also fail when the buffer is resized when calling
cache.put(..)
.A workaround is too disable the cache (cachePercent 0), use a cachePercent of at least 8, or not use small files. A fix would probably be to check for a too small cacheSize (< 8 ), and allocate at least 8 bytes in that case.
The text was updated successfully, but these errors were encountered: