bugfix: BufferOverflow when BranchSession size too large

[finalcola]

Ⅰ. Describe what this PR did

fix BufferOverflowException when FileTransactionStoreManager write a big BranchSession(size > 16KB).

Ⅱ. Does this pull request fix one issue?

fixes #1551

#1551 (comment)

Ⅲ. Why don't you add test cases (unit test/integration test)?

public class FileTransactionStoreManagerTest {

    FileTransactionStoreManager fileTransactionStoreManager;

    @BeforeEach
    public void beforeAll() throws IOException {
        fileTransactionStoreManager = new FileTransactionStoreManager("~/my/tmp/test/seata", null);
    }

    @Test
    public void writeBigBytes() {
        BranchSession branchSession = new BranchSession();
        branchSession.setApplicationData(createBigStr(1024 * 16 - 36));
        fileTransactionStoreManager.writeSession(TransactionStoreManager.LogOperation.BRANCH_ADD, branchSession);
    }

    public String createBigStr(int size){
        byte[] bytes = new byte[size];
        for (int i = 0; i < size; i++) {
            bytes[i] = 'a';
        }
        return new String(bytes);
    }
}

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

[codecov-io]

Codecov Report

Merging #1552 into develop will decrease coverage by 0.03%.
The diff coverage is 0%.

@@              Coverage Diff              @@
##             develop    #1552      +/-   ##
=============================================
- Coverage      46.44%   46.41%   -0.04%     
  Complexity      1717     1717              
=============================================
  Files            350      350              
  Lines          12847    12847              
  Branches        1618     1618              
=============================================
- Hits            5967     5963       -4     
- Misses          6232     6234       +2     
- Partials         648      650       +2

Impacted Files	Coverage Δ	Complexity Δ
...server/store/file/FileTransactionStoreManager.java	45.99% <0%> (-1.4%)	19 <1> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e191caa...bfcabf4. Read the comment docs.

[slievrly]

if (bs.length + 4 )> int.max, how to process?

[finalcola]

ByteBuffer byteBuffer = null;

        if (bs.length > MAX_WRITE_BUFFER_SIZE) {
            //allocateNew
            byteBuffer = ByteBuffer.allocateDirect(bs.length);
        } else {
            byteBuffer = writeBuffer;
            //recycle
            byteBuffer.clear();
        }
        byteBuffer.putInt(bs.length);
        byteBuffer.put(bs);
        return writeDataFileByBuffer(byteBuffer);

it's origin code.
when execute :

byteBuffer.putInt(bs.length);
byteBuffer.put(bs);

bytebuffer will write extra 4 byte content (bs.length), but when allocate DirectByteBuffer, the length of the DirectByteBuffer is bs.length ( should be bs.length +4 )

[slievrly]

@finalcola public static ByteBuffer allocateDirect(int capacity) accept a int param, if bs.len<int.max but bs.len+4>int.max , it need to judge?

[finalcola]

MAX_WRITE_BUFFER_SIZE is 16KB not int.max.
when bs.length>MAX_WRITE_BUFFER_SIZE, there will allocate a new ByteBuffer(size = bs.length).
but we put bs.length(int) and bs content into this ByteBuffer, so it will oversize 4B (bs.length) and cause BufferOverflowException.
@slievrly

[slievrly]

@finalcola if (bs.length=int.max-1 ) ，can it run normally?

[finalcola]

it can't , because byteBuffer always write extra 4 byte content（Larger than the allocated buffer）.
@slievrly

[slievrly]

it can't , because byteBuffer always write extra 4 byte content（Larger than the allocated buffer）.
@slievrly

So do we need to add code to judge bs.length over int.max?

[finalcola]

no,it's already judged before enter this method.

[slievrly]

no,it's already judged before enter this method.

1. private boolean writeDataFile(byte[] bs)
2. call writeDataFile method with param byte[] bs.(int.max-3<bs.length<int.max)

[finalcola]

so we should judge bs.length<=int.max-4?

[slievrly]

LGTM

[xingfudeshi]

LGTM.