HA may not avaiable when slave's commitLog not match master

[qsrg]

version：4.3.2，4.4.0
When the machine of slave broke down for some days，start up slave after the machine repaired。On this condition，master cannot send commitLog data to slave，this cause SYNC_MASTER return SLAVE_NOT_AVAIABLE when send message to this broker。search in master's broker.log，can find ‘Slave fall behind master：xxx '。

After analysis，I find the slave's commitLog offset does not existed in master's commitLog ，this cause master getCommitLogData by a out of range offset return null，then just send heartbeat

some resource code as follow：

SelectMappedBufferResult selectResult =
                        HAConnection.this.haService.getDefaultMessageStore().getCommitLogData(this.nextTransferFromWhere);
                    if (selectResult != null) {
                        int size = selectResult.getSize();
                        if (size > HAConnection.this.haService.getDefaultMessageStore().getMessageStoreConfig().getHaTransferBatchSize()) {
                            size = HAConnection.this.haService.getDefaultMessageStore().getMessageStoreConfig().getHaTransferBatchSize();
                        }

                        long thisOffset = this.nextTransferFromWhere;
                        this.nextTransferFromWhere += size;

                        selectResult.getByteBuffer().limit(size);
                        this.selectMappedBufferResult = selectResult;

                        // Build Header
                        this.byteBufferHeader.position(0);
                        this.byteBufferHeader.limit(headerSize);
                        this.byteBufferHeader.putLong(thisOffset);
                        this.byteBufferHeader.putInt(size);
                        this.byteBufferHeader.flip();

                        this.lastWriteOver = this.transferData();
                    } else {
                        service.getWaitNotifyObject().wakeupAll();
                        HAConnection.this.haService.getWaitNotifyObject().allWaitForRunning(100);
                    }

I think when master received slaveRequestOffset，calculate of nextTransferFromWhere should check slaveRequestOffset is legal，not just

this.nextTransferFromWhere = HAConnection.this.slaveRequestOffset;

[RongtongJin]

In this case, it is difficult to correct the offset self, so the slave is usually rebuilt and the store directory of the slave should be deleted.

[qsrg]

Before start up the slave，check the commigLog data is correctly by manual is inconvenient to execute 。From the machine of slave broke down to it repaired，we did not modify the data in the store directory of slave，this mistake maybe due to the deletion policy as it'is not time to deleteExpiredFiles at start up time or other reasons. so add check is necessary.

add the following checks when master received slaveRequestOffset at first，if slaveRequestOffset>maxOffset or slaveRequestOffset<minOffset,send last commitLog file. And change slave's dispatchReadRequest method to handle a correct offset from master.
Is this idea appropriate?

if (-1 == this.nextTransferFromWhere) {
                        if (0 == HAConnection.this.slaveRequestOffset) {
                            long masterOffset = HAConnection.this.haService.getDefaultMessageStore().getCommitLog().getMaxOffset();
                            masterOffset =
                                masterOffset
                                    - (masterOffset % HAConnection.this.haService.getDefaultMessageStore().getMessageStoreConfig()
                                    .getMapedFileSizeCommitLog());

                            if (masterOffset < 0) {
                                masterOffset = 0;
                            }
                            this.nextTransferFromWhere = masterOffset;
                        } else {
                           //changes
                            long maxOffset = HAConnection.this.haService.getDefaultMessageStore().getCommitLog().getMaxOffset();
                            long minOffset = HAConnection.this.haService.getDefaultMessageStore().getCommitLog().getMinOffset();
                            if (HAConnection.this.slaveRequestOffset> maxOffset || HAConnection.this.slaveRequestOffset< minOffset) {
                                long masterOffset =
                                        maxOffset
                                                - (maxOffset % HAConnection.this.haService.getDefaultMessageStore().getMessageStoreConfig()
                                                .getMapedFileSizeCommitLog());

                                if (masterOffset < 0) {
                                    masterOffset = 0;
                                }
                                this.nextTransferFromWhere = masterOffset;

                            }else {
                                this.nextTransferFromWhere = HAConnection.this.slaveRequestOffset;
                            }

                        }

                        log.info("master transfer data from " + this.nextTransferFromWhere + " to slave[" + HAConnection.this.clientAddr
                            + "], and slave request " + HAConnection.this.slaveRequestOffset);
                    }

[panzhi33]

HAConnection.this.slaveRequestOffset> maxOffset there is a problem with this situationt，ReputMessageService will down。like this kind of slave broker that has been offline for a long time,It is recommended to rebuild the slave and delete the store directory of the slave.

[RongtongJin]

Before start up the slave，check the commigLog data is correctly by manual is inconvenient to execute 。From the machine of slave broke down to it repaired，we did not modify the data in the store directory of slave，this mistake maybe due to the deletion policy as it'is not time to deleteExpiredFiles at start up time or other reasons. so add check is necessary.

add the following checks when master received slaveRequestOffset at first，if slaveRequestOffset>maxOffset or slaveRequestOffset<minOffset,send last commitLog file. And change slave's dispatchReadRequest method to handle a correct offset from master. Is this idea appropriate?

if (-1 == this.nextTransferFromWhere) {
                        if (0 == HAConnection.this.slaveRequestOffset) {
                            long masterOffset = HAConnection.this.haService.getDefaultMessageStore().getCommitLog().getMaxOffset();
                            masterOffset =
                                masterOffset
                                    - (masterOffset % HAConnection.this.haService.getDefaultMessageStore().getMessageStoreConfig()
                                    .getMapedFileSizeCommitLog());

                            if (masterOffset < 0) {
                                masterOffset = 0;
                            }
                            this.nextTransferFromWhere = masterOffset;
                        } else {
                           //changes
                            long maxOffset = HAConnection.this.haService.getDefaultMessageStore().getCommitLog().getMaxOffset();
                            long minOffset = HAConnection.this.haService.getDefaultMessageStore().getCommitLog().getMinOffset();
                            if (HAConnection.this.slaveRequestOffset> maxOffset || HAConnection.this.slaveRequestOffset< minOffset) {
                                long masterOffset =
                                        maxOffset
                                                - (maxOffset % HAConnection.this.haService.getDefaultMessageStore().getMessageStoreConfig()
                                                .getMapedFileSizeCommitLog());

                                if (masterOffset < 0) {
                                    masterOffset = 0;
                                }
                                this.nextTransferFromWhere = masterOffset;

                            }else {
                                this.nextTransferFromWhere = HAConnection.this.slaveRequestOffset;
                            }

                        }

                        log.info("master transfer data from " + this.nextTransferFromWhere + " to slave[" + HAConnection.this.clientAddr
                            + "], and slave request " + HAConnection.this.slaveRequestOffset);
                    }

This will cause the commitlog to be discontinuous. I think it's best to repair it manually in this case