Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade failed #1560

Closed
navyaijm2017 opened this issue Oct 30, 2019 · 13 comments
Labels

Comments

@navyaijm2017
Copy link

@navyaijm2017 navyaijm2017 commented Oct 30, 2019

FEATURE REQUEST

  1. Please describe the feature you are requesting.

I want to upgrade the version of rocketmq from V4_3_2 to V4_5_2

  1. Provide any additional detail on your proposed use case for this feature.
    a、Download the latest version of the binary package from the official website

https://www.apache.org/dyn/closer.cgi?path=rocketmq/4.5.2/rocketmq-all-4.5.2-bin-release.zip
b、Stop the old rocketmq service
c、Launch the rocketmq service with the new version of rocketmq and the old version of the configuration file,Startup failed

  1. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
    By looking at the store.log, I found that the error message is as follows:
    2019-10-30 19:43:11 WARN main - /chj/data/rocketmq/store/consumequeue/%DLQ%AISP_IAM_ACCOUNT_TOKEN_MQ_CONSUMER/0/00000000000000000000 300000 length not matched message store config value, please check it manually
@duhenglucky

This comment has been minimized.

Copy link
Contributor

@duhenglucky duhenglucky commented Oct 31, 2019

Would you like to provide more detail information and configuration? it seems that you changed the 'mappedFileSizeConsumeQueue' during your upgrade process.

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Oct 31, 2019

这是我的配置文件,同一个配置文件,用新版本的程序无法启动,用老程序正常

#所属集群名字
brokerClusterName=rocketmq-cluster-dev
#broker名字,注意此处不同的配置文件填写的不一样
brokerName=broker-b
#0 表示 Master,>0 表示 Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.46.23:9876;192.168.46.24:9876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=False
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=10911
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/chj/data/rocketmq/store
#commitLog 存储路径
storePathCommitLog=/chj/data/rocketmq/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/chj/data/rocketmq/store/consumequeue
#消息索引存储路径
storePathIndex=/chj/data/rocketmq/store/index
#checkpoint 文件存储路径
storeCheckpoint=/chj/data/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/chj/data/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER

#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
sendMessageThreadPoolNums=128
#拉消息线程池数量
pullMessageThreadPoolNums=128

#设置消息大小为1MB
maxMessageSize = 1048576

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Nov 1, 2019

V4_3_2 升级到 V4_5_2 服务无法启动的问题,报错“MQ_CONSUMER/0/00000000000000000000 300000 length not matched message store config value, please check it manually” ,可以通过truncate -s 6000000 队列文件名,把文件大小改成V4_5_2默认的文件大小(默认6000000),就可以正常启动了,这个有点暴力,不知道会不会有其他问题

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Nov 1, 2019

V4_3_2 升级到 V4_5_2 服务无法启动的问题,报错“MQ_CONSUMER/0/00000000000000000000 300000 length not matched message store config value, please check it manually” ,可以通过truncate -s 6000000 队列文件名,把文件大小改成V4_5_2默认的文件大小(默认6000000),就可以正常启动了,这个有点暴力,不知道会不会有其他问题

这样升级之后部分服务无法消费,又回滚了,还请官方指导正确的升级方式

@duhenglucky

This comment has been minimized.

Copy link
Contributor

@duhenglucky duhenglucky commented Nov 1, 2019

@navyaijm2017 /chj/data/rocketmq/store/consumequeue/%DLQ%AISP_IAM_ACCOUNT_TOKEN_MQ_CONSUMER/0/00000000000000000000这个文件在没有升级前是多大,没有truncate以前?

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Nov 1, 2019

@navyaijm2017 /chj/data/rocketmq/store/consumequeue/%DLQ%AISP_IAM_ACCOUNT_TOKEN_MQ_CONSUMER/0/00000000000000000000这个文件在没有升级前是多大,没有truncate以前?
没有truncate之前是 300000,因为我配置的就是300000,mapedFileSizeConsumeQueue=300000

升级不成功的问题应该不是我个例,我看群里有其他人也和我一样,只要有数据就无法启动

并且V4_5_2这个版本不会读取配置文件的这个参数了,我新启动已测试实例,配置文件写的是
mapedFileSizeConsumeQueue=300000,但是实际生成的文件是6000000,是不是代码写死了嘛

@duhenglucky

This comment has been minimized.

Copy link
Contributor

@duhenglucky duhenglucky commented Nov 1, 2019

@navyaijm2017 我明白你的意思了,在4.5.2里面确实把mapedFileSizeConsumeQueue 改为了mappedFileSizeConsumeQueue,你改下这个配置名称试试,抱歉,这个确实给兼容性带来了问题。

ps:mapedFileSizeConsumeQueue=300000这个配置的单位是字节,实际上CQ的每个record是20字节,可以看下这么配置是否符合本意哈。

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Nov 1, 2019

我在一个空的broker上把这个参数改成mappedFileSizeConsumeQueue是生效的,那这个问题有答案了是因为新版本参数名称改了,我原始的需求是V4_3_2 to V4_5_2的升级,我现在即使使用这个新参数,启动还是报错,服务无法启动,这个如何解决呢
2019-11-01 17:33:14 INFO AllocateMappedFileService - AllocateMappedFileService service started
2019-11-01 17:33:14 INFO main - last shutdown normally
2019-11-01 17:33:14 INFO main - load /chj/data/rocketmq/store/commitlog/00000000000000000000 OK
2019-11-01 17:33:14 INFO main - load commit log OK
2019-11-01 17:33:14 WARN main - /chj/data/rocketmq/store/consumequeue/test/13/00000000000000000000 6000000 length not matched message store config value, please check it manually
2019-11-01 17:33:14 INFO main - load consume queue test-13 Failed

@duhenglucky

This comment has been minimized.

Copy link
Contributor

@duhenglucky duhenglucky commented Nov 1, 2019

@navyaijm2017 这个test/13/00000000000000000000感觉是你们在4.5.2 启动之后新创建的,默认给创建成了6000000字节,但是由于配置是300000,所以不匹配了。

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Nov 1, 2019

@navyaijm2017 这个test/13/00000000000000000000感觉是你们在4.5.2 启动之后新创建的,默认给创建成了6000000字节,但是由于配置是300000,所以不匹配了。
那个是我测试的,我们业务用的集群用新版本无法启动,报错如下:
/chj/data/rocketmq/store/consumequeue/%DLQ%AISP_IAM_ACCOUNT_TOKEN_MQ_CONSUMER/0/00000000000000000000 300000 length not matched message store config value, please check it manually

@duhenglucky

This comment has been minimized.

Copy link
Contributor

@duhenglucky duhenglucky commented Nov 1, 2019

@navyaijm2017 这个test/13/00000000000000000000感觉是你们在4.5.2 启动之后新创建的,默认给创建成了6000000字节,但是由于配置是300000,所以不匹配了。
那个是我测试的,我们业务用的集群用新版本无法启动,报错如下:
/chj/data/rocketmq/store/consumequeue/%DLQ%AISP_IAM_ACCOUNT_TOKEN_MQ_CONSUMER/0/00000000000000000000 300000 length not matched message store config value, please check it manually

这个集群上面的配置名称mappedFileSizeConsumeQueue似乎忘记改了呢,文件本身的长度是对的跟你以前的配置是一模一样的,那很大可能就是这个参数又没有生效。

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Nov 1, 2019

已经加上mappedFileSizeConsumeQueue这个参数了,还是无法启动
image

@navyaijm2017

This comment has been minimized.

Copy link
Author

@navyaijm2017 navyaijm2017 commented Nov 4, 2019

broker的参数mapedFileSizeConsumeQueue 改为了mappedFileSizeConsumeQueue,使用新的参数,值保持不变就正常启动了

@duhenglucky duhenglucky closed this Nov 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.