Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: input/io error #3013

Open
1 task done
zcola opened this issue Jan 18, 2024 · 3 comments
Open
1 task done

[Bug]: input/io error #3013

zcola opened this issue Jan 18, 2024 · 3 comments
Assignees
@leonrayang @bboyCH4 @xiaochunhe
Labels
bug Something isn't working

Comments

@zcola
Copy link
Contributor

zcola commented Jan 18, 2024 •
edited

Contact Details

No response

Is there an existing issue for this?

  • I have searched all the existing issues

Priority

high

Environment

- CubeFS version: 3.3.0

Current Behavior

Application layer error reporting

java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Input/output error

Expected Behavior

No response

Steps To Reproduce

It has a low probability of happening. If it does, you can only delete the file and rebuild it. The file itself is not very important.

CubeFS Log

cfs-client log It seems that the same file cannot be written to all the time.

root@elk-host36-598:/cfs/elk_4192_data_02_hot# find . -inum 4486367
./meta_cache
root@elk-host36-598:/cfs/elk_4192_data_02_hot# ll ./meta_cache
-rw-rw-r-- 1 1000 1000 42949672960 1月 17 18:35 ./meta_cache

2024/01/18 11:06:41.439626 [ERROR] extent_client.go:239:
[extent_client.go 238] Write{ino(4486367)offset(3713466368)size(4096)}
IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.453792 [ERROR] super.go:207: Write: ino(4486367) offset(3713466368) len(4096) err([extent_client.go 238] Write{ino(4486367)offset(3713466368)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367))
2024/01/18 11:06:41.454018 [ERROR] extent_client.go:239:
[extent_client.go 238] Write{ino(4486367)offset(3713462272)size(4096)}
IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.454040 [ERROR] super.go:207: Write: ino(4486367) offset(3713462272) len(4096) err([extent_client.go 238] Write{ino(4486367)offset(3713462272)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367))
2024/01/18 11:06:41.488119 [ERROR] extent_client.go:239:
[extent_client.go 238] Write{ino(4486367)offset(3713466368)size(4096)}
IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.488144 [ERROR] super.go:207: Write: ino(4486367) offset(3713466368) len(4096) err([extent_client.go 238] Write{ino(4486367)offset(3713466368)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367))
2024/01/18 11:06:41.499848 [ERROR] extent_client.go:239:
[extent_client.go 238] Write{ino(4486367)offset(3713560576)size(4096)}
IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.499871 [ERROR] super.go:207: Write: ino(4486367) offset(3713560576) len(4096) err([extent_client.go 238] Write{ino(4486367)offset(3713560576)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367))
2024/01/18 11:06:41.523143 [ERROR] wrapper.go:234: getDataPartition: get data partitions fail: volume(elk_4192_data_02_hot) err(data partition not exists)
2024/01/18 11:06:41.528143 [ERROR] extent_client.go:239:
[extent_client.go 238] Write{ino(4486367)offset(3713605632)size(4096)}
IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.528167 [ERROR] super.go:207: Write: ino(4486367) offset(3713605632) len(4096) err([extent_client.go 238] Write{ino(4486367)offset(3713605632)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367))
2024/01/18 11:06:41.529660 [ERROR] super.go:207: Read: ino(4486367) req(Read [ID=0x320e9177 Node=0xbb9 Uid=1000 Gid=1000 Pid=20830] 0x82 4096 @0xdd578000 dir=false fl=0 lock=0 ffl=OpenReadWrite) err(partition[0] not exsit) size(0)
2024/01/18 11:06:41.530582 [ERROR] wrapper.go:234: getDataPartition: get data partitions fail: volume(elk_4192_data_02_hot) err(data partition not exists)
2024/01/18 11:06:41.530615 [ERROR] super.go:207: Read: ino(4486367) req(Read [ID=0x320e922b Node=0xbb9 Uid=1000 Gid=1000 Pid=20830] 0x82 4096 @0xdd578000 dir=false fl=0 lock=0 ffl=OpenReadWrite) err(partition[0] not exsit) size(0)
2024/01/18 11:06:41.578777 [ERROR] extent_client.go:239:
[extent_client.go 238] Write{ino(4486367)offset(3713531904)size(4096)}
IssueWriteRequest: stream writer in error status, ino(4486367)


2024/01/18 11:06:41.439640 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713466368)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.454025 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713462272)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.488127 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713466368)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.499856 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713560576)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.528150 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713605632)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.578785 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713531904)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.654748 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713642496)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.724928 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713548288)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.727316 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713544192)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.822783 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713572864)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.851923 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713540096)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.902916 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713536000)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.918409 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713634304)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.923903 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713515520)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.969990 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713564672)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.972852 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713568768)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.976485 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713617920)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.982713 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713576960)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.986237 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713552384)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.992807 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713581056)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:41.997916 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713613824)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:42.002681 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713630208)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)
2024/01/18 11:06:42.012791 [FATAL] alarm.go:48: _fuseclient_warning [extent_client.go 238] Write{ino(4486367)offset(3713527808)size(4096)} :: IssueWriteRequest: stream writer in error status, ino(4486367)





2024/01/18 11:06:41.523126 [WARN ] client.go:160: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/18 11:06:41.530565 [WARN ] client.go:160: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/18 11:06:41.792066 [WARN ] client.go:160: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/18 11:06:41.792942 [WARN ] client.go:160: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]

Summary:
ID : 319
Name : elk_4192_data_02_hot
Owner : cfs
Authenticate : Disabled
Capacity : 700 GB
Create time : 2023-10-09 07:32:57
DeleteLockTime : 0
Cross zone : Disabled
DefaultPriority : false
Dentry count : 15931
Description :
DpCnt : 50
DpReplicaNum : 3
Follower read : Disabled
Inode count : 15957
Max metaPartition ID : 906
MpCnt : 3
MpReplicaNum : 3
NeedToLowerReplica : Disabled
RwDpCnt : 10
Status : Normal
ZoneName : default
VolType : 0
DpReadOnlyWhenVolFull : false
Transaction Mask : off
Transaction timeout : 1
Tx conflict retry num : 10
Tx conflict retry interval(ms) : 20
Tx limit interval(s) : 0
Quota : Disabled

Data partitions:
ID REPLICAS STATUS ISRECOVER LEADER MEMBERS
8787 3 Read only false 10.90.141.168:17310 10.90.141.170:17310,10.90.141.168:17310,10.90.141.172:17310
8788 3 Read only false 10.90.141.166:17310 10.90.141.166:17310,10.90.141.165:17310,10.90.141.169:17310
8789 3 Read only false 10.90.141.171:17310 10.90.141.167:17310,10.90.141.170:17310,10.90.141.171:17310
8790 3 Read only false 10.90.141.172:17310 10.90.141.165:17310,10.90.141.172:17310,10.90.141.168:17310
8791 3 Read only false 10.90.141.169:17310 10.90.141.169:17310,10.90.141.166:17310,10.90.141.171:17310
8792 3 Read only false 10.90.141.167:17310 10.90.141.168:17310,10.90.141.170:17310,10.90.141.167:17310
8793 3 Read only false 10.90.141.169:17310 10.90.141.169:17310,10.90.141.172:17310,10.90.141.165:17310
8794 3 Read only false 10.90.141.166:17310 10.90.141.171:17310,10.90.141.166:17310,10.90.141.167:17310
8795 3 Read only false 10.90.141.168:17310 10.90.141.172:17310,10.90.141.170:17310,10.90.141.168:17310
8796 3 Read only false 10.90.141.169:17310 10.90.141.165:17310,10.90.141.169:17310,10.90.141.166:17310
9077 3 Read only false 10.90.141.169:17310 10.90.141.165:17310,10.90.141.167:17310,10.90.141.169:17310
9078 3 Read only false 10.90.141.170:17310 10.90.141.166:17310,10.90.141.170:17310,10.90.141.168:17310
9079 3 Read only false 10.90.141.169:17310 10.90.141.169:17310,10.90.141.172:17310,10.90.141.171:17310
9080 3 Read only false 10.90.141.166:17310 10.90.141.167:17310,10.90.141.166:17310,10.90.141.165:17310
9081 3 Read only false 10.90.141.170:17310 10.90.141.170:17310,10.90.141.168:17310,10.90.141.171:17310
9082 3 Read only false 10.90.141.172:17310 10.90.141.167:17310,10.90.141.172:17310,10.90.141.169:17310
9083 3 Read only false 10.90.141.166:17310 10.90.141.165:17310,10.90.141.166:17310,10.90.141.168:17310
9084 3 Read only false 10.90.141.170:17310 10.90.141.172:17310,10.90.141.170:17310,10.90.141.171:17310
9085 3 Read only false 10.90.141.167:17310 10.90.141.165:17310,10.90.141.167:17310,10.90.141.169:17310
9086 3 Read only false 10.90.141.168:17310 10.90.141.168:17310,10.90.141.166:17310,10.90.141.170:17310
9158 3 Read only false 10.90.141.166:17310 10.90.141.169:17310,10.90.141.168:17310,10.90.141.166:17310
9159 3 Read only false 10.90.141.165:17310 10.90.141.172:17310,10.90.141.171:17310,10.90.141.165:17310
9160 3 Read only false 10.90.141.167:17310 10.90.141.168:17310,10.90.141.170:17310,10.90.141.167:17310
9161 3 Read only false 10.90.141.166:17310 10.90.141.165:17310,10.90.141.166:17310,10.90.141.169:17310
9162 3 Read only false 10.90.141.171:17310 10.90.141.167:17310,10.90.141.172:17310,10.90.141.171:17310
9163 3 Read only false 10.90.141.170:17310 10.90.141.170:17310,10.90.141.169:17310,10.90.141.168:17310
9164 3 Read only false 10.90.141.171:17310 10.90.141.165:17310,10.90.141.166:17310,10.90.141.171:17310
9165 3 Read only false 10.90.141.172:17310 10.90.141.167:17310,10.90.141.172:17310,10.90.141.170:17310
9166 3 Read only false 10.90.141.166:17310 10.90.141.169:17310,10.90.141.166:17310,10.90.141.168:17310
9167 3 Read only false 10.90.141.165:17310 10.90.141.165:17310,10.90.141.171:17310,10.90.141.170:17310
9300 3 Read only false 10.90.141.166:17310 10.90.141.166:17310,10.90.141.168:17310,10.90.141.171:17310
9301 3 Read only false 10.90.141.170:17310 10.90.141.169:17310,10.90.141.165:17310,10.90.141.170:17310
9302 3 Read only false 10.90.141.172:17310 10.90.141.172:17310,10.90.141.167:17310,10.90.141.171:17310
9303 3 Read only false 10.90.141.166:17310 10.90.141.166:17310,10.90.141.165:17310,10.90.141.168:17310
9304 3 Read only false 10.90.141.171:17310 10.90.141.171:17310,10.90.141.170:17310,10.90.141.169:17310
9305 3 Read only false 10.90.141.172:17310 10.90.141.168:17310,10.90.141.172:17310,10.90.141.167:17310
9306 3 Read only false 10.90.141.165:17310 10.90.141.166:17310,10.90.141.170:17310,10.90.141.165:17310
9307 3 Read only false 10.90.141.169:17310 10.90.141.167:17310,10.90.141.169:17310,10.90.141.171:17310
9308 3 Read only false 10.90.141.168:17310 10.90.141.172:17310,10.90.141.168:17310,10.90.141.165:17310
9309 3 Read only false 10.90.141.166:17310 10.90.141.169:17310,10.90.141.170:17310,10.90.141.166:17310
9381 3 Writable false 10.90.141.171:17310 10.90.141.166:17310,10.90.141.172:17310,10.90.141.171:17310
9382 3 Writable false 10.90.141.167:17310 10.90.141.165:17310,10.90.141.170:17310,10.90.141.167:17310
9383 3 Writable false 10.90.141.168:17310 10.90.141.169:17310,10.90.141.168:17310,10.90.141.166:17310
9384 3 Writable false 10.90.141.165:17310 10.90.141.172:17310,10.90.141.171:17310,10.90.141.165:17310
9385 3 Writable false 10.90.141.167:17310 10.90.141.167:17310,10.90.141.168:17310,10.90.141.170:17310
9386 3 Writable false 10.90.141.166:17310 10.90.141.166:17310,10.90.141.169:17310,10.90.141.172:17310
9387 3 Writable false 10.90.141.165:17310 10.90.141.170:17310,10.90.141.171:17310,10.90.141.165:17310
9388 3 Writable false 10.90.141.167:17310 10.90.141.169:17310,10.90.141.168:17310,10.90.141.167:17310
9389 3 Writable false 10.90.141.172:17310 10.90.141.171:17310,10.90.141.172:17310,10.90.141.166:17310
9390 3 Writable false 10.90.141.167:17310 10.90.141.165:17310,10.90.141.170:17310,10.90.141.167:17310

Anything else? (Additional Context)

No response
@zcola zcola added the bug Something isn't working label Jan 18, 2024
@zcola
Copy link
Contributor Author

zcola commented Jan 18, 2024

dn log There is definitely free disk space. I don’t know why I am getting this error.

2024/01/18 04:16:19.340211 [ERROR] wrap_operator.go:103: action[OperatePacket] id[Req(2335818448)_Partition(9311)_Extent(57590)_ExtentOffset(0)_KernelOffset(0)_Size(8)_Opcode(OpCreateExtent)_CRC(0)_ResultMesg(DiskNoSpaceErr)] isPrimaryBackReplLeader[true] remote[10.90.140.71:52224], err[op(OpCreateExtent) error(ActionCreateExtent:_no space left on the device)].
2024/01/18 04:16:19.340226 [ERROR] repl_protocol.go:410: id[Req(2335818448)_Partition(9311)_Extent(57590)_ExtentOffset(0)_KernelOffset(0)_Size(8)_Opcode(OpCreateExtent)_CRC(0)_ResultMesg(DiskNoSpaceErr)] isPrimaryBackReplLeader[true] remote[10.90.140.71:52224], err[ActionCreateExtent:_no space left on the device]
2024/01/18 04:16:19.683479 [ERROR] wrap_operator.go:103: action[OperatePacket] id[Req(2335819498)_Partition(9311)_Extent(57591)_ExtentOffset(0)_KernelOffset(0)_Size(8)_Opcode(OpCreateExtent)_CRC(0)_ResultMesg(DiskNoSpaceErr)] isPrimaryBackReplLeader[true] remote[10.90.140.71:55662], err[op(OpCreateExtent) error(ActionCreateExtent:_no space left on the device)].
2024/01/18 04:16:19.683505 [ERROR] repl_protocol.go:410: id[Req(2335819498)_Partition(9311)_Extent(57591)_ExtentOffset(0)_KernelOffset(0)_Size(8)_Opcode(OpCreateExtent)_CRC(0)_ResultMesg(DiskNoSpaceErr)] isPrimaryBackReplLeader[true] remote[10.90.140.71:55662], err[ActionCreateExtent:_no space left on the device]
2024/01/18 04:16:31.276666 [ERROR] wrap_operator.go:103: action[OperatePacket] id[Req(2335845500)_Partition(8809)_Extent(52167)_ExtentOffset(0)_KernelOffset(0)_Size(8)_Opcode(OpCreateExtent)_CRC(0)_ResultMesg(DiskNoSpaceErr)] isPrimaryBackReplLeader[false] remote[10.90.141.169:50092], err[op(OpCreateExtent) error(ActionCreateExtent:_no space left on the device)].
2024/01/18 04:16:31.276682 [ERROR] repl_protocol.go:410: id[Req(2335845500)_Partition(8809)_Extent(52167)_ExtentOffset(0)_KernelOffset(0)_Size(8)_Opcode(OpCreateExtent)_CRC(0)_ResultMesg(DiskNoSpaceErr)] isPrimaryBackReplLeader[false] remote[10.90.141.169:50092], err[ActionCreateExtent:_no space left on the device]
2024/01/18 04:16:31.537151 [ERROR] wrap_operator.go:103: action[OperatePacket] id[Req(2335845894)_Partition(9119)_Extent(84277)_ExtentOffset(0)_KernelOffset(0)_Size(8)_Opcode(OpCreateExtent)_CRC(0)_ResultMesg(DiskNoSpaceErr)] isPrimaryBackReplLeader[false] remote[10.90.141.166:54942], err[op(OpCreateExtent) error(ActionCreateExtent:_no

@cubefs cubefs deleted a comment from github-actions bot Jan 19, 2024
@zcola
Copy link
Contributor Author

zcola commented Jan 25, 2024 •
edited

Configured "minWriteAbleDataPartitionCnt": 0 and used e94b9bf After optimization, there is still a certain probability of occurrence, but the probability of occurrence is reduced.

 2024/01/25 01:15:54.335788 [WARN ] client.go:174: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/25 01:15:54.344694 [WARN ] client.go:174: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/25 01:15:54.345639 [WARN ] client.go:174: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/25 01:15:54.396181 [WARN ] client.go:174: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/25 01:15:54.397069 [WARN ] client.go:174: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/25 01:15:54.335788 [WARN ] client.go:174: serveRequest: code[9], msg[data partition not exists], data[[110 117 108 108]]
2024/01/25 01:15:53.937261 [ERROR] extent_client.go:458:
IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:53.937312 [ERROR] super.go:409: Write: ino(4194309) offset(82162221056) len(131072) err(IssueWriteRequest: stream writer in error status, ino(4194309))
2024/01/25 01:15:54.129045 [ERROR] extent_client.go:458:
IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:54.129083 [ERROR] super.go:409: Write: ino(4194309) offset(82163269632) len(131072) err(IssueWriteRequest: stream writer in error status, ino(4194309))
2024/01/25 01:15:54.240142 [ERROR] extent_client.go:458:
IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:54.240181 [ERROR] super.go:409: Write: ino(4194309) offset(82164318208) len(131072) err(IssueWriteRequest: stream writer in error status, ino(4194309))
2024/01/25 01:15:54.335812 [ERROR] wrapper.go:387: getDataPartitionFromMaster: get data partitions fail: volume(elk_4192_data_05_hot) err(data partition not exists)
2024/01/25 01:15:54.335843 [ERROR] super.go:409: Read: ino(8388612) req(Read [ID=0x10f2de0 Node=0x7 Uid=1000 Gid=1000 Pid=39650] 0x4 24576 @0x61c1000 dir=false fl=0 lock=0 ffl=OpenReadWrite) err(partition[0] not exsit) size(4096)
2024/01/25 01:15:54.344707 [ERROR] wrapper.go:387: getDataPartitionFromMaster: get data partitions fail: volume(elk_4192_data_05_hot) err(data partition not exists)
2024/01/25 01:15:54.344737 [ERROR] super.go:409: Read: ino(4) req(Read [ID=0x10f2e4b Node=0x8 Uid=1000 Gid=1000 Pid=39650] 0x5 4096 @0xb6ac0000 dir=false fl=0 lock=0 ffl=OpenReadWrite) err(partition[0] not exsit) size(0)
2024/01/25 01:15:54.345646 [ERROR] wrapper.go:387: getDataPartitionFromMaster: get data partitions fail: volume(elk_4192_data_05_hot) err(data partition not exists)
2024/01/25 01:15:54.345659 [ERROR] super.go:409: Read: ino(4) req(Read [ID=0x10f2e54 Node=0x8 Uid=1000 Gid=1000 Pid=39650] 0x5 4096 @0xb6ac0000 dir=false fl=0 lock=0 ffl=OpenReadWrite) err(partition[0] not exsit) size(0)
2024/01/25 01:15:54.362116 [ERROR] extent_client.go:458:
IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:54.362147 [ERROR] super.go:409: Write: ino(4194309) offset(82011226112) len(131072) err(IssueWriteRequest: stream writer in error status, ino(4194309))
2024/01/25 01:15:54.396196 [ERROR] wrapper.go:387: getDataPartitionFromMaster: get data partitions fail: volume(elk_4192_data_05_hot) err(data partition not exists)
2024/01/25 01:15:54.396256 [ERROR] super.go:409: Read: ino(4194309) req(Read [ID=0x10f30b2 Node=0x9 Uid=1000 Gid=1000 Pid=40180] 0x6 4096 @0x1318272000 dir=false fl=0 lock=0 ffl=OpenReadWrite) err(partition[0] not exsit) size(0)
2024/01/25 01:15:54.397091 [ERROR] wrapper.go:387: getDataPartitionFromMaster: get data partitions fail: volume(elk_4192_data_05_hot) err(data partition not exists)
2024/01/25 01:15:54.397131 [ERROR] super.go:409: Read: ino(4194309) req(Read [ID=0x10f30b6 Node=0x9 Uid=1000 Gid=1000 Pid=40180] 0x6 4096 @0x1318272000 dir=false fl=0 lock=0 ffl=OpenReadWrite) err(partition[0] not exsit) size(0)
2024/01/25 01:15:53.438203 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:53.690704 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:53.734732 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:53.804704 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:53.937290 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:54.129066 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:54.240165 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:54.362126 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)
2024/01/25 01:15:54.475825 [FATAL] alarm.go:48: _fuseclient_warning IssueWriteRequest: stream writer in error status, ino(4194309)

image

@bboyCH4 bboyCH4 self-assigned this Feb 2, 2024
@bboyCH4
Copy link
Contributor

bboyCH4 commented Feb 2, 2024

i will handle this issue

@Victor1319 Victor1319 mentioned this issue Feb 2, 2024
Merged
@github-actions github-actions bot mentioned this issue Feb 27, 2024
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@leonrayang leonrayang

@bboyCH4 bboyCH4

@xiaochunhe xiaochunhe

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@zcola @leonrayang @bboyCH4 @xiaochunhe