New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix too long log split alarm #915
Conversation
core/profiler/LogtailAlarm.h
Outdated
@@ -90,7 +90,8 @@ enum LogtailAlarmType { | |||
OBSERVER_INIT_ALARM = 61, | |||
OBSERVER_RUNTIME_ALARM = 62, | |||
OBSERVER_STOP_ALARM = 63, | |||
ALL_LOGTAIL_ALARM_NUM = 64 | |||
ALL_LOGTAIL_ALARM_NUM = 64, | |||
LOG_TOO_LONG_SPLIT_ALARM = 65 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
上面有SPLIT_LOG_FAIL_ALARM,这里是不是直接复用一下。
另外,增加ALARM类型的话,ALL_LOGTAIL_ALARM_NUM总应该在最后,表达ALARM类型总数
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
core/reader/LogFileReader.cpp
Outdated
@@ -1632,6 +1632,13 @@ void LogFileReader::ReadUTF8(char*& bufferptr, size_t* size, int64_t end, bool& | |||
} | |||
|
|||
if (moreData && nbytes == 0) { | |||
LOG_WARNING(sLogger, ("Log is too long and is splitted at position", mLastFilePos + nbytes)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个warning信息不全,最好加上文件名和inode信息
Log is too long and forced to be split at offset: xxx file: xxx inode: xxx log: xxx(前1024B)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
超长日志被截断可以分为两种情况:
moreData
为 true,nBytes
为 0。moreData
为 true,readCharCount
为 0。moreData
为 true,readCharCount
为 0(因为之前的日志被LastMatchedLine
截断了,因此,如果需要匹配BeginReg,那么一定是在开头,所以nBytes
为 0。)moreData
为 true,resultCharCount
为 0。