Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix in log date parsing at predecoding stage #15826

Merged
merged 4 commits into from Feb 2, 2023

Conversation

ncvicchi
Copy link
Member

@ncvicchi ncvicchi commented Jan 4, 2023

Related issue
#15772

Description

This pull request fixes an incorrect log date parsing reported by a community user, where a syslog-ng date format has a fraction of seconds with 3 digits instead of 6.

Along with the fix, several date format unit tests have been added to perform verifications.

Thanks to these unit tests, a problem in parsing proftpd 1.3.5 date format was also found and fixed.

ProFTPD 1.3.5 log fixed error

An error was found when a unit test was developed for this use case. The date format is similar to:
2015-04-16 21:51:02,805. But the parsed date missed the last character, resulting in 2015-04-16 21:51:02,80.

Tests

  • Compilation without warnings in every supported platform
    • Linux
  • Review logs syntax and correct language
  • Added unit tests

Unit Tests Log

2/160 Testing: test_cleanevent
2/160 Test: test_cleanevent
Command: "/home/beto/wazuh/wazuh/src/unit_tests/build/analysisd/test_cleanevent"
Directory: /home/beto/wazuh/wazuh/src/unit_tests/build/analysisd
"test_cleanevent" start time: Jan 04 19:32 -03
Output:
----------------------------------------------------------
[==========] Running 25 test(s).
[ RUN      ] test_OS_CleanMSG_fail
[       OK ] test_OS_CleanMSG_fail
[ RUN      ] test_OS_CleanMSG_fail_short_msg
[       OK ] test_OS_CleanMSG_fail_short_msg
[ RUN      ] test_OS_CleanMSG_ossec_min_msg
[       OK ] test_OS_CleanMSG_ossec_min_msg
[ RUN      ] test_OS_CleanMSG_ossec_arrow_msg
[       OK ] test_OS_CleanMSG_ossec_arrow_msg
[ RUN      ] test_OS_CleanMSG_ossec_test_msg
[       OK ] test_OS_CleanMSG_ossec_test_msg
[ RUN      ] test_OS_CleanMSG_ossec_syslog_msg
[       OK ] test_OS_CleanMSG_ossec_syslog_msg
[ RUN      ] test_OS_CleanMSG_syslog_ipv4_msg
[       OK ] test_OS_CleanMSG_syslog_ipv4_msg
[ RUN      ] test_OS_CleanMSG_syslog_ipv6_msg
[       OK ] test_OS_CleanMSG_syslog_ipv6_msg
[ RUN      ] test_OS_CleanMSG_syslog_isodate_timestamp
[       OK ] test_OS_CleanMSG_syslog_isodate_timestamp
[ RUN      ] test_OS_CleanMSG_rsyslog_timestamp
[       OK ] test_OS_CleanMSG_rsyslog_timestamp
[ RUN      ] test_OS_CleanMSG_syslog_ng_isodate_timestamp
[       OK ] test_OS_CleanMSG_syslog_ng_isodate_timestamp
[ RUN      ] test_OS_CleanMSG_proftpd_1_3_5_timestamp
[       OK ] test_OS_CleanMSG_proftpd_1_3_5_timestamp
[ RUN      ] test_OS_CleanMSG_macos_ULS_syslog_timestamp
[       OK ] test_OS_CleanMSG_macos_ULS_syslog_timestamp
[ RUN      ] test_OS_CleanMSG_timestamp
[       OK ] test_OS_CleanMSG_timestamp
[ RUN      ] test_OS_CleanMSG_osx_asl_timestamp
[       OK ] test_OS_CleanMSG_osx_asl_timestamp
[ RUN      ] test_OS_CleanMSG_apache_timestamp
[       OK ] test_OS_CleanMSG_apache_timestamp
[ RUN      ] test_OS_CleanMSG_suricata_timestamp
[       OK ] test_OS_CleanMSG_suricata_timestamp
[ RUN      ] test_OS_CleanMSG_snort_timestamp
[       OK ] test_OS_CleanMSG_snort_timestamp
[ RUN      ] test_OS_CleanMSG_xferlog_timestamp
[       OK ] test_OS_CleanMSG_xferlog_timestamp
[ RUN      ] test_extract_module_from_message
[       OK ] test_extract_module_from_message
[ RUN      ] test_extract_module_from_message_arrow
[       OK ] test_extract_module_from_message_arrow
[ RUN      ] test_extract_module_from_message_end_error
[       OK ] test_extract_module_from_message_end_error
[ RUN      ] test_extract_module_from_message_arrow_error
[       OK ] test_extract_module_from_message_arrow_error
[ RUN      ] test_extract_module_from_location
[       OK ] test_extract_module_from_location
[ RUN      ] test_extract_module_from_location_arrow
[       OK ] test_extract_module_from_location_arrow
[==========] 25 test(s) run.
[  PASSED  ] 25 test(s).
<end of output>
Test time =   0.15 sec
----------------------------------------------------------
Test Passed.
"test_cleanevent" end time: Jan 04 19:32 -03
"test_cleanevent" time elapsed: 00:00:00
----------------------------------------------------------

@ncvicchi ncvicchi self-assigned this Jan 4, 2023
@ncvicchi ncvicchi added type/bug Something isn't working module/analysis Issues related to the Analysis daemon reporter/community labels Jan 4, 2023
@ncvicchi ncvicchi changed the title 15730 fix syslog timestamp Fix in log date parsing at predecoding stage Jan 4, 2023
@ncvicchi ncvicchi marked this pull request as ready for review January 4, 2023 22:49
chemamartinez
chemamartinez previously approved these changes Jan 18, 2023
Copy link
Contributor

@chemamartinez chemamartinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

jmv74211
jmv74211 previously approved these changes Feb 1, 2023
Copy link
Contributor

@jmv74211 jmv74211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA review

Note: The development has been approved even though the Solaris build checks are 🔴. The node where these checks are launched is under maintenance.

@chemamartinez chemamartinez changed the base branch from master to 4.5 February 2, 2023 08:30
@chemamartinez chemamartinez dismissed jmv74211’s stale review February 2, 2023 08:30

The base branch was changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module/analysis Issues related to the Analysis daemon reporter/community type/bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Syslog timestamp with milliseconds granularity is not correctly decoded
3 participants