Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate Wazuh symbols for macOS, linux and windows and upload along with installation package. #23760

Open
wants to merge 2 commits into
base: 4.10.0
Choose a base branch
from

Conversation

ncvicchi
Copy link
Member

@ncvicchi ncvicchi commented May 29, 2024

Related issue
Closes #9913

Description

Configuration options

Logs/Alerts example

Tests

  • Compilation without warnings in every supported platform
    • Linux
    • Windows
    • MAC OS X
  • Source installation
  • Package installation
  • Source upgrade
  • Package upgrade
  • Review logs syntax and correct language
  • QA templates contemplate the added capabilities
  • Memory tests for Linux
    • Scan-build report
    • Coverity
    • Valgrind (memcheck and descriptor leaks check)
    • Dr. Memory
    • AddressSanitizer
  • Memory tests for Windows
    • Scan-build report
    • Coverity
    • Dr. Memory
  • Memory tests for macOS
    • Scan-build report
    • Leaks
    • AddressSanitizer
  • Retrocompatibility with older Wazuh versions
  • Working on cluster environments
  • Configuration on demand reports new parameters
  • The data flow works as expected (agent-manager-api-app)
  • Added unit tests (for new features)
  • Stress test for affected components
  • Decoder/Rule tests
    • Added unit testing files ".ini"
    • runtests.py executed without errors

@Dwordcito Dwordcito marked this pull request as ready for review May 31, 2024 15:11
@Dwordcito Dwordcito self-requested a review May 31, 2024 15:18
Copy link
Member

@rauldpm rauldpm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wazuh agent/WPK workflows are stored in a different repository, blocking this PR as it does not seem to be fully updated

@aritosteles aritosteles requested a review from rauldpm June 7, 2024 15:06
@aritosteles aritosteles changed the title Enhancement/9913 generate debug symbols epic Generate Wazuh symbols for macOS, linux and windows and upload along with installation package. Jun 7, 2024
Copy link
Member

@rauldpm rauldpm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add GHA runs for the following cases:

  • Wazuh manager RPM (all archs)
  • Wazuh manager DEB (all archs)

For every system and arch listed, add a build where the following parameters have:

  • Case 1
    • is_stage = true
    • checksum = true
  • Case 2
    • is_stage = false
    • checksum = false
  • Case 3
    • is_stage = true
    • checksum = false
  • Case 4
    • is_stage = false
    • checksum = true

Also, is the debug input needed now?

Comment on lines 131 to 132
echo "PACKAGE_NAME=$(find /tmp -maxdepth 1 -type f -name *manager_*.${{ inputs.system }} -exec basename {} 2>/dev/null \;)" | tee -a $GITHUB_ENV
echo "PACKAGE_SYMBOLS_NAME=$(find /tmp -maxdepth 1 -type f -name *manager-*.${{ inputs.system }} -exec basename {} 2>/dev/null \;)" | tee -a $GITHUB_ENV
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a is_stage=true build to #23760 (comment), we need to be sure that the production nomenclature generation works

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are 4.8.0 agent references in a 4.9.0 manager package generation, this is not correct

cpio: wazuh-agent-4.8.0/src/external/libarchive/libarchive/archive_write_set_options.c: Cannot stat: No such file or directory
cpio: wazuh-agent-4.8.0/src/external/libarchive/libarchive/archive_write_set_passphrase.c: Cannot stat: No such file or directory
cpio: wazuh-agent-4.8.0/src/external/libarchive/libarchive/archive_xxhash.h: Cannot stat: No such file or directory
cpio: wazuh-agent-4.8.0/src/external/libarchive/libarchive/filter_fork.h: Cannot stat: No such file or directory
cpio: wazuh-agent-4.8.0/src/external/libarchive/libarchive/filter_fork_posix.c: Cannot stat: No such file or directory
cpio: wazuh-agent-4.8.0/src/external/libarchive/libarchive/xxhash.c: Cannot stat: No such file or directory
23812 blocks
+ exit 0
Processing files: wazuh-manager-4.9.0-test9913.x86_64
warning: File listed twice: /var/ossec/wodles/aws
warning: File listed twice: /var/ossec/wodles/aws/__init__.py
warning: File listed twice: /var/ossec/wodles/aws/aws-s3
warning: File listed twice: /var/ossec/wodles/aws/aws-s3.py
warning: File listed twice: /var/ossec/wodles/aws/aws_tools.py

@@ -118,4 +118,4 @@ build_deps $legacy
build_package $package_name $debug "$short_commit_hash" "$wazuh_version"

# Post-processing
get_package_and_checksum $wazuh_version $short_commit_hash $src
get_package_and_checksum $wazuh_version $short_commit_hash $src
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

End line

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

export SRC_NAME=$(ls -R ${rpm_build_dir}/SRPMS | grep "\.src\.rpm$")

if [[ "${checksum}" == "yes" ]]; then
cd "${rpm_build_dir}/RPMS" && sha512sum $RPM_NAME > /var/local/wazuh/$RPM_NAME.sha512
if [[ "${src}" == "yes" ]]; then
cd "${rpm_build_dir}/SRPMS" && sha512sum $SRC_NAME > /var/local/wazuh/$SRC_NAME.sha512
sha512sum $SYMBOLS_NAME > /var/local/wazuh/$SYMBOLS_NAME.sha512
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the debug sha512sum file supposed to be uploaded too?

@@ -264,4 +264,4 @@ function main() {
return 0
}

main "$@"
main "$@"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

End line

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -141,5 +141,6 @@ def test_agentd_state_config(test_configuration, test_metadata, remove_state_fil

# Follow ossec.log to find desired messages by a callback
wazuh_log_monitor = FileMonitor(WAZUH_LOG_PATH)
time.sleep(1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seem to recall this was needed because the test was failing due to a timing issue of some sort.

@juliancnn juliancnn changed the base branch from 4.9.0 to 4.10.0 July 15, 2024 19:01
@juliancnn juliancnn force-pushed the enhancement/9913-generate-debug-symbols-epic branch 3 times, most recently from fd280b7 to 533bb13 Compare July 15, 2024 21:06
@juliancnn juliancnn self-requested a review July 15, 2024 21:21
@juliancnn juliancnn force-pushed the enhancement/9913-generate-debug-symbols-epic branch from 533bb13 to 8c27b18 Compare July 16, 2024 15:48
@juliancnn juliancnn self-assigned this Jul 17, 2024
@juliancnn juliancnn removed their request for review July 17, 2024 13:58
@juliancnn
Copy link
Member

juliancnn commented Jul 19, 2024

Update

RPM

I have tested the manager and agent packages without problems in centos 7. The problem occurred in centos 5. Although the processes did not have the symbols, the libraries did, this caused the RPM package of the centos 5 agent to double its size.

Analysis of this problem led to the determination that not only did the makefile have errors, but that CMake builds libraries with symbols regardless of the operating system.

# ls -lh output/**

output/debug:
total 366M
-rw-r--r-- 1 root root  15M Jul 23 16:23 wazuh-agent-debuginfo_4.10.0-0_x86_64_f63e9e9.rpm
-rw-r--r-- 1 root root  22M Jul 23 16:27 wazuh-agent_4.10.0-0.el5_x86_64_f63e9e99dfd.rpm
-rw-r--r-- 1 root root  11M Jul 23 16:23 wazuh-agent_4.10.0-0_x86_64_f63e9e9.rpm
-rw-r--r-- 1 root root  33M Jul 23 16:16 wazuh-manager-debuginfo_4.10.0-0_x86_64_f63e9e9.rpm
-rw-r--r-- 1 root root 287M Jul 23 16:20 wazuh-manager_4.10.0-0_x86_64_f63e9e9.rpm

output/release:
total 365M
-rw-r--r-- 1 root root  20M Jul 23 17:21 wazuh-agent-debuginfo_4.10.0-0_x86_64_f63e9e9.rpm
-rw-r--r-- 1 root root  19M Jul 23 17:25 wazuh-agent_4.10.0-0.el5_x86_64_f63e9e99dfd.rpm
-rw-r--r-- 1 root root 8.9M Jul 23 17:21 wazuh-agent_4.10.0-0_x86_64_f63e9e9.rpm
-rw-r--r-- 1 root root  32M Jul 23 17:12 wazuh-manager-debuginfo_4.10.0-0_x86_64_f63e9e9.rpm
-rw-r--r-- 1 root root 286M Jul 23 17:16 wazuh-manager_4.10.0-0_x86_64_f63e9e9.rpm

juliancnn and others added 2 commits July 23, 2024 20:28
Generate Wazuh symbols for macOS, linux and windows and upload along with installation package.

Co-authored-by: Ariel Martin <ariel.martin@wazuh.com>
Co-authored-by: Marcel Kemp Muñoz <marcel.kemp@wazuh.com>
Co-authored-by: Manuel <manuel.cano@wazuh.com>
Co-authored-by: Raul Del Pozo Moreno <raul.delpozo@wazuh.com>
Co-authored-by: Norberto Vicchi <nvicchi@gmail.com>
- Fixes optimizations in RPM builds
- Fixes package building in centos 5 including debug mode
- Correctly split packeages with symbols for centos >= 6
@juliancnn juliancnn force-pushed the enhancement/9913-generate-debug-symbols-epic branch from f63e9e9 to 115577f Compare July 24, 2024 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generate Wazuh symbols for macOS, linux and windows separate from the binary file
4 participants