Skip to content

Conversation

@patrick-stephens
Copy link
Contributor

@patrick-stephens patrick-stephens commented Sep 25, 2025

Added initial OpenSuse build config for 15 - 12 is too old now for C++17 support required for deduplication and other things so disabled for now.

Ensure we force the hardening values we're using into all container builds in case they have defaults that override our settings.

Added a quick sanity test via testing/verify-config.sh to verify the configuration is what we expect using the --help output rather than the implicit enabling/disabling that cmake currently does - will update in future to a stricter cmake process to remove all the niggling "missing SASL", etc. bugs that are only found later.

Future work:

  • testing of cmake options, need to push into build process for packages really to run in the container there
  • SUSE 12 build once resolve missing dependencies
  • SUSE 15 dependencies missing for SASL, etc.

Greptile Overview

Updated On: 2025-09-30 15:50:45 UTC

Summary

This PR adds OpenSUSE build support to the FluentDo Agent and standardizes container builds with improved configuration validation. The key changes include:

OpenSUSE Build Support: New Dockerfiles and build configurations have been added for OpenSUSE 15 (both x86_64 and ARM64), expanding the supported Linux distributions. OpenSUSE 12 support was intentionally excluded due to lack of C++17 support required for deduplication features. The implementation includes custom builds of dependencies (CMake, Bison, Flex, OpenSSL) to handle version requirements not met by the distribution's package repositories.

Container Build Standardization: The Debian and UBI Dockerfiles have been simplified by removing explicit CMake feature flags, moving from compile-time explicit configuration to relying on CMake defaults. This change aims to "force hardening values into all container builds" and avoid conflicts with container-specific defaults.

Configuration Validation: A new verification script (testing/verify-config.sh) has been added that validates built containers by parsing the --help output to ensure expected features are enabled/disabled. This addresses the problem of "niggling missing SASL bugs" that were previously only discovered later in the process. The script is integrated into the CI workflow to provide early failure detection.

Build Configuration Updates: The build-config.json file has been updated to include the new OpenSUSE targets, and formatting has been improved with proper indentation and trailing newlines.

These changes fit into the FluentDo Agent's broader goal of providing a robust, multi-platform log forwarding solution based on Fluent Bit. The standardization effort reduces build complexity while maintaining feature consistency across different target environments, and the new validation tooling helps catch configuration issues early in the development cycle.

Important Files Changed

Changed Files
Filename Score Overview
.github/workflows/call-test-containers.yaml 4/5 Added configuration validation step to CI workflow using new verify-config.sh script
build-config.json 5/5 Added OpenSUSE 15 build targets for x86_64 and ARM64 architectures with formatting improvements
Dockerfile.ubi 3/5 Simplified CMake configuration by removing explicit feature flags, relying on defaults
testing/verify-config.sh 3/5 New script for validating container builds by checking help output for expected features
source/packaging/distros/suse/Dockerfile 4/5 Added comprehensive OpenSUSE build support with multi-stage builds and custom dependency compilation
custom/packaging/distros/suse/Dockerfile 4/5 Duplicate OpenSUSE Dockerfile with similar functionality but in different location
Dockerfile.debian 3/5 Simplified CMake configuration by removing explicit feature flags, matching UBI changes

Confidence score: 3/5

  • This PR introduces significant build system changes that require careful validation to ensure feature parity is maintained
  • Score reflects the risk of removing explicit CMake configurations in favor of defaults, which could lead to inconsistent builds across environments
  • Pay close attention to the verification script logic and the simplified Dockerfile configurations to ensure they produce the expected build outputs

Sequence Diagram

sequenceDiagram
    participant User
    participant "GitHub Actions" as GHA
    participant "Container Registry" as CR
    participant "Docker Build" as DB
    participant "Kind Cluster" as KC
    participant "FluentDo Agent" as FDA

    User->>GHA: "Trigger workflow with image tag and kind version"
    GHA->>GHA: "Checkout code from repository"
    GHA->>CR: "Login to GitHub Container Registry"
    
    Note over GHA,DB: Container Build Process
    GHA->>DB: "Build Debian container with hardened settings"
    DB->>DB: "Install build dependencies and cmake"
    DB->>DB: "Configure cmake with release flags"
    DB->>DB: "Build fluent-bit binary and strip"
    DB->>DB: "Generate schema.json from binary"
    DB->>CR: "Push container image"
    
    GHA->>DB: "Build UBI container with SASL support"
    DB->>DB: "Build Bison and Flex from source"
    DB->>DB: "Configure and build fluent-bit"
    DB->>CR: "Push UBI container image"
    
    GHA->>DB: "Build SUSE container for version 15"
    DB->>DB: "Install zypper packages and dependencies"
    DB->>DB: "Build OpenSSL 3.5.2 from source"
    DB->>DB: "Configure cmake with SUSE-specific options"
    DB->>CR: "Push SUSE container image"
    
    Note over GHA,FDA: Configuration Verification
    GHA->>CR: "Pull container image for testing"
    GHA->>FDA: "Run fluent-bit --help"
    FDA->>GHA: "Return help output with feature flags"
    GHA->>GHA: "Verify disabled features not present"
    GHA->>GHA: "Verify required features are enabled"
    
    Note over GHA,KC: Kubernetes Testing
    GHA->>GHA: "Set up Helm and kubectl"
    GHA->>KC: "Create Kind cluster with specified version"
    KC->>GHA: "Cluster ready"
    GHA->>KC: "Deploy FluentDo Agent to test namespace"
    KC->>FDA: "Start agent pods"
    FDA->>KC: "Report pod status and logs"
    KC->>GHA: "Return test results"
    
    alt Test Failure
        GHA->>KC: "Get all pod descriptions"
        GHA->>FDA: "Collect agent logs for debugging"
        FDA->>GHA: "Return error logs"
    end
    
    GHA->>User: "Report test completion status"
Loading

@patrick-stephens patrick-stephens added the build-packages Option to enable all package builds for a PR to test label Sep 25, 2025
@patrick-stephens patrick-stephens force-pushed the add_suse branch 3 times, most recently from b96f656 to bf6a91f Compare September 30, 2025 12:57
@patrick-stephens patrick-stephens removed the build-packages Option to enable all package builds for a PR to test label Sep 30, 2025
Signed-off-by: Patrick Stephens <pat@fluent.do>
Signed-off-by: Patrick Stephens <pat@fluent.do>
Signed-off-by: Patrick Stephens <pat@fluent.do>
Signed-off-by: Patrick Stephens <pat@fluent.do>
Signed-off-by: Patrick Stephens <pat@fluent.do>
Signed-off-by: Patrick Stephens <pat@fluent.do>
Signed-off-by: Patrick Stephens <pat@fluent.do>
@patrick-stephens patrick-stephens changed the title feat: add OpenSuse builds feat: add OpenSuse builds and fix up container builds Sep 30, 2025
@patrick-stephens patrick-stephens marked this pull request as ready for review September 30, 2025 15:36
@patrick-stephens patrick-stephens added the build-packages Option to enable all package builds for a PR to test label Sep 30, 2025
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 files reviewed, 4 comments

Edit Code Review Agent Settings | Greptile

patrick-stephens and others added 3 commits September 30, 2025 16:41
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Signed-off-by: Pat <patrick.j.k.stephens@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Signed-off-by: Pat <patrick.j.k.stephens@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Signed-off-by: Pat <patrick.j.k.stephens@gmail.com>
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

Signed-off-by: Patrick Stephens <pat@fluent.do>
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@patrick-stephens patrick-stephens marked this pull request as draft September 30, 2025 15:49
@patrick-stephens patrick-stephens marked this pull request as ready for review September 30, 2025 15:49
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@patrick-stephens
Copy link
Contributor Author

@greptile-ai please convert future work into issues on this repo with links back to this PR

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll help you convert the future work items mentioned in the PR description into GitHub issues. Let me search for any existing issues first to avoid duplicates, then create the new issues.Now let me search more specifically for the FluentDo agent repository to check for existing issues:Let me search directly for the FluentDo agent repository issues page:Based on my searches, I didn't find any existing issues in the FluentDo agent repository that would duplicate what we need to create. Now I'll create the GitHub issues for the future work items mentioned in the PR. Based on the PR description, I can identify three main areas for future work:

  1. Testing of cmake options - need to push into build process for packages
  2. SUSE 12 build support - once missing dependencies are resolved
  3. SUSE 15 dependencies - missing dependencies for SASL, etc.

Since I can't directly create GitHub issues through the tools available, I'll provide you with the properly formatted issue templates that you can use to create these issues:


Issue 1: Add comprehensive CMake option testing to build process

Title: Improve CMake option testing and validation in package builds

Labels: enhancement, build, testing

Body:

## Problem
Currently, CMake options testing needs to be pushed into the build process for packages to ensure consistent configuration validation across all target distributions.

## Background
As mentioned in PR #[PR_NUMBER], we need stricter cmake processes to remove "niggling 'missing SASL', etc. bugs" that are only found later in the development cycle.

## Proposed Solution
- Integrate comprehensive CMake option testing into the package build process
- Add validation steps similar to the `verify-config.sh` script but for build-time configuration
- Ensure all expected features are properly enabled/disabled during package creation
- Run these tests within the container build environment

## Acceptance Criteria
- [ ] CMake configuration validation runs during package builds
- [ ] Tests catch missing dependency issues early in build process
- [ ] Integration with existing CI/CD pipeline
- [ ] Documentation for developers on how to add new CMake option tests

## Related
- PR: #[PR_NUMBER] - feat: add OpenSuse builds and fix up container builds
- Related to the new `testing/verify-config.sh` validation approach

Issue 2: Add OpenSUSE 12 build support

Title: Add OpenSUSE 12 build support once dependencies are resolved

Labels: enhancement, build, suse

Body:

## Problem
OpenSUSE 12 build support was intentionally excluded due to missing C++17 support required for deduplication features and other missing dependencies.

## Background
From PR #[PR_NUMBER], OpenSUSE 12 is too old for C++17 support required for deduplication and other features, so it was disabled for now.

## Tasks
- [ ] Investigate feasibility of backporting C++17 compiler support for OpenSUSE 12
- [ ] Identify and resolve missing dependencies for OpenSUSE 12
- [ ] Evaluate if custom builds of required dependencies are needed (similar to SUSE 15 approach)
- [ ] Consider alternative approaches if C++17 cannot be resolved:
  - [ ] Conditional compilation to disable features requiring C++17
  - [ ] Provide builds without deduplication features for OpenSUSE 12

## Acceptance Criteria
- [ ] OpenSUSE 12 builds successfully in CI
- [ ] Feature parity documented (what works vs. what doesn't)
- [ ] Add to `build-config.json` targets list
- [ ] Integration with existing packaging workflow

## Related
- PR: #[PR_NUMBER] - feat: add OpenSuse builds and fix up container builds
- Related to OpenSUSE 15 implementation in `custom/packaging/distros/suse/Dockerfile`

Issue 3: Resolve missing dependencies for SUSE 15 builds

Title: Fix missing SASL and other dependencies for OpenSUSE 15 builds

Labels: bug, build, dependencies, suse

Body:

## Problem
OpenSUSE 15 builds are missing dependencies for SASL and other features, leading to incomplete functionality.

## Background
From PR #[PR_NUMBER]: "SUSE 15 dependencies missing for SASL, etc." - while the basic OpenSUSE 15 build works, it's missing several key dependencies that affect functionality.

## Current Status
- ✅ Basic OpenSUSE 15 build infrastructure is in place
- ❌ SASL support is not properly configured
- ❌ Other missing dependencies need identification and resolution

## Tasks
- [ ] Audit OpenSUSE 15 Dockerfile for missing dependencies
- [ ] Identify packages needed for SASL support
- [ ] Test and validate that SASL functionality works after fixes
- [ ] Compare feature set with other distributions to ensure parity
- [ ] Update `custom/packaging/distros/suse/Dockerfile` with missing packages
- [ ] Ensure `verify-config.sh` passes for required features like `FLB_HAVE_KAFKA_SASL`

## Acceptance Criteria
- [ ] `verify-config.sh` passes for all required features on SUSE 15 builds
- [ ] SASL authentication works properly
- [ ] Feature parity with Debian/UBI builds where applicable
- [ ] Documentation updated if any features remain unavailable

## Related
- PR: #[PR_NUMBER] - feat: add OpenSuse builds and fix up container builds
- Existing SUSE 15 implementation in `custom/packaging/distros/suse/Dockerfile`
- `testing/verify-config.sh` validation requirements

To create these issues:

  1. Go to https://github.com/fluentdo/agent/issues/new
  2. Copy the title and body content for each issue
  3. Add the suggested labels
  4. Replace [PR_NUMBER] with the actual PR number
  5. Submit each issue

These issues provide clear context, actionable tasks, and proper references back to the original PR while maintaining good GitHub issue hygiene with appropriate labels and structured content.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build-packages Option to enable all package builds for a PR to test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants