fix(fluentbit): render namespaced rewrite_tag filter as YAML when configFileFormat=yaml

[pujitha24]

Motivation:
When a FluentBitConfig's ClusterFluentBitConfig sets
spec.configFileFormat: yaml, and a namespaced Filter/FluentBitConfig
triggers the operator's auto-generated rewrite_tag filter (used to
tag records emitted from a given namespace), the operator always
rendered that filter as a hand-written classic TOML snippet
([Filter]\n Name rewrite_tag\n...) and spliced it directly into
the otherwise-YAML fluent-bit.yaml secret. The result is a config file
that mixes TOML and YAML syntax, which fluent-bit fails to parse at
startup. This only affects the yaml config format combined with the
namespaced rewrite-tag scenario; the classic/TOML format and
non-namespaced setups are unaffected, and the operator's reconcile
loop itself completes without error since it has no visibility into
the malformed content it writes.

Fixes #1689

Approach:
Instead of hand-building a format-specific string,
generateRewriteTagConfig now constructs the existing
filter.RewriteTag plugin struct (same Rule/EmitterName/
EmitterStorageType/EmitterMemBufLimit values as before) wrapped in a
synthetic ClusterFilterList, and renders it through the same
Load()/LoadAsYaml() methods already used for user-defined Filter and
ClusterFilter resources. A new configFileFormat parameter is threaded
from ClusterFluentBitConfig.Spec.ConfigFileFormat through
processNamespacedFluentBitCfgs into generateRewriteTagConfig to pick
the right renderer. The classic TOML output is unchanged apart from a
harmless reordering of the Emitter_* keys (order doesn't matter in
fluent-bit's classic format).

Validation:

• go build ./...
• go test ./controllers/... ./apis/fluentbit/v1alpha2/...
  (all packages pass, including a new TestGenerateRewriteTagConfigYaml
  covering both the yaml and classic output paths)
• go vet ./...

Fixed a bug where namespaced Filter resources combined with
`configFileFormat: yaml` produced an invalid fluent-bit.yaml config
(TOML text mixed into YAML) for the auto-generated rewrite_tag filter,
causing fluent-bit to fail to parse its configuration.

Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[pujitha24]

The "Run linter" check was failing on a goconst finding (string \yaml` has 3 occurrences, make it a constant) at controllers/fluentbitconfig_controller.go:290. I've fixed that by extracting the literal into a shared configFileFormatYamlconstant incontrollers/consts.goand using it at both call sites plus the test. Confirmed locally with golangci-lint v2.11.4 (matching CI): 0 issues after the change, and reverting it reproduces the exact reported issue.go build, go vet`, and the affected test packages all still pass. No other feedback is pending on this PR right now.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

controllers/consts.go:17

• configFileFormatYaml is a fixed string constant but is declared in a var block, so it can be mutated at runtime. Declaring it as a const makes the intent explicit and avoids accidental modification.

	fluentdAgentMode  = "agent"

	configFileFormatYaml = "yaml"
)

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

controllers/fluentbitconfig_controller.go:521

• In the YAML path, this renders the rewrite_tag snippet via ClusterFilterList.LoadAsYaml(...), which always includes a filters: header (see apis/fluentbit/v1alpha2/clusterfilter_types.go:166-213). But the full YAML config renderer (ClusterFluentBitConfig.RenderMainConfigInYaml) later appends the cluster filter sections, which also start with filters: (cluster filters) or writes its own filters: header (when only namespaced filters exist). That will produce duplicate filters: keys under pipeline: in common cases (e.g., when any cluster filters or any namespaced filters are present), which is invalid YAML / may cause fluent-bit to ignore one section.

Suggested direction: make rewrite_tag output be filter list items only (no filters: header) and update RenderMainConfigInYaml to emit a single filters: header when any of {cluster filters, rewrite_tag, namespaced filters} exist, then append rewrite_tag entries into that section; or alternatively merge the synthetic rewrite_tag filter into the cluster filter list before calling filters.LoadAsYaml so only one header is emitted. Please also extend the new test to cover assembly into the full YAML main config so this doesn’t regress.

	sl := plugins.NewSecretLoader(nil, "")
	if configFileFormat != nil && *configFileFormat == configFileFormatYaml {
		return filterList.LoadAsYaml(sl, 1)
	}
	return filterList.Load(sl)

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[pujitha24]

This is rebased on master and green — happy to make any changes that would help move review along, just let me know.

[joshuabaird]

@pujitha24 Hi, thanks for the contribution? Could you please address or comment on #2019 (review)? Thanks!

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

controllers/consts.go:16

• configFileFormatYaml is an immutable string; it should be declared as a const (not in a var block) to make intent explicit and prevent accidental mutation.

	fluentbitApiGVStr = fluentbitv1alpha2.SchemeGroupVersion.String()
	fluentdApiGVStr   = fluentdv1alpha1.SchemeGroupVersion.String()
	fluentdAgentMode  = "agent"

	configFileFormatYaml = "yaml"

controllers/fluentbitconfig_controller.go:528

• strings.TrimPrefix will silently do nothing if rendered doesn’t start with the exact header (e.g., if LoadAsYaml output format/indentation changes), causing a filters: header to leak into the merged YAML and potentially reintroduce the duplicate-key/invalid-config issue. Consider validating strings.HasPrefix(rendered, header) and returning an error if it doesn’t match (or remove the first filters: line using a more resilient approach), so failures are detected instead of writing malformed config.

	sl := plugins.NewSecretLoader(nil, "")
	if configFileFormat != nil && *configFileFormat == configFileFormatYaml {
		rendered, err := filterList.LoadAsYaml(sl, 1)
		if err != nil {
			return "", err
		}
		// Strip the "filters:" header so callers can merge this into the
		// single "filters:" section of the main YAML config instead of
		// emitting a second, duplicate key (see RenderMainConfigInYaml).
		header := fmt.Sprintf("%sfilters:\n", utils.YamlIndent(1))
		return strings.TrimPrefix(rendered, header), nil

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (3)

controllers/fluentbitconfig_controller.go:505

• This changes behavior vs the previous implementation: Emitter_Storage.type and Emitter_Mem_Buf_Limit used to be emitted only when non-empty, but now the fields are always assigned (possibly to empty strings). If the renderer does not omitempty these fields, this can produce config entries with empty values (and/or change the classic output unexpectedly). To preserve prior behavior, only set EmitterStorageType / EmitterMemBufLimit when the corresponding spec fields are non-empty (or ensure the underlying struct tags omit empty values in both YAML and classic renderers).

	if cfg.Spec.Service != nil {
		if cfg.Spec.Service.EmitterName != "" {
			rewriteTag.EmitterName = cfg.Spec.Service.EmitterName
		} else {
			rewriteTag.EmitterName = fmt.Sprintf("re_emitted_%x", md5.Sum([]byte(cfg.Namespace)))
		}
		rewriteTag.EmitterStorageType = cfg.Spec.Service.EmitterStorageType
		rewriteTag.EmitterMemBufLimit = cfg.Spec.Service.EmitterMemBufLimit
	}

controllers/fluentbitconfig_controller.go:528

• Stripping the YAML header via an exact TrimPrefix match is brittle: it assumes the renderer will always produce exactly YamlIndent(1) + \"filters:\\n\" (line endings, indentation, or formatting changes can break this silently and reintroduce duplicate filters: keys). A more robust approach is to have the YAML renderer support emitting a list without the parent key (preferred), or to unmarshal the rendered YAML and re-marshal only the filters sequence items so the merge logic does not depend on string formatting.

	sl := plugins.NewSecretLoader(nil, "")
	if configFileFormat != nil && *configFileFormat == configFileFormatYaml {
		rendered, err := filterList.LoadAsYaml(sl, 1)
		if err != nil {
			return "", err
		}
		// Strip the "filters:" header so callers can merge this into the
		// single "filters:" section of the main YAML config instead of
		// emitting a second, duplicate key (see RenderMainConfigInYaml).
		header := fmt.Sprintf("%sfilters:\n", utils.YamlIndent(1))
		return strings.TrimPrefix(rendered, header), nil

controllers/consts.go:16

• configFileFormatYaml is a constant value but is declared in a var block. Consider making it a const (either in a separate const block or by converting this block if feasible) to prevent accidental runtime mutation and clarify intent.

	fluentbitApiGVStr = fluentbitv1alpha2.SchemeGroupVersion.String()
	fluentdApiGVStr   = fluentdv1alpha1.SchemeGroupVersion.String()
	fluentdAgentMode  = "agent"

	configFileFormatYaml = "yaml"