Compatibility Question: Helm Chart v3.3.0 with Fluent Bit Package v4.0.0

[truongnht]

Hi Maintainers,

We're evaluating our Fluent Operator setup. We are on Helm chart v3.3.0, which bundles your Fluent Bit package v3.2.5. We see that package 4.0.0 is now available.

We understand that we can technically override the fluentBit.image.tag in the v3.3.0 chart's values to point to 4.0.0. However, we are concerned about potential compatibility issues, such as:

Untested combination for this chart release.
Potential breaking changes in Fluent Bit configuration syntax or behavior between 3.2.5 and 4.0.0 that the operator logic in v3.3.0 might not handle correctly.
Could you please advise if using the v3.3.0 chart with the 4.0.0 Fluent Bit package is supported or known to work? Or is upgrading the Helm chart the strictly recommended path to use the newer Fluent Bit package?

Thanks in advance,

[truongnht]

@cw-Guo @joshuabaird

[joshuabaird]

Technically, a user is able to run whatever version of fluent-bit/fluentd they want with any given operator release, however the maintainers of this project will not likely test new versions of fluentbit/fluent with older fluent-operator releases -- so compatibility cannot be guaranteed. With that being said, I believe several users are using fluentbit:4.0 with fluent-operator:3.3.0 successfully.

If a user decides to veer from the fluentbit/fluentd versions that we provide as default values with any given fluent-operator Helm release, my opinion is that it is their responsibility to verify compatibility.

Officially, from a support standpoint, I think it makes sense to recommend that users stick to versions of fluentbit/fluentd that we provide in each fluent-operator Helm release. If everyone is in agreement here, we should probably clarify this stance in the operator's docs.

Also, regarding release cadence between the three main components of fluent-operator (fluent-operator, fluentbit, fluentd) -- The operator version does not track fluent-bit or fluentd (meaning that v5.0.0 of the fluent-operator Helm chart may specify fluent-operator:v4.9.5 and fluent-bit:v4.3.0 for example).

[truongnht]

thanks for the answer, @joshuabaird

[cw-Guo]

in my opinion, if there are no breaking changes from fluent-bit or fluentd in terms of the config file, them it should be safe to assume newer versions of fluent-bit or fluentd will be compatible to fluent-operator.

in production, i would say a testing environment will always be a must, if there are any render issue from fluent-operator, we can find it before pushing to production.

[truongnht]

@cw-Guo does this example mean that we should not bluntly assume things are always compatible? Can we trace where it starts breaking and what would you do differently?

[cw-Guo]

hi @truongnht , I don't think it's a good example. Yaml configure file was introduced without support of parsers, multiline parsers, and so on, and that's the assumption we have when we implemented the yaml config file support in fluent-operator. With that being said, I think it may be broken since day 1 when we released the feature. 😭