Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(fluentd): Input plugin CRs #972

Merged
merged 3 commits into from Nov 2, 2023

Conversation

MisterMX
Copy link
Contributor

What this PR does / why we need it:

This adds two new CRs Input and ClusterInput to fluentd that are collected like Output and ClusterOutput. They allow specifying Fluentd input plugins individually as an alternative to spec.globalInputs in the FluentD resource.

It also adds the ability to define custom plugins in the input.Input struct.

---
apiVersion: fluentd.fluent.io/v1alpha1
kind: ClusterInput
metadata:
  name: cluster-fluentd-input-cw
  labels:
    input.fluentd.fluent.io/scope: "cluster"
    input.fluentd.fluent.io/enabled: "true"
spec:
  inputs:
    - customPlugin:
        config: |
          <source>
            @type cloudwatch_logs
            tag cloudwatch.in.kafka
            include_metadata true
            log_group_name /aws/kafka/%s
            log_stream_name %s
            use_log_group_name_prefix true
            use_log_stream_name_prefix true
            region {{ $region }}
            use_aws_timestamp true
            format none
            throttling_retry_seconds %s
            <storage>
              @type local
              path /var/log/fluent/cloudwatch.in.kafka.state
            </storage>
          </source>

Support custom plugins in input.Input struct.

Which issue(s) this PR fixes:

Fixes #971

Does this PR introduced a user-facing change?

It adds two new structs but no breaking changes.

Additional documentation, usage docs, etc.:


Add `Input` and `ClusterInput` to fluentd that are collected like
`Output` and `ClusterOutput`.

Support custom plugins in `input.Input` struct.

Signed-off-by: Maximilian Blatt <maximilian.blatt-extern@deutschebahn.com>
…8bdea11b1aa

Fix issue with failing CI job.

Signed-off-by: Maximilian Blatt <maximilian.blatt-extern@deutschebahn.com>
@benjaminhuo
Copy link
Member

Thanks @MisterMX for this great feature. Would you help to tix the lint-test? https://github.com/fluent/fluent-operator/actions/runs/6694949950/job/18290100007?pr=972

You may also need to run make docs-update to generate docs

@MisterMX MisterMX force-pushed the feat/fluentd-input branch 2 times, most recently from b2f0847 to bfb9e4e Compare November 2, 2023 12:00
@MisterMX
Copy link
Contributor Author

MisterMX commented Nov 2, 2023

I bumped helm/chart-testing-action to cb49023b9227b1097e5eddd8824f48bdea11b1aa due to helm/chart-testing-action#132 (comment).

Signed-off-by: Benjamin Huo <benjamin@kubesphere.io>
@benjaminhuo
Copy link
Member

I bumped helm/chart-testing-action to cb49023b9227b1097e5eddd8824f48bdea11b1aa due to helm/chart-testing-action#132 (comment).

Great! Thanks!

@benjaminhuo benjaminhuo merged commit cd0096e into fluent:master Nov 2, 2023
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Fluentd Input Plugin CRDs
2 participants