Kinesis Firehose output not working as expected

[thanostak]

Describe the bug:
The kinesisFirehose plugin as described in the documentation:
seems to not be working properly. By investigating the history of the repo, it looks like it was merged as part of PR:
#531
However, we see it missing from OutputSpec:
https://kube-logging.dev/docs/configuration/crds/v1beta1/output_types/
And missing from few files in the code too:
https://github.com/kube-logging/logging-operator/blob/master/pkg/sdk/logging/api/v1beta1/output_types.go
https://github.com/kube-logging/logging-operator/blob/release-4.3/config/crd/bases/logging.banzaicloud.io_outputs.yaml

Expected behaviour:
We should be able to configure a kinesisFirehose output by adding an example like:

spec:
  kinesisFirehose:
    delivery_stream_name: example-stream-name
    region: us-east-1
    format:
      type: json

Steps to reproduce the bug:
In our test cluster adding the output fails to be recognised as valid output:

k apply -f - <<"EOF"
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
  name: firehose-test
  namespace: testing-namespace
spec:
  kinesisFirehose:
    delivery_stream_name: test-logstream
    region: us-east-2
    format:
      type: json
EOF

error: error validating "STDIN": error validating data: ValidationError(Output.spec): unknown field "kinesisFirehose" in io.banzaicloud.logging.v1beta1.Output.spec; if you choose to ignore these errors, turn validation off with --validate=false

Additional context:
Add any other context about the problem here.

Environment details:

• Kubernetes version (e.g. v1.15.2): 1.24
• Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): AWS
• logging-operator version (e.g. 2.1.1): 3.17.10
• Install method (e.g. helm or static manifests):
• Logs from the misbehaving component (and any other relevant logs):
• Resource definition (possibly in YAML format) that caused the issue, without sensitive data:

k apply -f - <<"EOF"
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
  name: firehose-test
  namespace: testing-namespace
spec:
  kinesisFirehose:
    delivery_stream_name: test-logstream
    region: us-east-2
    format:
      type: json
EOF

error: error validating "STDIN": error validating data: ValidationError(Output.spec): unknown field "kinesisFirehose" in io.banzaicloud.logging.v1beta1.Output.spec; if you choose to ignore these errors, turn validation off with --validate=false

/kind bug

[aslafy-z]

Hi @thanostak, I quickly fixed that issue you reported by adding the missing mappings to the Output CRD. Can you please have a try and report if it works for you? Thank you!

[thanostak]

Hey @aslafy-z tried the new config and works, we can successfully define Kinesis Firehose as Output. Thank you for the quick fix!!