Add Support for User Defined Annotations

[frankgreco]

What this PR does / why we need it:

There is no way to inject annotations into FluentBit DaemonSet pods. This is problematic as I cannot add the necessary annotations to my Prometheus scraper to scape the metrics that these pods expose.

Which issue(s) this PR fixes:

Fixes #311

Does this PR introduced a user-facing change?

Support for user defined annotations has been added. To add annotations to FluentBit pods, specify them in FluentBit.Spec.Annotations.

Additional documentation, usage docs, etc.:

I'm not sure what should go here. Is this referring to FluentBit documentation?

[frankgreco]

@wenchajun could you give approval for workflow runs on this PR?

[wenchajun]

@wenchajun could you give approval for workflow runs on this PR?

Okay, but it looks like you submitted an extra commit, can you rebase it?

[frankgreco]

Here's the local e2e tests I ran locally.

$ make run

$ cat <<EOF | kubectl apply -f -
apiVersion: fluentbit.fluent.io/v1alpha2
kind: FluentBit
metadata:
  name: fluent-bit
  namespace: fluent
  labels:
    app.kubernetes.io/name: fluent-bit
spec:
  image: kubesphere/fluent-bit:v1.8.11
  fluentBitConfigName: fluent-bit-config
  annotations:
    foo.io: bar
EOF

$ kubectl -n fluent get po/fluent-bit-dfbp7 -o json | jq .metadata.annotations
{
  "foo": "bar"
}

[zhu733756]

Do we add an annotation to codes like https://github.com/fluent/fluent-operator/blob/master/controllers/fluentbit_controller.go#L147?

[wenchajun]

/lgtm

[benjaminhuo]

Thanks for the contribution @frankgreco