[stanza] stanza transformer helper should optionally log stanza.Entry

[timannguyen]

Component(s)

No response

Is your feature request related to a problem? Please describe.

when error occurs in transformer, https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/operator/helper/transformer.go#L98, It would log the Stanza.Entry object. This contains the events being processed. This can contain Secrets or PII. The behavior is undesirable in production environment.

Example:

{"level":"ERROR","time":"2023-09-12T00:20:31.005Z","location":"helper/transformer.go:98","message":"Failed to process entry","service":"edge-processor","hostname":"hostname","commit":"1760f17b","version":"1.0.0","kind":"receiver","name":"syslog","data_type":"logs","operator_id":"syslog_input_internal_parser","operator_type":"syslog_parser","error":"expecting a structured data section containing one or more elements (`[id( key=\"value\")*]+`) or a nil value [col 64]","action":"send","entry":{"observed_timestamp":"2023-09-12T00:20:31.005387926Z","timestamp":"0001-01-01T00:00:00Z","body":"<13>1 2023-09-11T19:08:06.152733-04:00 NicksSurfacePro nick - - hello world","severity":0,"scope_name":""},"callstack":"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*TransformerOperator).HandleEntryError\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.83.0/operator/helper/transformer.go:98\ngithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ParseWith\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.83.0/operator/helper/parser.go:140\ngithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWithCallback\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.83.0/operator/helper/parser.go:112\ngithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/parser/syslog.(*Parser).Process\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.83.0/operator/parser/syslog/syslog.go:153\ngithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*WriterOperator).Write\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.83.0/operator/helper/writer.go:53\ngithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/input/tcp.(*Input).handleMessage\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.83.0/operator/input/tcp/tcp.go:324\ngithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/input/tcp.(*Input).goHandleMessages.func1\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.83.0/operator/input/tcp/tcp.go:285"}

Describe the solution you'd like

Similarly with onError flag, https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/operator/helper/transformer.go#L28, I would suggest to another flag with something like logEntry.

type TransformerConfig struct {
	WriterConfig        `mapstructure:",squash"`
	OnError      string `mapstructure:"on_error"`
	IfExpr         string `mapstructure:"if"`
	logEntry     bool   `mapstructure:"log_entry"`
}

and on https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/operator/helper/transformer.go#L98, it could be

  if logEntry {
    t.Errorw("Failed to process entry", zap.Any("error", err), zap.Any("action", t.OnError), zap.Any("entry", entry))
  } else {
     t.Errorw("Failed to process entry", zap.Any("error", err), zap.Any("action", t.OnError))
  } 

Describe alternatives you've considered

we wrote an zapcore.Encoder to intercept and filter stanza.Entry if it's on the log

Additional context

No response

[github-actions]

Pinging code owners:

• stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

Good catch @timannguyen. Will you open the PR?

[github-actions]

Pinging code owners for pkg/stanza: @djaglowski. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• pkg/stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• pkg/stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.