Invalid UTF-8 causes export failure while marshaling

[win5do]

otel@v1.7.0/handler.go:44       otel err: rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8
go.opentelemetry.io/otel.(*delegator).Handle
        /go/pkg/mod/go.opentelemetry.io/otel@v1.7.0/handler.go:44
go.opentelemetry.io/otel.Handle
        /go/pkg/mod/go.opentelemetry.io/otel@v1.7.0/handler.go:97
go.opentelemetry.io/otel/sdk/trace.(*batchSpanProcessor).processQueue
        /go/pkg/mod/go.opentelemetry.io/otel/sdk@v1.7.0/trace/batch_span_processor.go:299
go.opentelemetry.io/otel/sdk/trace.NewBatchSpanProcessor.func1
        /go/pkg/mod/go.opentelemetry.io/otel/sdk@v1.7.0/trace/batch_span_processor.go:125

According to #2314 ，truncation of multibyte UTF-8 strings configured via AttributeValueLengthLimit can cause this error.

eg:

s := "你好"
fmt.Println(len(s)) // output: 6
fmt.Println(utf8.ValidString(s[:5])) // output: false

Looks like there are no plans to change string to byte in proto in the near future.

Checking each span may have performance loss, so is it possible to logging the span info or invalid string related to the error?

We can only fix manually now like this:

func TruncateAttr(s string) string {
	if len(s) < 1000 {
		return s
	}
	return truncateAttr(s, 200, "...")
}

func truncateAttr(s string, limit int, add string) string {
	rn := []rune(s)
	if len(rn) > limit {
		rn = rn[:limit]
		return string(rn) + add
	}
	return s
}

[jmacd]

I have seen this issue in the wild, too. I believe the correct thing to do is to suppress gRPC and protobuf's definition of a well-formed string so that SDKs pass invalid UTF-8 through. We can still say that users SHOULD use UTF-8, but we can require SDKs not to fail in the presence of invalid UTF-8.