bulk processor error with Opensearch 2.11

[mugambikimathi]

Expected Behavior

Seamless integration with Opensearch 2.11

Actual Behavior

Getting an error on the console and ui
temporal | {"level":"error","ts":"2023-11-09T06:36:44.512Z","msg":"elastic: bulk processor \"visibility-processor\" failed but may retry: elastic: Error 400 (Bad Request): Illegal character ((CTRL-CHAR, code 31)): only regular white space (\\r, \\n, \\t) is allowed between tokens\n at [Source: (byte[])\"\\u001F�\\u0008\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000���Ko�F\\u0017\\u0006���3f�1�~�2�\"; line: 1, column: 2] [type=json_parse_exception]","logging-call-at":"logger.go:48","stacktrace":"go.temporal.io/server/common/log.(*zapLogger).Error\n\t/home/builder/temporal/common/log/zap_logger.go:156\ngo.temporal.io/server/common/persistence/visibility/store/elasticsearch/client.(*errorLogger).Printf\n\t/home/builder/temporal/common/persistence/visibility/store/elasticsearch/client/logger.go:48\ngithub.com/olivere/elastic/v7.(*Client).errorf\n\t/go/pkg/mod/github.com/olivere/elastic/v7@v7.0.32/client.go:840\ngithub.com/olivere/elastic/v7.(*bulkWorker).commit.func2\n\t/go/pkg/mod/github.com/olivere/elastic/v7@v7.0.32/bulk_processor.go:559\ngithub.com/olivere/elastic/v7.RetryNotify\n\t/go/pkg/mod/github.com/olivere/elastic/v7@v7.0.32/retry.go:51\ngithub.com/olivere/elastic/v7.(*bulkWorker).commit\n\t/go/pkg/mod/github.com/olivere/elastic/v7@v7.0.32/bulk_processor.go:580\ngithub.com/olivere/elastic/v7.(*bulkWorker).work\n\t/go/pkg/mod/github.com/olivere/elastic/v7@v7.0.32/bulk_processor.go:501"}

Steps to Reproduce the Problem

1. Install Opensearch
2. Connect temporal to opensearch 2.11 for advanced visibility

Specifications

• Version: Opensearch 2.11
• Platform: Server 1.2.0 / UI 2.11.0

[aromanovich]

opensearch-project/OpenSearch#10802 seems to be the reason.

[yiminc]

Thanks @aromanovich , looks like that issue on open search is resolved.

[rodrigozhou]

Thanks @aromanovich for pointing that out. It seems that it's not a Temporal issue, so I'm gonna close the issue.
@mugambikimathi feel free to open again if you think it might also be related to Temporal.