How to get the raw search response

[RGB314]

https://forum.opensearch.org/t/how-to-get-the-raw-search-response/17626

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch-java:2.8.0
opensearch-rest-client:2.11.0
opensearch-rest-high-level-client:2.11.0

Describe the issue:
Using the steps in https://github.com/opensearch-project/opensearch-java/blob/main/guides/search.md#get-raw-json-results I am unable to get the complete org.opensearch.client.opensearch.core.SearchResponse as JSON string objectMapper.writeValueAsString(searchResponse). All I get is an empty string (when the object is converted to JSON string by Spring REST) but if I use the object-mapper (see code below) to convert to String, the same object returns a JSON. This is a significant performance overhead. What could I be missing?

Alternatively (preferred), is there no way to get the raw byte stream instead of having to deserialize to org.opensearch.client.opensearch.core.SearchResponse and then serialize again to a String? I am looking for a faster option where our users hit our REST endpoint, we call the OpenSearch cluster and return the raw response as-is with minimum latency. I'm constrained by this flow and the Java API (could try Python if something similar is available).

Configuration:

import static com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility.ANY;
import static com.fasterxml.jackson.annotation.JsonInclude.Include.NON_NULL;
import static com.fasterxml.jackson.annotation.PropertyAccessor.CREATOR;
import static com.fasterxml.jackson.annotation.PropertyAccessor.FIELD;
import static com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_MISSING_CREATOR_PROPERTIES;
import static com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES;
import static com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATES_AS_TIMESTAMPS;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import com.fasterxml.jackson.module.paramnames.ParameterNamesModule;
import org.opensearch.client.json.jackson.JacksonJsonpMapper;
import org.opensearch.client.opensearch.OpenSearchClient;
import org.opensearch.client.transport.aws.AwsSdk2Transport;
import org.opensearch.client.transport.aws.AwsSdk2TransportOptions;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.regions.Region;

@Configuration
public class Config {

    @Bean
    public OpenSearchClient openSearchClient(final Region region) {
        final var jsonpMapper = new JacksonJsonpMapper(getOpenSearchObjectMapper());
        final var transportOptions = AwsSdk2TransportOptions.builder().setMapper(jsonpMapper).build();
        final var awsSdk2Transport = new AwsSdk2Transport(ApacheHttpClient.builder().build(),
                                                          "host",
                                                          region,
                                                          transportOptions);
        return new OpenSearchClient(awsSdk2Transport, transportOptions);
    }

    private static ObjectMapper getOpenSearchObjectMapper() {

        final ObjectMapper mapper = new ObjectMapper();

        mapper.registerModule(new Jdk8Module());
        mapper.registerModule(new JavaTimeModule());

        mapper.disable(WRITE_DATES_AS_TIMESTAMPS);
        mapper.setSerializationInclusion(NON_NULL);
        mapper.configure(FAIL_ON_UNKNOWN_PROPERTIES, false);

        mapper.setVisibility(FIELD, ANY);
        mapper.setVisibility(CREATOR, ANY);

        mapper.registerModule(new ParameterNamesModule());
        mapper.configure(FAIL_ON_MISSING_CREATOR_PROPERTIES, false);

        return mapper;
    }

}

final var request = new SearchRequest.Builder()
            .index(indexName)
            .from(0)
            .size(size)
            .sort(sortOptions)
            .trackScores(true)
            .query(matchPhrasePrefix(...))
            .build();

// Unnecessary casting/deserialisation imo
final var response = openSearchClient.search(request, ObjectNode.class);

// Unnecessary conversion
final var str = objectMapper.writeValueAsString(response);

[dblock]

+1 here, but I think there should be a way to expose the underlying raw request/response in a well documented way. I don't think you need to de-serialized a serialized response, see how to do that (ie. access the underlying object)?

This is related to #257 and #377.