Re-serialization of serialized object that was sent with request can lead to OutOfMemoryError

[mike-harris0n]

Starting at com.microsoft.graph.http.CoreHttpProvider#processResponse, if a request fails it attempts to serialize (again) the input object that was sent in the request at com.microsoft.graph.http.GraphServiceException#createFromResponse(com.microsoft.graph.http.IHttpRequest, T, com.microsoft.graph.serializer.ISerializer, okhttp3.Response, com.microsoft.graph.logger.ILogger). Why does it need to serialize again? It would be great if it was possible to avoid this and just return a usable exception object.

Our app sends customer-created emails via multiple services including MS365 via Graph. Sometimes the sent emails can be large. We are already having to use a CustomRequest as a crutch in order to properly send pre-built MIME email strings. I'm beginning to think we might be better off using the Java Graph API to create the URLs and use our own HTTP client so that we don't have to deal with this...

Expected behavior

Error responses can be handled without excessive additional memory consumption.

Actual behavior

pool-768-thread-1
  at java.lang.OutOfMemoryError.<init>()V (Unknown Source)
  at java.util.Arrays.copyOf([BI)[B (Unknown Source)
  at java.lang.AbstractStringBuilder.ensureCapacityInternal(I)V (Unknown Source)
  at java.lang.AbstractStringBuilder.append(Ljava/lang/CharSequence;II)Ljava/lang/AbstractStringBuilder; (Unknown Source)
  at java.lang.StringBuffer.append(Ljava/lang/CharSequence;II)Ljava/lang/StringBuffer; (Unknown Source)
  at java.io.StringWriter.write(Ljava/lang/String;II)V (Unknown Source)
  at com.google.gson.stream.JsonWriter.string(Ljava/lang/String;)V (JsonWriter.java:637)
  at com.google.gson.stream.JsonWriter.value(Ljava/lang/String;)Lcom/google/gson/stream/JsonWriter; (JsonWriter.java:426)
  at com.google.gson.internal.bind.TypeAdapters$28.write(Lcom/google/gson/stream/JsonWriter;Lcom/google/gson/JsonElement;)V (TypeAdapters.java:805)
  at com.google.gson.internal.bind.TypeAdapters$28.write(Lcom/google/gson/stream/JsonWriter;Ljava/lang/Object;)V (TypeAdapters.java:698)
  at com.google.gson.internal.Streams.write(Lcom/google/gson/JsonElement;Lcom/google/gson/stream/JsonWriter;)V (Streams.java:72)
  at com.google.gson.JsonElement.toString()Ljava/lang/String; (JsonElement.java:316)
  at com.microsoft.graph.serializer.DefaultSerializer.serializeObject(Ljava/lang/Object;)Ljava/lang/String; (DefaultSerializer.java:233)
  at com.microsoft.graph.http.GraphServiceException.createFromResponse(Lcom/microsoft/graph/http/IHttpRequest;Ljava/lang/Object;Lcom/microsoft/graph/serializer/ISerializer;Lokhttp3/Response;Lcom/microsoft/graph/logger/ILogger;)Lcom/microsoft/graph/http/GraphServiceException; (GraphServiceException.java:368)
  at com.microsoft.graph.http.CoreHttpProvider.handleErrorResponse(Lcom/microsoft/graph/http/IHttpRequest;Ljava/lang/Object;Lokhttp3/Response;)V (CoreHttpProvider.java:512)
  at com.microsoft.graph.http.CoreHttpProvider.processResponse(Lokhttp3/Response;Lcom/microsoft/graph/http/IHttpRequest;Ljava/lang/Class;Ljava/lang/Object;Lcom/microsoft/graph/http/IStatefulResponseHandler;)Ljava/lang/Object; (CoreHttpProvider.java:442)
  at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(Lcom/microsoft/graph/http/IHttpRequest;Ljava/lang/Class;Ljava/lang/Object;Lcom/microsoft/graph/http/IStatefulResponseHandler;)Ljava/lang/Object; (CoreHttpProvider.java:408)
  at com.microsoft.graph.http.CoreHttpProvider.send(Lcom/microsoft/graph/http/IHttpRequest;Ljava/lang/Class;Ljava/lang/Object;Lcom/microsoft/graph/http/IStatefulResponseHandler;)Ljava/lang/Object; (CoreHttpProvider.java:225)
  at com.microsoft.graph.http.CoreHttpProvider.send(Lcom/microsoft/graph/http/IHttpRequest;Ljava/lang/Class;Ljava/lang/Object;)Ljava/lang/Object; (CoreHttpProvider.java:202)
  at com.microsoft.graph.http.BaseRequest.send(Lcom/microsoft/graph/http/HttpMethod;Ljava/lang/Object;)Ljava/lang/Object; (BaseRequest.java:335)
  at com.microsoft.graph.http.CustomRequest.post(Ljava/lang/Object;)Ljava/lang/Object; (CustomRequest.java:154)

Steps to reproduce the behavior

1. Send custom POST request with large MIME email string, that results in 400 error response

[mike-harris0n]

To summarize, we are passing in a MIME email string, an error response occurs, and before we can get our hands on the error, the graph library is unnecessarily creating a copy of that exact same input string...

[ddyett]

can we verify this is fixed in preview versions?

[baywet]

@ddyett for reference the customer is using this content type on the endpoint. This is not being described in the metadata (I don't think there's way to describe this in OData terms).

@mike-harris0n could you provide a snippet of the code that you're using to send the request please? You shouldn't be using the fluent API at all for that call since it'll expect a model (and try to serialize it to JSON).

[mike-harris0n]

could you provide a snippet of the code that you're using to send the request please? You shouldn't be using the fluent API at all for that call since it'll expect a model (and try to serialize it to JSON).

Ah, so we shouldn't bother using com.microsoft.graph:microsoft-graph at all in this scenario? Our app already creates a full MIME email message so we want to send as-is rather than have to use the provided builder API. The only way I was able to get the Graph API to work with this use-case is ugly:

String url = client.me().sendMail(new UserSendMailParameterSet()).getRequestUrl();
if (url == null) {
    throw new IllegalStateException("Graph API request URL could not be determined for host " + acHost.getBaseUrl());
}
if (url.contains(BaseClient.DEFAULT_GRAPH_ENDPOINT)) { //need to strip base URL so it doesn't get added again...
    url = url.substring(BaseClient.DEFAULT_GRAPH_ENDPOINT.length());
}
CustomRequest<String> request = client.customRequest(url, String.class).buildRequest(CONTENT_TYPE_TEXT_PLAIN);
authorizeRequest(request, creds);
request.post(encodedContent);

Are we better off just making the request and handling the response ourselves? I appreciate that in above code we are essentially using the SDK as a glorified REST client.

If we were to go without the SDK we lose out on the built-in error response handling it provides.

[baywet]

@mike-harris0n you can still get the benefits of authentication, retry/redirect/etc... handling by using a barebone okhttpclient augmented with our middleware handlers suite. more information
I'd go that route for this specific case. And you can of course keep using the full service client (chained method API surface) for everything else.