Multipart - how to disable Content-Length for a part?

[rafakob]

Question:
How to disable Content-Length header for a part? I need to do this because an API that I'm using does not allow it (and I can't do anything about it).

I'm sure that the problem is that header, because I've tried to upload a file manually using HttpURLConnection and it works great without Content-Length, and it fails immediately when I add it.

I've tried to add interceptor to the request (both addInterceptor and addNetworkInterceptor) in order to remove unwanted header. But it is not here (I can see only Content-Disposition). Is there another way to disable it?

Mentioning
#2138

[dave-r12]

@rafakob I think the workaround would be to copy/paste MultipartBody into your project and remove the code that adds the Content-Length.

[rafakob]

But Retrofit's @Part annotation doesn't allow any other custom type. Could you explain me a little bit more how to do this?

[swankjesse]

You might not be able to use Retrofit for this.

[rafakob]

Ok, so I've commented out lines 157-159 in my CustomMultipartBody.java and manually performed request (without Retrofit). It works! Maybe not so elegant but definitely better than using HttpURLConnection because I can use my global OkHttpClient instance with all interceptors, cookies, authorization etc.

Anyway, did anyone else report similar issues? If so, maybe it would be a good idea to make that header optional?

[swankjesse]

You’re the only one. You should petition the owner of the server who has this limitation to fix their server so we don’t have to fix our client.

[cjeker]

This is not a server issue but actually a problem on the client. Non of the RFC handling multipart/form-data do allow Content-Length as a header field for individual parts.
See also:
https://www.ietf.org/rfc/rfc2388.txt
https://www.ietf.org/rfc/rfc2045.txt
https://www.ietf.org/rfc/rfc2046.txt

https://www.ietf.org/rfc/rfc1867.txt has this:

The HTTP protocol may require a content-length for the overall
transmission. Even if it were not to do so, HTTP clients are
encouraged to supply content-length for overall file input so that a
busy server could detect if the proposed file data is too large to be
processed reasonably ....

And the HTTP spec mentions the following:

  Note: HTTP's use of Content-Length for message framing differs
  significantly from the same field's use in MIME, where it is an
  optional field used only within the "message/external-body"
  media-type.

The server may be a bit strict rejecting the POST but the client should not send it in the first place and just gets lucky that other servers are ignoring the field.

[JakeWharton]

Non of the RFC handling multipart/form-data do allow Content-Length as a header field for individual parts.

Not quite. A more accurate statement would be that none of them require, define the semantics of, mention, and certainly do not forbid a Content-Length header in a body part.

RFC 2046 also states:

All other header fields may be ignored in body parts. Although they should generally be retained if at all possible, they may be discarded by gateways if necessary. Such other fields are permitted to appear in body parts but must not be depended on.

[ryn007]

@swankjesse actually,he is not the only one.

[daviduzan]

same request here

[ct-taras]

Another same request

[jesusbm]

same request, please

[MartinWagner-Shape]

Yeah i have the same request here, please

[vicobz]

Would be great here too ! :)

[yuriry]

The reality is that because of this issue okhttp cannot be used as is with many existing APIs
https://www.odata.org/documentation/odata-version-3-0/batch-processing/
http://www.andrewconnell.com/blog/part-2-sharepoint-rest-api-batching-exploring-batch-requests-responses-and-changesets
https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/make-batch-requests-with-the-rest-apis

[bpappin]

+1
I need to stream content that I don't know the length of.

[JakeWharton]

That isn't what this issue is tracking. This is just for the inclusion or exclusion of the header when known.

[code-n-roll]

@swankjesse when to expect this enhancement? As I see 3.13 milestone was overdue.

[swankjesse]

@code-n-roll this might get you unstuck:

  /** Returns a request body with no length available. */
  private RequestBody stripLength(RequestBody delegate) {
    return new RequestBody() {
      @Override public @Nullable MediaType contentType() {
        return delegate.contentType();
      }

      @Override public void writeTo(BufferedSink sink) throws IOException {
        delegate.writeTo(sink);
      }
    };
  }

Use it like this:

    RequestBody part = ...
    MultipartBody body = new MultipartBody.Builder("123")
        .addPart(stripLength(part))
        .build();

[swankjesse]

No further action to take on this.

[stiemannkj1]

I wanted to remove the Content-Length header from each part and avoid building the parts myself, so I used a modified version of @swankjesse's code to work around this problem:

  private static final Pattern CONTENT_LENGTH_PATTERN =
      Pattern.compile(
          "\r?\ncontent-length:\\s*[0-9]+", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE);

  private static RequestBody stripPartContentLengthHeaders(final RequestBody delegate) {
    return new RequestBody() {

      @Override
      public @Nullable MediaType contentType() {
        return delegate.contentType();
      }

      @Override
      public void writeTo(final BufferedSink sink) throws IOException {
        final Buffer buffer = new Buffer();
        delegate.writeTo(buffer);
        final ByteArrayOutputStream output = new ByteArrayOutputStream();
        buffer.writeTo(output);
        final MediaType contentType = delegate.contentType();
        final Charset charset = contentType != null ? delegate.contentType().charset(StandardCharsets.ISO_8859_1)
          : StandardCharsets.ISO_8859_1;
        final String bodyWithoutContentLengthHeaders =
            CONTENT_LENGTH_PATTERN.matcher(output.toString(charset)).replaceAll("");
        sink.writeString(bodyWithoutContentLengthHeaders, charset);
      }
    };
  }

Use the method like so:

new Request.Builder()
  .url(url)
  .post(
    stripPartContentLengthHeaders(
        new MultipartBody.Builder()
            .setType(
                MediaType.parse(
                    MultipartBody.FORM + "; charset=" + StandardCharsets.ISO_8859_1))
            .addFormDataPart("name", "Kyle")
            .addFormDataPart(
                "file",
                "kyles-file.txt",
                RequestBody.create(
                    MediaType.parse("text/plain"),
                    "This is Kyle's File."))
            .build()))
.build()

stripPartContentLengthHeaders() does read the entire body in as a String, so you may need to modify it if your request body is large. You can probably just modify the Buffer or OutputStream to ignore the Content-Length header when the body attempts to write it.

[veita]

Same request here. The Content-Length header does not seem to be good for anything but trouble.

[swankjesse]

FYI, We're dropping this bad behavior in 5.0.