Provide a post() method on WS for POSTing multipart/form-data

[julien-lafont]

There is actually no way to post a multipart/form-data, without encoding manually the body (and this is tricky!)

It's sad to not have a WS method which allow to call a controller like the "File upload example"

[ryanoglesby08]

This functionality would be very helpful. Agreed.

A method as part of FakeRequest that is something along the lines of "fakeRequest().withMultipartFormData(formData);" would help immensely with testing controllers using multipart form data.

[joost-de-vries]

Another strong vote for this feature. I've closed the other issue #1302 in favour of this one.

[ewilson]

Please consider implementing this.

[ornicar]

👍

[AlexGalays]

👍

[cxvvs]

In the meantime, here is a workaround that was greatly inspired by some code in Play

import com.ning.http.client.{Response => AHCResponse, _}
import play.api.libs.ws._
import play.api.libs.ws.ning.NingWSResponse
import scala.concurrent.{Future, Promise}
import play.api.Play.current

  object WSUtil {

    def post(url: String, bodyParts: List[Part]): Future[Response] = {
      val client = WS.client(current).underlying.asInstanceOf[AsyncHttpClient]

      val builder = client.preparePost(url)

      builder.setHeader("Content-Type", "multipart/form-data")
      bodyParts.foreach(builder.addBodyPart)

      var result = Promise[NingWSResponse]()

      client.executeRequest(builder.build(), new AsyncCompletionHandler[AHCResponse]() {
        override def onCompleted(response: AHCResponse) = {
          result.success(NingWSResponse(response))
          response
        }

        override def onThrowable(t: Throwable) = {
          result.failure(t)
        }
      })

      result.future
    }

[toggm]

Based on WSClient and WSRequestHolder I implemented a multipart based request.

package helper

import com.ning.http.client.{ Response => AHCResponse, Part => AHCPart, StringPart => AHCStringPart, FilePart => AHCFilePart, _ }
import play.api.libs.ws._
import play.api.libs.ws.ning.NingWSResponse
import scala.concurrent.{ Future, Promise }
import play.api.Play.current
import play.api.libs.ws.WS
import com.ning.http.client.AsyncHttpClient._
import play.api.libs.iteratee.Enumerator
import collection.JavaConversions._
import java.io.File
import com.ning.http.multipart.{ FilePart => MPFilePart }

sealed trait Part {
  def toAHCPart: AHCPart
}
case class StringPart(key: String, value: String) extends Part {
  def toAHCPart = {
    new AHCStringPart(key, value)
  }
}
case class FilePart(key: String, file: File, contentType: String, charset: String) extends Part {
  def toAHCPart = {
    new MPFilePart(key, file, contentType, charset)
  }
}

case class MultiPartWSRequestHolder(client: AsyncHttpClient, builder: AsyncHttpClient#BoundRequestBuilder, url: String) extends WSRequestHolder {

  def withPart(part: Part): MultiPartWSRequestHolder = {
    builder.addBodyPart(part.toAHCPart)
    this
  }

  def withParts(parts: Part*): MultiPartWSRequestHolder = {
    parts.forall(p => builder.addBodyPart(p.toAHCPart) != null)
    this
  }

  /**
   * The method for this request
   */
  val method: String = ""

  /**
   * The body of this request
   */
  val body: WSBody = EmptyBody

  /**
   * 7
   * The headers for this request
   */
  val headers: Map[String, Seq[String]] = Map()

  /**
   * The query string for this request
   */
  val queryString: Map[String, Seq[String]] = Map()

  /**
   * A calculator of the signature for this request
   */
  val calc: Option[WSSignatureCalculator] = None

  /**
   * The authentication this request should use
   */
  val auth: Option[(String, String, WSAuthScheme)] = None

  /**
   * Whether this request should follow redirects
   */
  val followRedirects: Option[Boolean] = None

  /**
   * The timeout for the request
   */
  val requestTimeout: Option[Int] = None

  /**
   * The virtual host this request will use
   */
  val virtualHost: Option[String] = None

  /**
   * The proxy server this request will use
   */
  val proxyServer: Option[WSProxyServer] = None

  /**
   * sets the signature calculator for the request
   * @param calc
   */
  def sign(calc: WSSignatureCalculator): MultiPartWSRequestHolder = {
    this
  }

  /**
   * sets the authentication realm
   */
  def withAuth(username: String, password: String, scheme: WSAuthScheme): MultiPartWSRequestHolder = {
    //not yet supported
    this
  }

  /**
   * adds any number of HTTP headers
   * @param hdrs
   */
  def withHeaders(hdrs: (String, String)*): MultiPartWSRequestHolder = {
    hdrs.forall(h => builder.setHeader(h._1, h._2) != null)
    this
  }

  def withMethod(method: String): MultiPartWSRequestHolder = {
    builder.setMethod(method)
    this
  }

  /**
   * Sets the maximum time in milliseconds you expect the request to take.
   * Warning: a stream consumption will be interrupted when this time is reached.
   */
  def withRequestTimeout(timeout: Int): MultiPartWSRequestHolder = {
    this
  }

  /**
   * Sets the virtual host to use in this request
   */
  def withVirtualHost(vh: String): MultiPartWSRequestHolder = {
    this
  }

  /**
   * Sets the proxy server to use in this request
   */
  def withProxyServer(proxyServer: WSProxyServer): MultiPartWSRequestHolder = {
    this
  }

  /**
   * Sets the body for this request
   */
  def withBody(body: WSBody): MultiPartWSRequestHolder = {
    this
  }

  /**
   * adds any number of query string parameters to the
   */
  def withQueryString(parameters: (String, String)*): MultiPartWSRequestHolder = {
    val map = parameters.groupBy(_._1).map { case (k, v) => (k, v.map(_._2)) }
    val builderMap = new FluentStringsMap()
    map.forall { case (key, values) => builderMap.add(key, values) != null }

    builder.setQueryParameters(builderMap)

    this
  }

  /**
   * Sets whether redirects (301, 302) should be followed automatically
   */
  def withFollowRedirects(follow: Boolean): MultiPartWSRequestHolder = {
    builder.setFollowRedirects(follow)
    this
  }

  /**
   * Execute this request
   */
  def execute(): Future[WSResponse] = {
    var result = Promise[WSResponse]()

    client.executeRequest(builder.build(), new AsyncCompletionHandler[AHCResponse]() {
      override def onCompleted(response: AHCResponse) = {
        result.success(NingWSResponse(response))
        response
      }

      override def onThrowable(t: Throwable) = {
        result.failure(t)
        ()
      }
    })

    result.future
  }

  /**
   * Execute this request and stream the response body in an enumerator
   */
  def stream(): Future[(WSResponseHeaders, Enumerator[Array[Byte]])] = {
    Future.failed(new RuntimeException("Not supported"))
  }
}

case class MultPartWSClient(wsClient: WSClient) {
  val client = wsClient.underlying.asInstanceOf[AsyncHttpClient]

  def url(url: String): MultiPartWSRequestHolder = {
    val builder = client.preparePost(url)
    builder.setHeader("Content-Type", "multipart/form-data")
    MultiPartWSRequestHolder(client, builder, url)
  }
}

It can be used i..e:

val multiPartClient = MultPartWSClient(client)
    multiPartClient.url(url)
      .withParts(StringPart("title", title), 
          StringPart("someAttr1", attr1),
          StringPart("someAttr2", attr2),
          StringPart("someAttr3", attr3),
          FilePart("document", document, contentType, "UTF-8"))
      .withMethod("POST")
      .execute()

It is still not fully implemented, but can be of use to anyone. Feel free to integrate that into play or use it in your own project.

[nafg]

Simpler solution, based on http://stackoverflow.com/a/18723326/333643:

(I'm calling the mailgun API)

// Use the Ning AsyncHttpClient multipart class to get the bytes
val parts = Array[Part](
      new StringPart("from", from),
      new StringPart("to", to),
      new StringPart("subject", subject),
      new StringPart("text", text),
      new FilePart("attachment", file)
    )
    val mpre = new MultipartRequestEntity(parts, new FluentCaseInsensitiveStringsMap)
    val baos = new ByteArrayOutputStream
    mpre.writeRequest(baos)
    val bytes = baos.toByteArray
    val contentType = mpre.getContentType

// Now just send the data to the WS API
    client.url(endpoint)
      .post(bytes)(Writeable.wBytes, ContentTypeOf(Some(contentType)))

I would have liked to be able to create implicit Writeable and ContentTypeOf instances for MultipartRequestEntity. Unfortunately however, their API is based on the assumption of one content type per typeclass instance (i.e. it would have to be the same for all MultipartRequestEntity instances).

However since it has to include the boundary value, this assumption is broken. If the typeclass could wrap an A => Option[String] rather than just an Option[String], we could just have an implicit Writeable[MultipartRequstEntity] etc.

[naderghanbari]

Is there any plan to fix this? this will simplify things to a great extent (without this supported out of the box, sending a multipart request is very tedious).

[cdmckay]

I wrote a Writeable to use the existing MultipartFormData object for POSTing multipart form data:

https://gist.github.com/cdmckay/4b269e9017a30111556a

And here's an example of usage:

https://gist.github.com/cdmckay/5f05cbcb5327259df1cf

So you just need to drop it in and it should work with the existing WS flow.

[brianwawok]

@cdmckay @nafg @toggm do you have a version that works in play 2.4? I had my own home rolled attempt, but so far all my multipart form posts using play get back a "The request body is too large" from my API I am hitting..

[david-bouyssie]

@cdmckay : is there a solution to observe the progression of the upload ?
Currently I'm using scalaj-http for this purpose (https://github.com/scalaj/scalaj-http) but I would like to switch to Play WS in order to avoid extra dependencies and use asynchronous requests.
Thanks !

[ergomesh]

@brianwawok

I have a similar problem I am migrating from 2.3 to 2.4 and get this now

Compilation error[object multipart is not a member of package com.ning.http] using the@cdmckay example. Anyone have this working in play2.4.x ?

[brianwawok]

@ergomesh yes I was able to make this work in 2.4. I had to forcibly override my async http client version to 1.9.29, and then I did:

https://gist.github.com/brianwawok/eb9ac542eb7b4795ac06

A few things like

"Content-Type" -> mpre.getContentType)

were key, because I needed to add back in the boundary (which was lost in the play upgrade)

[mkurz]

Could someone provide a pull request? This would be great!

[brianwawok]

@mkurz i have it in a hack state with no tests, not sure I am going to have time to clean up into a proper PR

[mkurz]

@brianwawok If you could find some time to submit that PR that would be really awesome and would make some people really happy! 😄

[mkurz]

@brianwawok If you do some work on this and provide a PR I could maybe help out testing things.

[jakob85]

Not sure if you guys fixed the issue but I answered my own question at stackoverflow and got it working thanks to all posts here =)
http://stackoverflow.com/questions/35325942/timeout-when-sending-multipart-form-data-in-play2-2-4-6

[mkurz]

@jakob85 If you are keen to provide a pull request with implements this feature directly in Play you are welcome to do so.

[david-bouyssie]

A solution including the monitoring of upload progression would be really awesome.
This feature is provided by this library: https://github.com/scalaj/scalaj-http

[schmitch]

Actually all the solutions here would be missing the ending, i.e:
Something like that:
--BRY7v0J3h3fIl91lUeQ6SxMJWqNHbRYi4Exp2C3s--

What would actually work is using the asyncHttpClient.preparePost().addBodyPart funtionallity i.e.:

val asyncHttpClient: AsyncHttpClient = ws.client.underlying

        val postBuilder = asyncHttpClient.preparePost(url).addBodyPart(filePart).addBodyPart(textPart)
          .addHeader(AUTHORIZATION, s"Basic $auth")
        val request = postBuilder.build()

        val result = Promise[NingWSResponse]()
        asyncHttpClient.executeRequest(request, new AsyncCompletionHandler[AHCResponse]() {
          override def onCompleted(response: AHCResponse) = {
            result.success(NingWSResponse(response))
            response
          }

          override def onThrowable(t: Throwable) = {
            result.failure(t)
          }
        })
        result.future

Not sure if a Writeable has access to that. I'm looking forward to maybe add a PR.

[schmitch]

Actually it would be great if somebody could help me with the PR on tests.

Actually I've left out the documentation since I'm not sure if it's good to either have something like .post(Seq(part1, part2)) and on the Java part a List. Actually maybe somebody from the core team could give me some hints.

Edit: Too bad this prolly won't go into 2.5. Actually I came too late across this.

[mkurz]

Maybe we can can still get this into 2.5?

[gmethvin]

@mkurz At this point there should be no problem getting it into 2.5. It would only be an issue if there are major API changes.

[mkurz]

@gmethvin Cool! 😄

[schmitch]

That would be great. I actually came across this since we are sending data to another provider which uses insecure SSL, so using the underlying AhcClient takes more work than just fixing this issue..

[david-bouyssie]

Could it be possible to have access to onContentWriteProgress callback or is it too complicated to interface it with the Play API ?
https://asynchttpclient.github.io/async-http-client/apidocs/com/ning/http/client/AsyncCompletionHandler.html#onContentWriteProgress%28long,%20long,%20long%29

This would be a very useful feature !

[david-bouyssie]

Do you have any tip to monitor the upload progress ?
Do I have to use the underlying client to achieve that ?
I was thinking about using reactive streams to be informed periodically about the upload progress update.
Thanks !

[ilinandrii]

Is there any workaround for multipart/form-data for play 2.5?

[schmitch]

it will when 2.5.1 hit @AndreyIlin

[ilinandrii]

Thanks @schmitch, waiting.