Data or File Data Member of Decodable Parsed Incorrectly on Linux

[c1wren]

Short backstory, I pushed a new version of my backend a few days ago after testing it on my Mac extensively first, including file/photo upload. After pushing it and making sure things were still working, I discovered that file upload was broken everywhere. I boiled the issue down to an update in multipart to 3.0.4 from 3.0.3 that was the cause of the issue.

This issue at least exists on Linux running Swift 4.2 and later.

Pictures and files were being uploaded but the files saved were not what was uploaded. The files being saved included part of the actual file data but also included the parts of the multipart requests including the boundaries. The file/picture data portion is always cut off and never complete.

For example, I have a request that includes an integer and then a file for the other part. The file from the request once it has been decoded would look like:

--__X_PAW_BOUNDARY__
Content-Disposition: form-data; name="id"

10
--__X_PAW_BOUNDARY__
Content-Disposition: form-data; name="image"; filename="image.png"
Content-Type: image/png

âPNG
�
���
IHDR���Y���>�����@˙∑M���ziCCPICC Profile��(ë}êM+DQ�«�3à�çba°‹º≠Ü�%6 L�S≥ò�ÂmsÁö�53nwÆêçÖ≤Uîÿx[	ÿX(k•�)Ÿ¯�ƒF∫ûchºîߌ9øÛúÁ˘˜ú?∏}∫ifJ;!õ≥≠X(†ççOhÂ�∏RC#•∫ë7�¢—��_Áœxπñjâ´v•ı˜˝fl®öN‰
pU�˜�¶e��	7œ€¶b•Wg…P¬ÀäS�fiP�/—GÕH,(|*¨�i}Z¯Nÿg§≠,∏ï~K¸[MÍ�g3s∆Á<Í'ûDntXŒ&Y
‰â�"ÄFòAÇÙ–EüÏ=¥„ßCnÿâ�[5�gÕEk&ï∂µ�q"°ÖsFáOÛwvıÅÚı∑_≈‹Ï.Ù>C…Z1�flÑìU®ø-ÊZv¿ª�«Á¶nÈ�©�YÓd���°z�j/°r2üψ�~‰	@ŸΩ„<µB˘:º≠9ŒÎû„ºÌK≥xtñ+xÙ©≈¡
å,A‰�∂∂°M¥ΩSÔfi¥g�R‰¿����	pHYs��.#��.#�x•?v����iTXtXML:com.adobe.xmp�����<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
   <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
      <rdf:Description rdf:about=""
            xmlns:xmp="http://ns.adobe.com/xap/1.0/"
            xmlns:tiff="http://ns.adobe.com/tiff/1.0/"
            xmlns:xmpMM="http://ns.adobe.com/xap/1.0/mm/"
            xmlns:stRef="http://ns.adobe.com/xap/1.0/sType/ResourceRef#"
            xmlns:stEvt="http://ns.adobe.com/xap/1.0/sType/ResourceEvent#"
            xmlns:exif="http://ns.adobe.com/exif/1.0/"
            xmlns:dc="http://purl.org/dc/elements/1.1/"
            xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/">
         <xmp:ModifyDate>2018-12-24T10:05:48-06:00</xmp:ModifyDate>
         <xmp:CreateDate>2012-01-16T10:46:07-06:00</xmp:CreateDate>
... (there's more just I'm not including it here)

I'll also include a file that is exactly what is saved when decoded from multipart 3.0.4 that was running on my server. If you open the file, you can see that the image portion of the request is cut off and incomplete.

Here's the struct I was using:

struct BackflowPayload: Decodable {
     var image: File
     var id: Int
 }

I also tried this struct to see if there was a difference:

    struct BackflowPayload: Decodable {
        var image: Data
        var id: Int
    }

I did do some testing to see if I could find the exact cause. The image size in bytes is reported correctly on Linux when compared against the backend running on my Mac. I think the data start offset that is copied into the respective data member is incorrect for parts of type Data or File on Linux. My guess is that the number of bytes taken up by the other parts of the multipart requests is why the data/file portion being saved is cut off.

test.txt

[tanner0101]

I'm having trouble re-creating this on Linux. Here's what I tried:

struct Payload: Content {
    var image: File
    var id: Int
}

router.post("multipart-file") { req -> Future<Response> in
    return try req.content.decode(Payload.self).map { payload in
        let res = req.response()
        res.http.contentType = .png
        res.http.body = .init(data: payload.image.data)
        return res
    }
}

[c1wren]

I'm working on a way of reproducing the issue without exposing my codebase.

[c1wren]

Not a multipart issue after all. Pretty sure it is a Foundation issue on linux with URLSession.