High CPU utilization when uploading a file

[nsleader]

This code uses 99-100% CPU:

drop.post("upload") { request in
    guard let file = request.multipart?["zip"]?.file, let name = file.name else {
        throw Abort.badRequest
    }

    try Data(bytes: file.data).write(to: URL(fileURLWithPath: "/tmp/\(name)"))
    return "OK"
}

How to optimize uploading file?

Update:

After upgrading Vapor to version 1.2 uploading time reduced to 10 seconds.
The server and client running on one local machine (MacBook Pro).

Tested code:

drop.post("upload") { request in
    // No work here. Only Vapor testing.
    return "OK"
}

POST /upload HTTP/1.1
Content-Type: multipart/form-data; charset=utf-8; boundary=__X_PAW_BOUNDARY__
Host: 0.0.0.0:8080
Connection: close
User-Agent: Paw/3.0.12 (Macintosh; OS X/10.12.1) GCDHTTPRequest
Content-Length: 37379899

--__X_PAW_BOUNDARY__
Content-Disposition: form-data; name="ipa"; filename="file.ipa"
Content-Type: application/octet-stream

PK��
�������yI����������������Payload/UX
...

File size: 37379899 bytes
vapor 1.1.x: 110 seconds (Very long time!)
vapor 1.2: 10 seconds (Still a long! It's just a copying file on the local machine)

[andreasley]

One of the culprits seems to be this line in TCPSocket.swift: https://github.com/vapor/socks/blob/bd9af15409002dbc924b92f9220788c25c278d53/Sources/SocksCore/TCPSocket.swift#L47

Replacing the line

let out = Array(finalBytes.map({ UInt8($0) }))

with

let out = Array(finalBytes)

improves time to about 1.5 seconds.

[nsleader]

New tests

File size: 20585454 bytes
Vapor (1.3.1): 113 seconds (Something went wrong :()
Code:

drop.post("upload") { request in
    guard let file = request.multipart?["zip"]?.file, let name = file.name else {
        throw Abort.badRequest
    }
    try Data(bytes: file.data).write(to: URL(fileURLWithPath: "/tmp/\(name)"))
    return "OK"
}

Perfect (2.1.8): 605 milliseconds !!! (Perfect! :))
Code:

routes.add(method: .post, uri: "/upload") { (request, response) in
    
    let fileDir = Dir(Dir.workingDir.path + "uploads")
    do {
        try fileDir.create()
    } catch {
        print(error)
    }
    
    if let uploads = request.postFileUploads, uploads.count > 0 {
        for upload in uploads {
            let thisFile = File(upload.tmpFileName)
            do {
                let _ = try thisFile.moveTo(path: fileDir.path + upload.fileName, overWrite: true)
            } catch {
                print(error)
            }
        }
    }
    
    response.appendBody(string: "OK")
    response.completed()
}

[tanner0101]

@nsleader we're seeing multiple issues related to multipart parsing. We're working on them now.

Until then, you can use the raw request.body as a workaround for accessing uploaded files.

[loganwright]

Closing since we have a workaround and multipart is an ongoing discussion. It should be noted that there are third party multipart libraries that seem quite sufficient!