Response body gets cut off

[bohdankoshyrets]

When trying to get all the data from response body with DataReader it works fine when receiving small amount of data, but when trying to get a bigger body response, it doesn’t return the whole data, but only a part of it. I could fix it by increasing recvChunkSize to the size bigger that I was trying to read. But seems like the problem is in chunking itself, but I wasn’t able to debug it more deeply.

Has anyone encountered this kind of problem?

[fangpenlin]

Hey @bohdankoshyrets, thanks so much for reporting this issue, do you mind provide a sample code that can reproduce that problem? It would be helpful for me to dig deeper see what's going on there, but if not, it's okay. I will find a time to look into this.

[bohdankoshyrets]

Hey, @fangpenlin, thanks for quick response, the router code I'm using is:

        self["/sample"] = JSONResponse { request, response in
            let input = request["swsgi.input"] as! SWSGIInput
            DataReader.read(input) { data in
                print(data)
                let readingOtions: JSONSerialization.ReadingOptions = [.allowFragments]
                do {
                    let jsonObject = try JSONSerialization.jsonObject(with: data, options: readingOtions) as! JSON
                    print(jsonObject)
                } catch let error as NSError {
                    print(error.description)
                }
            }
            response(["status": 200])
        }

It works just fine with the request, which has total size (including headers) less than 1024B, every request bigger than that gets its body data cut. And when it happens, i get a JSON parsing error with something like this:
["Error Domain=NSCocoaErrorDomain Code=3840 \"Unterminated string around character 693.\" UserInfo={NSDebugDescription=Unterminated string around character 693.}"]

Please let me know if you need any other information.

[fangpenlin]

Ah, I see. Ambassador and Embassy are async http server, which means all the things are done in async manner. For swsgi.input, it's designed to not to read the body until you want to, which is, passing in a callback to it.

And as the nature of HTTP, if you respond anything, usually it means you've already read all the header and payload you need (not necessary true in all cases I guess). So in the sample code

response(["status": 200])

is called immediately after the call to DataReader.read, but as DataReader.read is also an async operation, it won't block there and wait until all the body data is read.

So, as a result, you will see Embassy is trying to read more data from DataReader.read, but since you call response already, the HTTP connection is then closed, so no more data will be read even when there's more

To solve the problem, you will need to move the call to response inside DataReader.read callback like this

        self["/sample"] = JSONResponse { request, response in
            let input = request["swsgi.input"] as! SWSGIInput
            DataReader.read(input) { data in
                print(data)
                let readingOtions: JSONSerialization.ReadingOptions = [.allowFragments]
                do {
                    let jsonObject = try JSONSerialization.jsonObject(with: data, options: readingOtions) as! JSON
                    print(jsonObject)
                    response(["status": 200])
                } catch let error as NSError {
                    print(error.description)
                }
            }
        }

In this way, the response will only be called when reading data is finished.

[bohdankoshyrets]

Wow, such a silly mistake on my part, I moved response inside DataReader.read and it worked. Thank you, Fang-Pen Lin!

[fangpenlin]

@bohdankoshyrets no worries, it's not that intuitive to see, as it's all async, I should have improved README to emphasize this. Or maybe provide easier helper for most of easy simple req/rep usage