MG_EV_HTTP_CHUNK Questions

[pmeskoni]

MG_EV_HTTP_CHUNK Questions

I am migrating from 6.x to 7.x, specifically addressing the multipart uploads. Our 6.x code processed all the events to deal with multipart and worked well to upload very large files. I am unsure of the best way to migrate to 7.x in this area.

I attempted these 2 things:

1. Keep my MG_MAX_RECV_BUF_SIZE small and process large uploads 1 chunk at a time (similar to 6.x model), or
2. Resize the receive buffer once via mg_iobuf_resize() instead of several times as the large file was received

To attempt the first one, I processed MG_EV_HTTP_CHUNK and did this each time through:

• Copy message->chunk.len bytes out of message->chunk.ptr
• mg_http_delete_chunk(conn,message)

This gave me my entire file, but when the next file was sent, the entire message was treated as data of the first file by Mongoose. It seemed I didn't properly tell Mongoose code that the file was done or interfered in that somehow. Is there something more I need to do when processing each chunk than just calling delete_chunk?

To attempt the second item above, I wanted to process the "first" MG_EV_HTTP_CHUNK for each file sent.
I did this by:

• Check the chunk.len, the first chunk I saw for each file contained headers and a chunk size of 0
• If it was 0, I found my "filesize" header and resized the receive buffer once via mg_iobuf_resize() to fit the entire file

This seems to work for me, despite the documentation stating that the LAST chunk will have a size of 0, but says nothing of the first. However, this ultimately has a file size limit of MG_MAX_RECV_BUF_SIZE.

Thanks,
PM

[cpq]

Is there something more I need to do when processing each chunk than just calling delete_chunk? - send response when the last chunk is received. See https://github.com/cesanta/mongoose/blob/master/examples/file-upload/main.c

entire message was treated as data of the first file by Mongoose - I did not get this question. Could you expand on this please?

[pmeskoni]

Ah, yes. I wasn't sending the response. The file-upload example bundled with 7.6 did not have that code. I modeled the code after the streaming example which just disconnects when the the streaming is complete. The "entire message was treated..." comment is the result of not sending the response at the proper time, so this is taken care of.

I am still interested in your thoughts about the buffer resizing approach I'm using. (the second part of my original message)
Thanks,
PM