How do I get the parsed length in on_message_complete callback?

[doraxcyle]

How do I get the parsed length in on_message_complete callback?

[indutny]

I don't think there is a way to do it right now. Are you looking for parsed length of whole request (including method, url, headers and so on)?

[doraxcyle]

If I change the callback(on_message_complete(llhttp_t* parser)) to this(on_message_complete(llhttp_t* parser, const char* at, std::size_t length)), I can get the length (at - begin). But that's not necessary.

[davidhcefx]

My input is a stream of bytes, and I want to know the span of the whole request when I'm in the on_message_complete callback. I wonder if there is a way to achieve this without modifying the llhttp code?

My first thought was to hook on_header_value and on_body to update a pointer always pointing to the last byte. However, this would not cover the case when there is only a single REQUEST-LINE and no headers at all, because there is no callback for that. Another approach would be scanning and partitioning the input by CRLF, and feed each segments separately to the parser, but this doesn't seems to be an elegant way. Do you have any thoughts on this?

[indutny]

Here's an idea. You can store a pointer to your structure in the ->data field of the parser, and among other things store the number of bytes parsed since the on_message_begin called. Since the data in callbacks is always relative to the input pointer that was given to the llhttp_execute() - it is always possible to compute at what input position each callback was invoked. You could use error_pos or just input size (depending on llhttp_execute() return value) to move by whole chunks forward.

I'm not entirely against adding a field like that to the parser, but each such addition will slow it down for users that don't need this feature and implementing this feature in user's code seems generally not to be too complicated.

[davidhcefx]

Thanks for your instant feedback!
I understand this code is for Node.js mainly, so I also don't prefer of adding redundant stuff. In dealing with the problem, I have tried llhttp_get_error_pos(), but it seems to return a non-null value only when an error occurred. After hacking around, I found modifying the code the easiest way of solving it. So in case someone is interested, here is my diff:

diff --git a/src/native/api.c b/src/native/api.c

@@ -138,7 +138,7 @@ llhttp_errno_t llhttp_finish(llhttp_t* parser) {
 
   switch (parser->finish) {
     case HTTP_FINISH_SAFE_WITH_CB:
-      CALLBACK_MAYBE(parser, on_message_complete);
+      SPAN_CALLBACK_MAYBE(parser, on_message_complete, "", 0);
       if (err != HPE_OK) return err;
 
     /* FALLTHROUGH */
@@ -322,7 +322,7 @@ int llhttp__on_headers_complete(llhttp_t* s, const char* p, const char* endp) {
 
 int llhttp__on_message_complete(llhttp_t* s, const char* p, const char* endp) {
   int err;
-  CALLBACK_MAYBE(s, on_message_complete);
+  SPAN_CALLBACK_MAYBE(s, on_message_complete, p, endp - p);
   return err;
 }

diff --git a/src/native/api.h b/src/native/api.h

@@ -42,7 +42,7 @@ struct llhttp_settings_s {
   llhttp_data_cb on_body;
 
   /* Possible return values 0, -1, `HPE_PAUSED` */
-  llhttp_cb      on_message_complete;
+  llhttp_data_cb on_message_complete;
 
   /* When on_chunk_header is called, the current chunk length is stored
    * in parser->content_length.

After that, the at parameter in the new callback int on_message_complete(llhttp_t*, const char *at, size_t length) would point next to the last byte of the previous request. Example:

HEAD / HTTP/1.1\r\n\r\nGET / HTTP/1.1\r\nHost: www.google.com\r\nAccept: */*\r\n\r\nPUT / HTTP/1.1\r\nContent-Length: 10\r\n\r\nhelloworldPOST / HTTP/1.0
at: "GET / HTTP/1.1..."
at: "PUT / HTTP/1.1..."
at: "POST / HTTP/1.0"