Simulate httputil.DumpResponse

[FrankIT60]

Hello, I'm new to echo.
I can simulate or use httputil.DumpResponse in echo?

Thanks in advance
Franco

[vishr]

Looking at https://golang.org/pkg/net/http/httputil/#example_DumpResponse, yes you can. Are you using it for testing?

[FrankIT60]

I write this example.
DumpRequest works but DumpResponse go in panic.
Can you help me with example of middleware for print DumpRequest and DumpResponse?

T.i.a.
Franco

package main 
import (
    "fmt"
    "net/http"
    "net/http/httputil"

    "github.com/labstack/echo"
    "github.com/labstack/echo/middleware"
 )

 func hello(c echo.Context) error {
    dumpRequest, err := httputil.DumpRequest(c.Request(), true)
    if err != nil {
        fmt.Println(err.Error())
        return err
    }
    c.String(http.StatusOK, string(dumpRequest))
    resp := c.Request().Response

    dumpResponse, err := httputil.DumpResponse(resp, true)
    if err != nil {
        fmt.Println(err.Error())
        return err
    }
    fmt.Println(dumpResponse)
    return nil
}

func main() {
// Echo instance
e := echo.New()

// Middleware
e.Use(middleware.Logger())
e.Use(middleware.Recover())

// Routes
e.GET("/", hello)

// Start server
e.Logger.Fatal(e.Start(":1323"))
}

[vishr]

I don't think you are doing it right. What do you want to achieve? I think DumpRequest is for servers to debug incoming request and likewise DumpResponse is for clients. You already know what response you are sending to the client?

[FrankIT60]

Yes I know. I can save a copy of response. But in case of controversy with the client regarding the content it is better to have a track produced by an effective API rather than something written by me.

[vishr]

Related #658, #910

How about exposing Context#Received() & Context#Sent() for storing request and response body with limit? Will it be useful? How will it affect memory usage?

/cc @Anon-Penguin @ansel1 @mtojek @heynemann @montanaflynn

[montanaflynn]

@vishr we're already doing this with for the request body to use it in logging middleware, all of our json request and response data is small and would fit in a 1kb limit so I don't think it would affect memory terribly.

[FrankIT60]

I think the better is exposing Context#Received() & Context#Sent() with parameter for include o esclude body. When there are disputes with other vendors using APIs at the same customer, it is important to be able to demonstrate what is actually being transmitted. Not always but often happens.
Now we use Apache with mod_dumpio.
But if we want to replace with microservices written in Go we have to guarantee the same possibility.
The best would be to enable or disable this option at run-time, for the single API.
This is a possibility that, in my research, I did not find in any Go Web Framework.

[vishr]

I am not wrong the main idea of this requirement is to log messages with request and response body, so make a change to logger middleware should do it? I am planning to include optional request_body and response_body tags to the template and if they are set, dump the body via logger to the output.

[mathvav]

Although adding tags to the logger would be handy, it could easily become an issue:

• Formatting issues, since HTTP bodies take up more than 1 line
• stdout seems to be limited on some systems in terms of speed (compared to writing straight to disk)

Although I'm not against adding these tags, I feel like doing this in custom middleware might be more useful to most people.

Should we add something to the docs for this use case? (Do we have analytics for if people even read the docs?)

[vishr]

@Anon-Penguin If people are using body tags then I would expect them to redirect logs to a file or external system and not display on the terminal. Custom middleware is also an option.

Do we have analytics for if people even read the docs?

Yes, about 500 users / day.

[FrankIT60]

The logging system is as good as it is.
I vote for a specific and dedicated middleware.

[vishr]

I am ok with a new middleware, it is just that there may be overlap of some of the fields/tags.

Questions:

• Middleware name (BodyDump, DumpIO, DumpBody???)
• What information do we want to capture (request body, response body???)
• JSON format is OK?

[mathvav]

Maybe it could be named Interceptor?

---

Proposal for implementation:

The configuration for the middleware is this a "handler" function (such as func(c echo.Context, requestBody, responseBody []byte)). The user could print to stdout, or do a log, or whatever they want inside the handler.