New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: new HTTP client package #23707

Open
bradfitz opened this Issue Feb 5, 2018 · 3 comments

Comments

Projects
None yet
3 participants
@bradfitz
Member

bradfitz commented Feb 5, 2018

Tracking bug for higher-level more usable HTTP client functionality.

  • take contexts
  • more timeout control per-request, rather than per-transport/client
  • ...Options pattern?
  • decode into JSON/etc easily?
  • treat non-2xx as error (by default?).
  • slurp entire response from server (by default?)
  • control over max response size (some sane default limit when slurping is enabled)

I thought we had a bug for this, but maybe not.

@pciet

This comment has been minimized.

Contributor

pciet commented Feb 5, 2018

This is a pattern in a test client I wrote:

// or Post, PostForm
r, err := client.Get(addr)
if err != nil {
    panic(err)
}
// do some parsing here then discard the rest
_, err = io.Copy(ioutil.Discard, r.Body)
if err != nil {
    panic(err)
}
err = r.Body.Close()
if err != nil {
    panic(err)
}
if r.StatusCode != http.StatusOK {
    panic(r)
}

Are you looking at adding new identifiers?

type BodyParser func(io.Reader) error

// http.ErrBadStatus for non-2xx, discards unread response body, and closes the connection.
func (c *Client) ClosingGet(url string, resp BodyParser) error

// repeated for the other methods
var t MyStruct
err := c.ClosingGet(addr, http.JSONBodyParser(&t))
@bradfitz

This comment has been minimized.

Member

bradfitz commented Feb 5, 2018

Are you looking at adding new identifiers?

There would be new API for this, yes. I don't have a concrete proposal yet.

@awreece

This comment has been minimized.

Contributor

awreece commented Mar 26, 2018

We wrote a wrapper for the http client internally that uses the ...Options pattern and achieves a lot of these goals:

https://github.com/AlphaFlow/gohttp

GET with parameters:

cli := http.NewClient()

var ret map[string]interface{}
err := cli.Get(context.Background(),
    "http://example.com",
    http.WithParam("debug", "1),
    http.WithJSONResponse(&ret))

POST with body:

ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()

body = map[string]string{
   "hello": "world",
}
err := cli.Post(ctx, "http://example.com",
    http.WithJSONBody(body),
    http.WithHeader("Authorization", "Bearer my-token"))
if bse, ok := err.(*http.BadStatusError); ok && bse.Code == 404 {
    fmt.Println(bse)
}

Mocking the HTTP request:

	return http.NewMockClient(func(ctx context.Context, req *http.Request) error {
		if e != nil {
			return e
		}
		if len(body) == 0 {
			return nil
		}
		if req.Output != nil {
			_, err := req.Output.Write(body)
			return err
		} else if req.JSONOutput != nil {
			return json.Unmarshal(body, req.JSONOutput)
		} else {
			panic("no place to write response")
		}
	})

This has greatly simplified our error handling/json marshaling.

Quirks

  • Mocking the client using this interface is a bit weird -- the mock needs to know an awful lot about how the code works. This could be simplified a bit, maybe.
  • We inherit standard go http client "follow redirect" idiom, when it would be nice for that to be an option.

@bradfitz bradfitz modified the milestones: Go1.11, inpl, Unplanned May 23, 2018

@bradfitz bradfitz changed the title from net/http: add high-level HTTP client operations to net/http: new HTTP client package Dec 3, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment