Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Support gzip compression #66

Open
rickb777 opened this issue Oct 22, 2020 · 3 comments
Open

Enhancement: Support gzip compression #66

rickb777 opened this issue Oct 22, 2020 · 3 comments

Comments

@rickb777
Copy link

rickb777 commented Oct 22, 2020

The Salesforce API allows gzip compression on either request entities or response entities (or both). It's not to hard to support this by using the necessary Content-Encoding and Accept-Encoding headers and then passing the json marshalling through compress/gzip functions.

The only method affected is ForceApi.request, I think.

@rickb777
Copy link
Author

rickb777 commented Oct 22, 2020

BTW in my experience, low compression levels are better than the defaults for this kind of use case; level 4 is used in the suggested code here. This amends ForceApi.request:

	// Build body
	var body io.Reader
	if payload != nil {
		buf := &bytes.Buffer{}

		gbuf, err := gzip.NewWriterLevel(buf, 4)
		if err != nil {
			return fmt.Errorf("Error compressing encoded payload: %v", err)
		}

		err = json.NewEncoder(gbuf).Encode(payload)
		if err != nil {
			return fmt.Errorf("Error marshaling encoded payload: %v", err)
		}

		body = buf
	}

	// Build Request
	req, err := http.NewRequest(method, uri.String(), body)
	if err != nil {
		return fmt.Errorf("Error creating %v request: %v", method, err)
	}

	// Add Headers
	req.Header.Set("User-Agent", userAgent)
	req.Header.Set("Content-Type", contentType)
	req.Header.Set("Content-Encoding", "gzip")
	req.Header.Set("Accept-Encoding", "gzip")
	req.Header.Set("Accept", responseType)
	req.Header.Set("Authorization", fmt.Sprintf("%v %v", "Bearer", forceApi.oauth.AccessToken))

	// Send
	forceApi.traceRequest(req)
	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		return fmt.Errorf("Error sending %v request: %v", method, err)
	}
	defer resp.Body.Close()
	forceApi.traceResponse(resp)

	// Sometimes the force API returns no body, we should catch this early
	if resp.StatusCode == http.StatusNoContent {
		return nil
	}

	body = resp.Body
	if resp.Header.Get("Content-Encoding") == "gzip" {
		body, err = gzip.NewReader(resp.Body)
		if err != nil {
			return fmt.Errorf("Error decompressing %v response: %v", method, err)
		}
	}
	respBytes, err := ioutil.ReadAll(body)
	...  etc

The new bits of code would probably be best in new helper methods instead of inline.

@rickb777
Copy link
Author

I would have made a PR but when I checked out the code I got many test failures.

@rickb777
Copy link
Author

BTW the suggested code above uses the standard encoding/json package instead of the forcejson package. I'm not clear why forcejson exists, but if it's really needed it should have NewEncoder and NewDecoder functions like the standard library does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant