Issue with Purging Cache for API Responses in cloudflare-go library

[shinngohamatani]

Confirmation

• My issue isn't already found on the issue tracker.
• I have replicated my issue using the latest version of the library and it is still present.

cloudflare-go version

v0.73.0

Go environment

❯ go env                                       
GO111MODULE="on"
GOARCH="amd64"

Expected output

I'm utilizing the Cloudflare v4API, specifically the Zone > Purge Cached Content API. I believe there's a files entry in the POST body during this operation. In this context, the files don't represent static files, but APIs whose responses are application/json.

Let's take an API endpoint like /api/v1/users for example. Suppose we only want to fetch users updated after a specific time, we might use a URL such as /api/v1/users?size=20&updated_at=2023-08-02T19%3A30%3A00%2B09%3A00. While this might not seem typical for an API, suppose we want to purge the cache for this /api/v1/users?size=20&updated_at=2023-08-02T19%3A30%3A00%2B09%3A00 endpoint.

In this case, we make a POST request to /client/v4/sonze/{identifier}/purge_cache, and I assume that the body should look something like this:

{
    "files": [
        "https://cdn.example.com/api/v1/users?size=20&updated_at=2023-08-02T19%3A30%3A00%2B09%3A00"
    ]
}

In Go, it can be described as follows:

	api, _ := cloudflare.NewWithAPIToken("***")
	_, _ = api.PurgeCacheContext(context.Background(), "***", cloudflare.PurgeCacheRequest{
		Files: []string{"https://cdn.example.com/api/v1/users?size=20&updated_at=2023-08-02T19%3A30%3A00%2B09%3A00"},
	})

However, when I actually try to execute this, json.Marshal seems to be performed, which causes the '&' to be escaped, preventing the cache from being purged successfully.

{
    "files": [
        "https://cdn.example.com/api/v1/users?size=20\u0026updated_at=2023-08-02T19%3A30%3A00%2B09%3A00"
    ]
}

I have a feeling that a modification to the library may be needed, but there's also the possibility that I'm using it incorrectly. If that's the case, could you please enlighten me with the correct usage in detail?

Actual output

When trying to purge the cache using the Cloudflare Go library, I've been encountering an issue. When trying to purge cache for an API endpoint with URL parameters, the '&' character in the URL is being escaped when converted into JSON. This escaping is preventing the cache from being successfully purged.

Code demonstrating the issue

	api, _ := cloudflare.NewWithAPIToken("***")
	_, _ = api.PurgeCacheContext(context.Background(), "***", cloudflare.PurgeCacheRequest{
		Files: []string{"https://cdn.example.com/api/v1/users?size=20&updated_at=2023-08-02T19%3A30%3A00%2B09%3A00"},
	})

Steps to reproduce

above

References

No response

[jacobbednarz]

thanks for the thorough issue!

i'm able to confirm this is a doing of stdlib handling of HTML characters in Marshal method - https://github.com/golang/go/blob/master/src/encoding/json/encode.go#L162. example: https://go.dev/play/p/sYrB0pXsduD

unfortunately, the only option is to side step the Marshal interface completely and implement your own custom method for handling the encoding via Encode. in our case, something like this would work.

+func (t *PurgeCacheRequest) JSON() ([]byte, error) {
+       buffer := &bytes.Buffer{}
+       encoder := json.NewEncoder(buffer)
+       encoder.SetEscapeHTML(false)
+       err := encoder.Encode(t)
+       return buffer.Bytes(), err
+}
+
 // PurgeCacheRequest represents the request format made to the purge endpoint.
 type PurgeCacheRequest struct {
        Everything bool `json:"purge_everything,omitempty"`
@@ -672,7 +681,8 @@ func (api *API) PurgeCache(ctx context.Context, zoneID string, pcr PurgeCacheReq
 // API reference: https://api.cloudflare.com/#zone-purge-individual-files-by-url-and-cache-tags
 func (api *API) PurgeCacheContext(ctx context.Context, zoneID string, pcr PurgeCacheRequest) (PurgeCacheResponse, error) {
        uri := fmt.Sprintf("/zones/%s/purge_cache", zoneID)
-       res, err := api.makeRequestContext(ctx, http.MethodPost, uri, pcr)
+       payload, _ := pcr.JSON()
+       res, err := api.makeRequestContext(ctx, http.MethodPost, uri, payload)
        if err != nil {
                return PurgeCacheResponse{}, err
        }

it's not a great solution though because we would have to do this on every instance where we may expect HTML characters to be in JSON payloads.

i might have a look at if we have any comparable JSON libraries that don't implement this and we can swap it out globally.

[github-actions]

This functionality has been released in v0.75.0.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!