Skip to content

[FEATURE] Honor stackql's --http.log.enabled in the GraphQL acquire path #103

@jeffreyaven

Description

@jeffreyaven

Two issues in pkg/graphql/graphql.go

  1. --http.log.enabled only surfaces the post-transform response. The wire request body (rendered GraphQL query) and the naked pre-transform response body are still swallowed. Both are essential for debugging request templating and response-transform bugs.
  2. Typos in error messages. At least cannot accomodate GraphQL pocessed response of type = ... (should be "accommodate" and "processed"). Also sweep neighbouring files in the same package for sibling typos.

Both belong in the same place (pkg/graphql/graphql.go and its tests). One PR, two contained changes.

Issue 1 - missing request body + naked response in --http.log.enabled

Summary

In the current build, --http.log.enabled shows the post-jsonpath, post-transform projection result for a GraphQL operation. Example output from stackql 0.10.489 against a working Cloudflare GraphQL operation:

stackql >> SELECT datetime, requests, bytes
        >> FROM cloudflare.zones.zone_http_requests_adaptive_groups
        >> WHERE zone_tag = '...' AND since = '...' AND until = '...';
processed http response body object: [
  {
    "bytes": 14396,
    "client_country_name": "AU",
    "datetime": "2026-05-28T19:45:00Z",
    "requests": 30,
    ...
  },
  ...
]

That is genuinely useful (the original framing of this issue, drafted against an older build, was wrong to claim the log line was empty - it surfaces the projected rows correctly when the transform succeeds). But two pieces are still missing that are critical for diagnosing transform / templating / dispatch issues:

  1. The wire request body - the rendered GraphQL query string that stackql actually POSTed. Without this we cannot tell whether {{ .limit }} rendered to 5 (good), <no value> (broken template), or anything else.
  2. The naked pre-transform response body - the raw JSON Cloudflare (or whichever GraphQL backend) returned, before stackql ran the response.transform. Without this we cannot tell whether the transform crashed because Cloudflare returned an error envelope ({"data": null, "errors": [...]}), an empty dataset, or an unexpected shape.

Both classes of failure cost real debugging time today. Concrete recent examples we hit while developing the Cloudflare provider:

  • Templating bug: the spec's limit: {{ .limit }} rendered to limit: <no value> because stackql didn't supply a value for the optional limit parameter. Cloudflare returned Syntax Error GraphQL (1:237) Unexpected character "<". The transform then crashed walking data.viewer (nil because data: null). The user saw graphql response transform failed: ... error calling index: index of untyped nil - which points at the transform, when the real cause was the malformed wire query. We only found this by inserting a debug transform that wrapped the entire response as a raw string column. With the request body in the log, this would have been a 5-second diagnosis.
  • Permission bug: the same shape but Cloudflare returned zone '...' does not have access to the field 'crossZoneSubrequests' from the path. Same nil template error surfaced. Same debugging dance. Same fix surfaces it instantly.

Where the gap is

pkg/graphql/graphql.go - StandardGQLReader.Read() around lines 113-185.

The reader:

  1. Calls renderQuery() to build the request body. Not logged.
  2. Calls gq.anySdkClient.Do(...) and gets httpResponse.
  3. Decodes httpResponse.Body into target. Not logged at this point.
  4. (If transform is configured) reshapes target via the transform template.
  5. Runs jsonpath.Get(responseJsonPath, target) and emits the resulting []map[string]interface{} to ss.stream.Write(response). This is what the existing log line surfaces via handlerCtx.LogHTTPResponseMap in graphql_single_select_acquire.go ~line 172.

So the existing log captures step 5's output and nothing else.

Proposed change

Add two log emissions in Read(), gated by the same flag the REST acquire consults:

  1. Between steps 1 and 2: log the rendered request body (the full GraphQL query string sent to the endpoint, with the wire URL).
  2. Between steps 2 and 3: log the raw response body bytes (pre-transform, pre-jsonpath).

Output format should match the existing REST log lines so users can pipe them through the same tooling.

Implementation sketch

In pkg/graphql/graphql.go, inside Read(), after req.Body = rb (around line 124) and after the response is received (around line 135):

// NEW: emit the rendered request if http logging is enabled. The exact
// gating flag depends on how the runtime context is plumbed - this needs
// the same hook the REST acquire calls.
if logger := getHTTPLoggerFromContext(req.Context()); logger != nil {
    bodyBytes, _ := io.ReadAll(req.Body)
    req.Body = io.NopCloser(bytes.NewReader(bodyBytes))  // restore for actual send
    logger.LogRequest("GraphQL POST", req.URL.String(), bodyBytes)
}

r, err := gq.anySdkClient.Do(...)
// ...
httpResponse, _ := r.GetHttpResponse()

// NEW: emit the raw response BEFORE the transform / jsonpath run.
// This is the most important addition - it lets users see what the
// server actually returned, separately from whatever the transform
// produced (which the existing `processed http response body object`
// log line already covers).
if logger := getHTTPLoggerFromContext(req.Context()); logger != nil {
    respBytes, _ := io.ReadAll(httpResponse.Body)
    httpResponse.Body = io.NopCloser(bytes.NewReader(respBytes))  // restore for downstream decode
    logger.LogResponse("GraphQL response (pre-transform)", httpResponse.StatusCode, respBytes)
}

err = json.NewDecoder(httpResponse.Body).Decode(&target)

The exact getHTTPLoggerFromContext interface depends on how the REST acquire plumbs its logger. Reuse, don't reinvent.

Expected output with the fix

stackql >> SELECT datetime, requests, bytes
        >> FROM cloudflare.zones.zone_http_requests_adaptive_groups
        >> WHERE zone_tag = '...' AND since = '...' AND until = '...';

http request: POST https://api.cloudflare.com/client/v4/graphql
content-type: application/json
authorization: Bearer ********

{"query": "query { viewer { zones(filter: {zoneTag: \"...\"}, limit: 1) { httpRequestsAdaptiveGroups(filter: {datetime_geq: \"...\", datetime_lt: \"...\"}, limit: 5, orderBy: [datetimeMinute_ASC]) { dimensions { datetime: datetimeMinute clientCountryName ... } sum { ... } count } } } }"}

http response (pre-transform): 200 OK

{"data":{"viewer":{"zones":[{"httpRequestsAdaptiveGroups":[{"count":30,"dimensions":{"clientCountryName":"AU","datetime":"2026-05-28T19:45:00Z",...},"sum":{"edgeResponseBytes":14396,"visits":0}},...]}]}}}

processed http response body object: [
  { "datetime": "2026-05-28T19:45:00Z", "client_country_name": "AU", "requests": 30, "bytes": 14396, ... },
  ...
]

Three distinct log entries now: request, raw response, post-transform projection. Operators can diff each layer independently.

Test cases

Add to pkg/graphql/graphql_test.go:

func TestRead_EmitsRequestBodyToHTTPLogWhenEnabled(t *testing.T) {
    var logged []string
    ctx := contextWithHTTPLogger(context.Background(), func(line string) {
        logged = append(logged, line)
    })

    reader := newReaderWithStubbedHTTPResponseAndContext(
        t, ctx, `{"data": {"rows": [{"id": 1}]}}`,
        "$.data.rows[*]", "$.data.__no_cursor[*]")
    reader.query = "query { rows { id } }"

    _, err := reader.Read()
    if err != nil && err != io.EOF {
        t.Fatalf("unexpected error: %v", err)
    }

    joined := strings.Join(logged, "\n")
    if !strings.Contains(joined, "query { rows { id } }") {
        t.Errorf("expected rendered request body in log, got:\n%s", joined)
    }
}

func TestRead_EmitsRawResponseToHTTPLogWhenEnabled(t *testing.T) {
    var logged []string
    ctx := contextWithHTTPLogger(context.Background(), func(line string) {
        logged = append(logged, line)
    })

    reader := newReaderWithStubbedHTTPResponseAndContext(
        t, ctx, `{"data": {"rows": [{"id": 1}]}}`,
        "$.data.rows[*]", "$.data.__no_cursor[*]")

    _, err := reader.Read()
    if err != nil && err != io.EOF {
        t.Fatalf("unexpected error: %v", err)
    }

    joined := strings.Join(logged, "\n")
    if !strings.Contains(joined, `"id": 1`) {
        t.Errorf("expected raw response body in log, got:\n%s", joined)
    }
}

func TestRead_DoesNotLogWhenHTTPLogDisabled(t *testing.T) {
    var logged []string
    ctx := context.Background()   // no logger attached

    reader := newReaderWithStubbedHTTPResponseAndContext(
        t, ctx, `{"data": {"rows": []}}`,
        "$.data.rows[*]", "$.data.__no_cursor[*]")

    _, err := reader.Read()
    if err != nil && err != io.EOF {
        t.Fatalf("unexpected error: %v", err)
    }
    if len(logged) != 0 {
        t.Errorf("expected no log output, got %d lines", len(logged))
    }
}

Out of scope

  • New logging configuration / formats. Reuse whatever the REST acquire emits today.
  • Header redaction policy (e.g. masking Authorization). Inherit whatever REST does.
  • A structured-log mode. Plain text aligned with the existing REST output is sufficient for v1.

File references

  • pkg/graphql/graphql.go - StandardGQLReader.Read() around lines 113-185 (request log insertion ~line 124, response log insertion ~line 135).
  • stackql/internal/stackql/primitivebuilder/graphql_single_select_acquire.go around lines 158-194 - dispatch site; may need to thread the runtime context through to the reader if not already.
  • REST-side reference: wherever stackql's REST acquire calls handlerCtx.LogHTTPRequest / LogHTTPResponse - mirror those call sites.

Issue 2 - typos in error messages

The error string in pkg/graphql/graphql.go (the switch pr := processedResponse.(type) block, default branch ~line 183) currently reads:

return nil, fmt.Errorf("cannot accomodate GraphQL pocessed response of type = '%T'", pr)

Two typos: accomodate -> accommodate, pocessed -> processed. The same message appears for the cursor-decode branch in the same function (cannot accomodate GraphQL pocessed response item of type = '%T') - fix both.

Suggested sweep

While the file is open, grep the same package for sibling typos to fix in one pass:

grep -n -iE 'accomodate|pocessed|recieve|seperate|occured|definately|occurence' pkg/graphql/

Common ones to watch for in error / log strings:

  • accomodate -> accommodate
  • pocessed -> processed
  • recieve -> receive
  • seperate -> separate
  • occured -> occurred
  • definately -> definitely
  • occurence -> occurrence
  • unmarshall (two l's) -> unmarshal (Go convention is one l, matching encoding/json.Unmarshal)
  • marshall -> marshal (same reason)

Only flagging - exact fixes depend on what the grep turns up. Don't widen the PR unnecessarily.

File references

  • pkg/graphql/graphql.go - both cannot accomodate error strings in Read().
  • Plus whatever the suggested-sweep grep surfaces.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions