Skip to content

feat(cloud-query): integrate Splunk connection support to cloud query#3350

Merged
michaeljguarino merged 10 commits intomasterfrom
sebastian/prod-4508-workbench-splunk-logs-support
Mar 30, 2026
Merged

feat(cloud-query): integrate Splunk connection support to cloud query#3350
michaeljguarino merged 10 commits intomasterfrom
sebastian/prod-4508-workbench-splunk-logs-support

Conversation

@floreks
Copy link
Copy Markdown
Member

@floreks floreks commented Mar 20, 2026

  • Add SplunkConnection module and corresponding fields in toolquery Protobuf definitions
  • Update tool enums and schema definitions to include Splunk
  • Implement Splunk connection configuration changesets
  • Extend GraphQL types to support Splunk connection attributes

Test Plan

Locally w/ plrl-dev-aws

Checklist

  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have added a meaningful title and summary to convey the impact of this PR to a user.

Plural Flow: console

- Add SplunkConnection module and corresponding fields in toolquery Protobuf definitions
- Update tool enums and schema definitions to include Splunk
- Implement Splunk connection configuration changesets
- Extend GraphQL types to support Splunk connection attributes
@floreks floreks self-assigned this Mar 20, 2026
@floreks floreks added the enhancement New feature or request label Mar 20, 2026
@linear
Copy link
Copy Markdown

linear bot commented Mar 20, 2026

@floreks floreks marked this pull request as draft March 20, 2026 14:39
@floreks floreks changed the title feat: integrate Splunk connection support across the platform feat(cloud-query): integrate Splunk connection support to cloud query Mar 20, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 20, 2026

Greptile Summary

This PR integrates Splunk as a new log-query data source across the full stack: a new SplunkConnection proto message and generated Go/Elixir bindings, a SplunkClient HTTP client, a SplunkProvider logs provider, Ecto schema with encrypted credentials, GraphQL types, and enum registration. The Splunk integration follows the exact same structural pattern as the existing Loki, Elastic, and Tempo providers.

Key changes:

  • clients/splunk.go — new resty-based HTTP client; correctly uses the Splunk <token> auth scheme, with opt-in InsecureSkipVerify via a URL query parameter that is stripped before use
  • provider_splunk.go — log provider with JSON-line scanning (10 MB buffer), preview-result filtering, zero-value guard for empty result lines, multi-format parseTime, and label extraction
  • lib/console/schema/workbench_tool.ex — Splunk embedded schema with EncryptedString fields for token and password; changeset validates that at least one auth method (token or username+password) is provided
  • lib/console/ai/workbench/conversion.ex — Elixir → proto conversion consistent with other providers
  • Proto and GraphQL definitions updated throughout

Issues found:

  • searchString returns \"search \" (no predicate) when the incoming query is empty, and without a limit there is no | head clause — this can trigger a full-index scan against Splunk
  • Provider-level unit tests for parseTime, toLogsQueryOutput, and searchString are absent; only URL-normalisation is tested

Confidence Score: 4/5

Safe to merge after addressing the unbounded empty-query scan risk; all prior P0/P1 concerns from previous review rounds have been resolved.

All previously flagged critical issues (Bearer vs Splunk auth, scanner buffer too small, preview-result duplication, ghost entries from non-result lines, non-deterministic timestamps) are now correctly handled. One new P1 remains: an empty input.Query bypasses the SPL predicate entirely and, without a limit, issues an unbounded full-index search against Splunk. Missing provider-level unit tests are P2.

go/cloud-query/internal/tools/provider_splunk.go — empty-query guard in searchString/Logs

Important Files Changed

Filename Overview
go/cloud-query/internal/tools/provider_splunk.go Core Splunk log provider; previous P1 issues (preview filtering, 10 MB scanner buffer, ghost entries, non-deterministic timestamps) are all addressed; unbounded search on empty query is a remaining concern.
go/cloud-query/internal/tools/clients/splunk.go New Splunk HTTP client; correctly uses Splunk auth scheme (not Bearer), opt-in InsecureSkipVerify via URL query param, and normalises the base URL before use.
go/cloud-query/internal/tools/clients/splunk_test.go Tests normalizeSplunkURL only; no coverage for provider_splunk.go parsing/filtering logic.
go/cloud-query/internal/tools/provider.go Splunk wired into newLogsProvider switch — clean and consistent with existing provider pattern.
lib/console/schema/workbench_tool.ex Splunk schema added with encrypted credentials, auth validation (token or username/password required), and correct category inference (:logs).
lib/console/ai/workbench/conversion.ex Splunk proto conversion added consistently with other providers, mapping all four connection fields.
go/cloud-query/api/proto/toolquery.proto SplunkConnection message and splunk oneof case (field 6) added correctly to proto definition.

Reviews (6): Last reviewed commit: "refactor(splunk): remove redundant token..." | Re-trigger Greptile

floreks added 3 commits March 20, 2026 16:00
- Introduce klog for informative logging when Splunk log timestamp is missing
- Default to zero time to avoid errors in parsing routines when timestamp is empty
- Introduce Splunk connection types and attributes in GraphQL schema
- Update Go client models to reflect Splunk integration
- Expand tool enums to include Splunk
@floreks
Copy link
Copy Markdown
Member Author

floreks commented Mar 20, 2026

@greptileai review again

- Add `Preview` field to `SplunkSearchResponse` structure
- Skip processing of preview responses or those lacking essential fields (Message, Timestamp)
@floreks
Copy link
Copy Markdown
Member Author

floreks commented Mar 20, 2026

@greptileai all fixed, check again

…nd increase scanner buffer size

- Set both 'Authorization' headers for compatibility with older versions.
- Increase scanner buffer size to 10MB in `toLogsQueryOutput` for handling larger responses.
@floreks
Copy link
Copy Markdown
Member Author

floreks commented Mar 30, 2026

@greptileai review again

floreks added 3 commits March 30, 2026 12:11
- Use klog to log errors instead of returning them.
- Continue processing other items after logging an error.
- Remove redundant unit tests for `SplunkProvider` that are no longer needed.
@floreks floreks marked this pull request as ready for review March 30, 2026 10:12
…ance label processing

- Eliminate unnecessary token auth code for backwards compatibility.
- Add logic to remove empty labels in `provider_splunk.go`.
- Simplify `parseTime` function by changing input type to string.
@floreks
Copy link
Copy Markdown
Member Author

floreks commented Mar 30, 2026

@greptileai review

@michaeljguarino michaeljguarino merged commit 4752767 into master Mar 30, 2026
54 of 55 checks passed
@michaeljguarino michaeljguarino deleted the sebastian/prod-4508-workbench-splunk-logs-support branch March 30, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants