Skip to content

Pr/har logging#1946

Merged
moshloop merged 5 commits into
mainfrom
pr/har-logging
May 7, 2026
Merged

Pr/har logging#1946
moshloop merged 5 commits into
mainfrom
pr/har-logging

Conversation

@moshloop
Copy link
Copy Markdown
Member

@moshloop moshloop commented May 1, 2026

Summary by CodeRabbit

  • New Features

    • Runtime properties schema and rich properties system added; observability hooks and per-feature HAR/HTTP logging exposed; health badges and health/status in search results; external group summary view (member/permission counts); expanded relationship and access models to include scraper context.
  • Bug Fixes

    • Ignore soft-deleted records more consistently; consolidate HTTP observability wiring.
  • Documentation

    • Added comprehensive PROPERTIES.md.
  • Tests

    • New observability, git-logging, startup, and DB-related tests added.
  • Chores

    • Build/task targets and ignore rules updated.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 1, 2026

Walkthrough

Adds HTTP observability/HAR middleware and context APIs; provides properties documentation and JSON schema; expands scraper-scoped DB/model keys and view logic; adds MergeConfigTrees and related query changes; updates transport wiring (HTTP/Git/K8s); adds tests and tooling/docs.

Changes

HTTP Observability & HAR Collection Middleware

Layer / File(s) Summary
Core Observability
connection/common.go
Introduces observabilityContext, effectiveHARCollector, applyHTTPObservability, httpObservabilityMiddleware, HAR logging and metadata helpers.
HTTP Client API
connection/http.go, types/client_options.go
Adds Transport/TransportWithContext, httpConnectionRoundTripper composition, OAuth TokenTransport chaining, AWS SigV4 wrapping, and ClientOptions.Feature/WithFeature.
Connection Integrations
connection/aws.go, connection/gcs.go, connection/eks.go, connection/cnrm.go, connection/gke.go, connection/opensearch.go, connection/prometheus.go, connection/kubernetes.go
Each connection now uses centralized applyHTTPObservability/httpObservabilityMiddleware or TransportWithContext to enable HAR/logging instrumentation and unify TLS/insecure branches.
Git Instrumentation
connection/git.go
Per-URL HTTP transport configuration, configureGitHTTPTransport with cleanup, RedactedURL helper, and enhanced clone/fetch logging and auth alignment.
Kubernetes Client
kubernetes/k8s.go
New NewClient* constructors with middleware support, appendWrapTransport helper, and kubeconfig path-or-config helpers.

Scraper Context & DB/View Changes

Layer / File(s) Summary
Models
models/config.go, models/config_access.go, models/common.go
Adds ScraperID to ConfigRelationship and ExternalUserGroup PKs, adds RoleExternalIDs, and adds Health.Badge method.
Schema
schema/config.hcl, schema/config_access.hcl
Adds non-null scraper_id columns with defaults, updates composite indices and primary keys.
Views & SQL
views/006_config_views.sql, views/038_config_access.sql, views/045_merge_external_entities.sql, views/9998_rls_enable.sql
Hardens lookups to ignore soft-deleted rows, adds config_path(), creates external_group_summary, updates merge upserts to include scraper_id, and sets security_invoker on new view.

Config Tree & Query Enhancements

Layer / File(s) Summary
Config Tree Merging
query/config_tree.go
Adds MergeConfigTrees, cloneConfigTree, mergeConfigTreeInto, and getExistingConfigsByIDs; refines ExpandConfigChildren SQL to exclude deleted items.
Query Enrichment
query/resource_selector.go
SelectedResource now includes Health and Status fields; id-based selector detection suppresses automatic agent scoping.

Properties Documentation & Schema

Layer / File(s) Summary
Documentation
PROPERTIES.md
Comprehensive runtime properties documentation covering sources, precedence, annotation resolution, observability flags, and debug endpoints.
Schema
PROPERTIES.schema.json
JSON Schema defining property keys, types, defaults, and pattern-based dynamic properties for validation and authoring.

Tooling, Tests & Startup

Layer / File(s) Summary
Tooling & Ignore
.gitignore, Makefile, Taskfile.yaml
Adds .gavel/ to .gitignore; Makefile adds captain and PROPERTIES.md targets and a commented gavel/test template; Taskfile.yaml includes a commented gavel test task.
PostgREST Startup
start.go, start_test.go, suite_test.go
Extracts configurePostgrest helper to validate/assign PostgREST URL/port/secret and decide local startup; adds tests for localhost/remote/disabled scenarios and test bootstrap.
Tests & Setup
connection/observability_test.go, connection/git_logging_test.go, tests/*
Adds HAR/HTTP logging tests, git logging tests, UpdateIsPushed and external_group_summary tests; reworks tests/setup to use execPostgres and deterministic DB naming.
Context Template Logging
context/template.go
RunTemplate gains per-template logger and level-based logging for changes and environment output.

Possibly related PRs

  • flanksource/duty#1931: Modifies test DB setup and database naming/URL handling, directly related to test setup changes in this PR.
  • flanksource/duty#1918: Touches query/config_tree.go and tree-building logic overlapping with MergeConfigTrees additions.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 26.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Pr/har logging' accurately reflects the main focus of this comprehensive pull request, which adds HTTP Archive (HAR) observability and logging capabilities throughout the codebase.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pr/har-logging
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch pr/har-logging

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Benchstat (RLS)

Base: 0fd35df9913688981e68886871dcaf026734a218
Head: 7ff338fcb73bfd92f7c03c57f71878cf49a6a7da

📊 3 minor regression(s) (all within 5% threshold)

Benchmark Base Head Change p-value
RLS/Sample-15000/catalog_changes/With_RLS-4 139.4m 144.7m +3.84% 0.002
RLS/Sample-15000/configs/With_RLS-4 134.6m 137.0m +1.80% 0.002
RLS/Sample-15000/analyzer_types/Without_RLS-4 3.719m 3.771m +1.39% 0.002
✅ 6 improvement(s)
Benchmark Base Head Change p-value
RLS/Sample-15000/change_types/With_RLS-4 5.437m 5.238m -3.66% 0.002
RLS/Sample-15000/config_detail/Without_RLS-4 4.836m 4.716m -2.48% 0.002
RLS/Sample-15000/config_changes/With_RLS-4 144.6m 141.8m -1.92% 0.002
RLS/Sample-15000/config_detail/With_RLS-4 137.8m 135.8m -1.40% 0.004
RLS/Sample-15000/config_classes/Without_RLS-4 4.120m 4.077m -1.03% 0.002
RLS/Sample-15000/config_names/Without_RLS-4 13.52m 13.44m -0.56% 0.009
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: AMD EPYC 7763 64-Core Processor                
                                               │ bench-base.txt │          bench-head.txt           │
                                               │     sec/op     │   sec/op     vs base              │
RLS/Sample-15000/catalog_changes/Without_RLS-4      5.290m ± 1%   5.271m ± 2%       ~ (p=0.394 n=6)
RLS/Sample-15000/catalog_changes/With_RLS-4         139.4m ± 1%   144.7m ± 1%  +3.84% (p=0.002 n=6)
RLS/Sample-15000/config_changes/Without_RLS-4       5.291m ± 1%   5.280m ± 1%       ~ (p=0.589 n=6)
RLS/Sample-15000/config_changes/With_RLS-4          144.6m ± 1%   141.8m ± 1%  -1.92% (p=0.002 n=6)
RLS/Sample-15000/config_detail/Without_RLS-4        4.836m ± 1%   4.716m ± 1%  -2.48% (p=0.002 n=6)
RLS/Sample-15000/config_detail/With_RLS-4           137.8m ± 1%   135.8m ± 1%  -1.40% (p=0.004 n=6)
RLS/Sample-15000/config_names/Without_RLS-4         13.52m ± 2%   13.44m ± 0%  -0.56% (p=0.009 n=6)
RLS/Sample-15000/config_names/With_RLS-4            138.2m ± 1%   137.7m ± 1%       ~ (p=0.180 n=6)
RLS/Sample-15000/config_summary/Without_RLS-4       95.07m ± 1%   94.81m ± 2%       ~ (p=0.485 n=6)
RLS/Sample-15000/config_summary/With_RLS-4          705.0m ± 0%   703.8m ± 1%       ~ (p=0.394 n=6)
RLS/Sample-15000/configs/Without_RLS-4              7.963m ± 3%   7.986m ± 2%       ~ (p=0.132 n=6)
RLS/Sample-15000/configs/With_RLS-4                 134.6m ± 1%   137.0m ± 1%  +1.80% (p=0.002 n=6)
RLS/Sample-15000/analysis_types/Without_RLS-4       3.884m ± 2%   3.898m ± 0%       ~ (p=0.310 n=6)
RLS/Sample-15000/analysis_types/With_RLS-4          3.940m ± 1%   3.903m ± 2%       ~ (p=0.065 n=6)
RLS/Sample-15000/analyzer_types/Without_RLS-4       3.719m ± 1%   3.771m ± 0%  +1.39% (p=0.002 n=6)
RLS/Sample-15000/analyzer_types/With_RLS-4          3.755m ± 1%   3.758m ± 3%       ~ (p=0.699 n=6)
RLS/Sample-15000/change_types/Without_RLS-4         5.280m ± 1%   5.231m ± 1%       ~ (p=0.240 n=6)
RLS/Sample-15000/change_types/With_RLS-4            5.437m ± 2%   5.238m ± 2%  -3.66% (p=0.002 n=6)
RLS/Sample-15000/config_classes/Without_RLS-4       4.120m ± 6%   4.077m ± 1%  -1.03% (p=0.002 n=6)
RLS/Sample-15000/config_classes/With_RLS-4          137.3m ± 1%   136.8m ± 2%       ~ (p=0.065 n=6)
RLS/Sample-15000/config_types/Without_RLS-4         4.734m ± 1%   4.704m ± 1%       ~ (p=0.065 n=6)
RLS/Sample-15000/config_types/With_RLS-4            136.3m ± 1%   136.6m ± 0%       ~ (p=0.310 n=6)
geomean                                             20.92m        20.85m       -0.34%

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Benchstat (Other)

Base: 0fd35df9913688981e68886871dcaf026734a218
Head: 7ff338fcb73bfd92f7c03c57f71878cf49a6a7da

📊 2 minor regression(s) (all within 5% threshold)

Benchmark Base Head Change p-value
ResourceSelectorQueryBuild/name-4 40.33µ 41.19µ +2.12% 0.002
ResourceSelectorQueryBuild/name_and_type-4 59.52µ 60.32µ +1.34% 0.002
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
                                                       │ bench-base.txt │           bench-head.txt            │
                                                       │     sec/op     │    sec/op      vs base              │
InsertionForRowsWithAliases/external_users.aliases-4       402.5µ ±  7%   402.5µ ±   1%       ~ (p=1.000 n=6)
InsertionForRowsWithAliases/config_items.external_id-4     927.0µ ± 16%   924.2µ ±  15%       ~ (p=0.818 n=6)
InsertionOfConfigsWithProperties-4                         2.805m ±  1%   2.822m ±   1%       ~ (p=0.093 n=6)
UpdateOfConfigsWithProperties-4                            5.700m ±  1%   5.695m ±   1%       ~ (p=0.937 n=6)
ResourceSelectorConfigs/name-4                             160.6µ ±  6%   164.3µ ±   3%       ~ (p=0.180 n=6)
ResourceSelectorConfigs/name_and_type-4                    180.4µ ±  4%   179.9µ ±   4%       ~ (p=0.937 n=6)
ResourceSelectorConfigs/tags-4                             53.50m ± 48%   32.00m ± 150%       ~ (p=0.699 n=6)
ResourceSelectorQueryBuild/name-4                          40.33µ ±  0%   41.19µ ±   1%  +2.12% (p=0.002 n=6)
ResourceSelectorQueryBuild/name_and_type-4                 59.52µ ±  0%   60.32µ ±   7%  +1.34% (p=0.002 n=6)
ResourceSelectorQueryBuild/tags-4                          15.52µ ± 11%   15.62µ ±   1%       ~ (p=0.394 n=6)
geomean                                                    450.5µ         430.7µ         -4.40%

@moshloop moshloop requested a review from yashmehrotra May 1, 2026 11:53
moshloop added 3 commits May 7, 2026 00:48
…r groups

Introduce scraper_id as part of composite primary keys for ConfigRelationship and ExternalUserGroup to support multi-scraper ownership. This allows multiple scrapers to independently maintain relationship tuples without colliding, while uuid.Nil represents legacy scraper-agnostic relationships.

Add Health.Badge() method to render health status as colored pills for dense table cells. Implement MergeConfigTrees() to collapse independently-built config trees into a unified forest by merging shared ancestors recursively.

Enhance SearchResources to populate Health and Status fields for configs and components. Add RoleExternalIDs to ConfigAccessSummary. Create external_group_summary view with member and permission counts.

Update SQL views to filter deleted relationships and add GROUP BY clauses for accurate counts. Simplify database setup helpers by removing unnecessary quoting and URL parsing utilities.

BREAKING CHANGE: ConfigRelationship and ExternalUserGroup primary keys now include scraper_id; existing code must populate this field from scraper context.
Replace ginkgo-based test targets with gavel, a unified test runner that provides better output formatting and cross-platform compatibility. Update test and test-concurrent targets to use gavel with consistent timeout and ignore patterns. Add gavel installation target and captain target for properties documentation generation. Add PROPERTIES.md and PROPERTIES.schema.json documenting all runtime properties across duty and related services. Update Taskfile.yaml test task to use gavel with passthrough argument support.
…ty middleware into reusable functions

Extract HTTP observability logic (HAR collection, HTTP logging) from individual connection types into shared middleware functions in connection/common.go. This eliminates code duplication across AWS, GCS, GKE, OpenSearch, and other connection implementations.

New observability functions:
- applyHTTPObservability: applies HAR and HTTP logging to a RoundTripper
- httpObservabilityMiddleware: returns a middleware for WrapTransport
- applyHTTPClientObservability: applies observability to commons HTTP clients
- harCollectorMiddleware: HAR-specific middleware with level-aware behavior
- httpLoggerWithContent: HTTP request/response logging with configurable detail
- metadataHARMiddleware: HAR metadata-only capture for debug-level logging

Add context methods for observability configuration:
- EffectiveHARLevel, EffectiveLogLevel: resolve per-feature log levels
- EffectiveHARCollector: select HAR collector with feature-aware config
- HTTPLoggingContent: determine header/body logging for a feature
- HARConfig: build feature-specific HAR configuration

Update connection implementations to use shared functions, reducing per-connection boilerplate. Add git HTTP transport observability support via configureGitHTTPTransport.

Add comprehensive tests for HAR and HTTP logging coexistence, ensuring body restoration across middleware layers.

Refs #1234
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
query/resource_selector.go (1)

293-322: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

agentID is shadowed; the outer value passed to getScopeID is always nil.

At line 295, agentID, err := getAgentID(...) re-declares agentID in the inner if block, so the outer agentID declared at line 293 stays nil. Consequently, getScopeID(ctx, ..., agentID) on line 306 never receives the resolved agent and the scope lookup is unscoped by agent. With the new searchSetID branch this is more visible, since callers may now reach the scope path with different expectations about agent scoping.

If this is intentional, a brief comment would help; otherwise the inner block should assign to the outer variable (agentID, err = getAgentID(...)).

🛠 Proposed fix
-	var agentID *uuid.UUID
-	if !searchSetAgent && !searchSetID && qm.HasAgents {
-		agentID, err := getAgentID(ctx, resourceSelector.Agent)
-		if err != nil {
-			return nil, err
-		}
-
-		if agentID != nil {
-			query = query.Where("agent_id = ?", *agentID)
-		}
-	}
+	var agentID *uuid.UUID
+	if !searchSetAgent && !searchSetID && qm.HasAgents {
+		agentID, err = getAgentID(ctx, resourceSelector.Agent)
+		if err != nil {
+			return nil, err
+		}
+
+		if agentID != nil {
+			query = query.Where("agent_id = ?", *agentID)
+		}
+	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@query/resource_selector.go` around lines 293 - 322, The agentID declared at
the top of the block is being shadowed by the inner short-declaration (agentID,
err := getAgentID(...)) inside the qm.HasAgents branch, so the outer agentID
remains nil and the subsequent getScopeID(ctx, resourceSelector.Scope, table,
agentID) call is unscoped; change the inner declaration to assign to the outer
variable (use agentID, err = getAgentID(...)) so the resolved agent is passed
into getScopeID, and ensure error handling remains the same; check the
qm.HasAgents, searchSetAgent, and searchSetID logic around
resourceSelector.Scope and table
(checks/config_items/components/config_changes/catalog_changes) to confirm
scoping behavior is preserved.
tests/setup/common.go (1)

278-299: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Quote or validate the derived database name before interpolating it into SQL.

newName includes the caller-supplied name, so these statements now break on valid-but-unquoted identifiers and reintroduce SQL-injection risk that the old helper avoided.

🧱 Suggested fix
+func quoteIdentifier(name string) string {
+	return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
+}
+
-	if err := ctx.DB().Exec(fmt.Sprintf("CREATE DATABASE %s", newName)).Error; err != nil {
+	if err := ctx.DB().Exec("CREATE DATABASE " + quoteIdentifier(newName)).Error; err != nil {
 		return nil, nil, err
 	}
@@
-		if err := ctx.DB().Exec(fmt.Sprintf("DROP DATABASE  %s (FORCE)", newName)).Error; err != nil {
+		if err := ctx.DB().Exec("DROP DATABASE " + quoteIdentifier(newName) + " (FORCE)").Error; err != nil {
 			logger.Errorf("error cleaning up db: %v", err)
 		}
 	}, nil
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/setup/common.go` around lines 278 - 299, The DB creation/drop SQL
interpolates caller-supplied newName directly (seen in the ctx.DB().Exec("CREATE
DATABASE %s", newName) and DROP DATABASE ... (FORCE) calls), reintroducing
SQL-injection risk; instead either validate newName against a strict identifier
regex (e.g. ^[A-Za-z_][A-Za-z0-9_]*$) before use or quote it safely with a
proper identifier-quoting helper (e.g. pq.QuoteIdentifier or your project's
equivalent) and use that quoted value in the Exec calls; update the code paths
around newName, the CREATE DATABASE and DROP DATABASE Exec invocations, to use
the validated or quoted identifier.
connection/http.go (1)

351-410: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Transport/TransportWithContext does not honor HTTPConnection.TLS settings.

The Transport path constructs the base from &netHTTP.Transport{} (line 357) without applying the connection's TLS settings (CA, Cert, Key, InsecureSkipVerify, HandshakeTimeout). The assignment at lines 397–399 is dead code — conn is a value copy of rt.HTTPConnection, so rt.TLS = conn.TLS assigns the embedded field to itself without configuring the actual underlying transport. Contrast with CreateHTTPClient (lines 445–456), which correctly applies TLS via client.TLSConfig(...).

A caller using conn.TransportWithContext(ctx) with TLS settings — such as InsecureSkipVerify: true for a self-signed certificate — will silently use Go's default TLS behavior and fail. The PrometheusConnection client at line 41 is directly affected if its underlying HTTPConnection has TLS settings.

Apply TLS configuration to the base netHTTP.Transport before wrapping with observability, and remove the no-op assignment from RoundTrip:

Suggested fix
 func (h HTTPConnection) TransportWithContext(ctx any, opts ...types.ClientOption) netHTTP.RoundTripper {
 	o := types.NewClientOptions(opts...)
+	tr := &netHTTP.Transport{}
+	if !h.TLS.IsEmpty() {
+		tr.TLSClientConfig = &tls.Config{
+			InsecureSkipVerify: h.TLS.InsecureSkipVerify,
+			// ... populate from h.TLS.CA, h.TLS.Cert, h.TLS.Key
+		}
+		if h.TLS.HandshakeTimeout != 0 {
+			tr.TLSHandshakeTimeout = h.TLS.HandshakeTimeout
+		}
+	}
-	base := applyHTTPObservability(ctx, "http", &netHTTP.Transport{}, o.HARCollector)
+	base := applyHTTPObservability(ctx, "http", tr, o.HARCollector)
 	rt := &httpConnectionRoundTripper{
 		HTTPConnection: h,
 		Base:           base,
 		TokenTransport: harTokenTransport(ctx, "http", o.HARCollector),
 	}
 	return rt
 }

And remove lines 397–399.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@connection/http.go` around lines 351 - 410, TransportWithContext currently
constructs base as &netHTTP.Transport{} via applyHTTPObservability without
applying HTTPConnection.TLS, so TLS settings are ignored and the later rt.TLS =
conn.TLS in httpConnectionRoundTripper.RoundTrip is a no-op; update
TransportWithContext (the Transport and TransportWithContext functions) to take
conn := h (or access h.TLS) and apply conn.TLS into the underlying
netHTTP.Transport (CA, Cert/Key, InsecureSkipVerify, HandshakeTimeout etc.)
before calling applyHTTPObservability/creating base, mirroring how
CreateHTTPClient/client.TLSConfig applies TLS, and remove the dead assignment in
httpConnectionRoundTripper.RoundTrip that sets rt.TLS = conn.TLS.
🧹 Nitpick comments (13)
models/common.go (1)

91-95: 💤 Low value

Redundant HealthUnknown case duplicates the default branch.

The HealthUnknown case returns the exact same badge as the default branch. Either drop the explicit case, or differentiate HealthUnknown (e.g. with distinct styling, as Pretty() does) so the case earns its keep.

♻️ Proposed simplification
 	case HealthWarning:
 		return api.Badge(label, "bg-yellow-100", "text-yellow-800", "capitalize")
-	case HealthUnknown:
-		return api.Badge(label, "bg-gray-100", "text-gray-600", "capitalize")
 	default:
 		return api.Badge(label, "bg-gray-100", "text-gray-600", "capitalize")
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@models/common.go` around lines 91 - 95, The HealthUnknown case in the badge
switch duplicates the default branch in the function that returns api.Badge
(located in models/common.go), so remove the explicit "case HealthUnknown:"
branch to rely on the default branch, or if you want a distinct presentation
keep the case but change its styling to differ from default (e.g., use the same
styling as Pretty() or a unique bg/text class) and update the api.Badge call in
that branch accordingly.
query/config_tree.go (3)

35-98: ⚡ Quick win

Document that the merged forest is a DAG (shared node pointers across roots).

cloneConfigTree returns the existing pointer when an ID is re-encountered, so the same *ConfigTreeNode may appear under multiple roots — this is exercised by TestMergeConfigTreesSharedInternalNode (BeIdenticalTo). Callers that mutate, JSON-serialize, or walk the result without a visited set may be surprised: mutations propagate across roots, and serializers will emit the shared subtree once per parent. The current docstring talks about "trees" and "forest" but doesn't call this out.

A short note in the MergeConfigTrees docstring would prevent misuse.

📝 Suggested doc tweak
 // EdgeType="target" is preserved when the same node appears as a target in one
 // input and as a non-target (parent/child/related) in another.
+//
+// The returned forest is a DAG: a node that appears under multiple roots is
+// aliased (same *ConfigTreeNode pointer), not deep-copied. Callers that mutate
+// or walk the result must be aware that subtrees can be shared across roots.
 func MergeConfigTrees(trees []*ConfigTreeNode) []*ConfigTreeNode {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@query/config_tree.go` around lines 35 - 98, Update the MergeConfigTrees
docstring to explicitly state that the returned "forest" can be a DAG with
shared *ConfigTreeNode pointers across roots (because cloneConfigTree returns
existing nodes from byID), so callers should not assume disjoint trees; note
that mutations to a shared node will affect all roots that reference it, JSON
serializers may duplicate the shared subtree under each parent, and walkers
should use a visited set to avoid repeated processing (refer to
MergeConfigTrees, cloneConfigTree, and the behavior exercised by
TestMergeConfigTreesSharedInternalNode).

81-98: 💤 Low value

Relation from src is silently dropped when dst.Relation differs.

mergeConfigTreeInto only promotes EdgeType to "target"; it never touches Relation. If the same node is reached via different relations in different inputs (e.g. dst.Relation="" from a parent edge, src.Relation="trades-with" from a related edge), the more informative relation is lost — and it's order-dependent on which tree was processed first.

In practice the only path where the same node legitimately gets a non-empty Relation is the related edge, and EdgeType="target" already dominates over "related" here, so the impact is narrow. Still, worth confirming this is intentional or deciding on an explicit precedence (e.g. prefer non-empty src.Relation when dst.Relation == "").

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@query/config_tree.go` around lines 81 - 98, mergeConfigTreeInto currently
only updates EdgeType and can silently drop a non-empty Relation from src when
dst.Relation is empty; change the merge logic in mergeConfigTreeInto so that
when visiting the same node (function mergeConfigTreeInto) you also merge
Relation with explicit precedence: if dst.Relation == "" and src.Relation != ""
then set dst.Relation = src.Relation (but do not overwrite a non-empty
dst.Relation), and ensure this applies both when promoting EdgeType to "target"
and when appending cloned children (cloneConfigTree) so the preserved relation
survives merges and is not lost due to processing order.

225-238: 💤 Low value

Behavior change worth flagging in commit/PR notes: silent skip of missing IDs.

getExistingConfigsByIDs swallows gorm.ErrRecordNotFound, whereas the previously-used GetConfigsByIDs propagated it (query/config.go:388-400). Call sites at lines 135 and 182 already cope with missing entries (the byID-map filtering in resolveParentsFromPath, and downstream buildConfigTree skipping unknown IDs), so functionally this is a deliberate relaxation. A debug-level log on skip would help diagnose stale-cache or cross-shard issues without spamming production logs, but it's optional for this PR.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@query/config_tree.go` around lines 225 - 238, getExistingConfigsByIDs
silently skips missing IDs (errors.Is(err, gorm.ErrRecordNotFound)) which
changes behavior; keep the same skip but emit a debug-level log when
ConfigItemFromCache returns gorm.ErrRecordNotFound so stale-cache or cross-shard
misses are visible without noise. Modify getExistingConfigsByIDs to, inside the
errors.Is(err, gorm.ErrRecordNotFound) branch, call the project's debug logger
(consistent with existing logging usage in this package) to log the missing id
(id.String()) and brief context, then continue; leave other error handling and
return values unchanged.
connection/http.go (1)

355-364: ⚖️ Poor tradeoff

Consider tightening TransportWithContext's ctx any signature.

ctx any accepts anything, including nil and arbitrary types, and pushes the type-narrowing down into applyHTTPObservability/harTokenTransport. This is fragile — a caller who passes the wrong concrete type silently loses observability with no compile-time signal. If only context.Context and ConnectionContext are valid, two typed overloads (or a single context.Context parameter, with Transport() keeping the no-context shortcut) would be both safer and self-documenting.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@connection/http.go` around lines 355 - 364, Change TransportWithContext's
first parameter from ctx any to a typed context (preferably ctx context.Context)
to avoid accepting arbitrary types; update TransportWithContext(ctx
context.Context, opts ...types.ClientOption) in the HTTPConnection receiver and
adjust callers; then modify applyHTTPObservability and harTokenTransport to
accept context.Context (or provide an adapter that extracts a ConnectionContext
from context.Context) so observability and token transport get a well-typed
context; alternatively, if you want two overloads, add
TransportWithConnectionContext(connCtx ConnectionContext, opts...) in addition
to Transport() so callers must pass either no-context or a strongly-typed
context, and update httpConnectionRoundTripper construction in
TransportWithContext to use the new typed APIs.
Makefile (1)

94-101: 💤 Low value

Verify that claude CLI availability is handled.

Unlike gavel and captain, the claude CLI used at line 96 isn't auto-installed (and can't be via go install). Running make PROPERTIES.md on a fresh checkout where claude is missing will fail with a confusing command not found. Consider gating with a friendly check (e.g., @command -v claude >/dev/null || { echo "claude CLI required: see https://docs.anthropic.com/..." >&2; exit 1; }) or documenting the prerequisite near the target.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Makefile` around lines 94 - 101, The Makefile target PROPERTIES.md invokes
the external claude CLI without checking availability, which causes a confusing
"command not found" for fresh checkouts; update the PROPERTIES.md target (the
rule that pipes the claude command into captain) to first check for the claude
binary (e.g., use a shell check like command -v claude) and print a clear error
and non-zero exit if missing, or gate the target with a phony prerequisite that
validates claude presence and emits a helpful install/documentation message
before attempting to run claude.
connection/gke.go (1)

110-114: 💤 Low value

Functionally safe but inconsistent with the appendWrapTransport pattern.

restConfig is constructed locally at Lines 103-109 with no WrapTransport set, so directly assigning it here is correct. However, this duplicates the wrapping logic that kubernetes.appendWrapTransport already encapsulates (and that handles the compose-with-existing case). If that helper is exported (see comment on kubernetes/k8s.go), this block becomes a one-liner and the two call sites stay in lockstep.

♻️ Use the shared helper (after exporting it)
-	if middleware := httpObservabilityMiddleware(ctx, "kubernetes", o.HARCollector); middleware != nil {
-		restConfig.WrapTransport = func(rt http.RoundTripper) http.RoundTripper {
-			return middleware(rt)
-		}
-	}
+	dutyKubernetes.AppendWrapTransport(restConfig, httpObservabilityMiddleware(ctx, "kubernetes", o.HARCollector))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@connection/gke.go` around lines 110 - 114, The code duplicates
transport-wrapping logic instead of using the shared helper: replace the manual
WrapTransport assignment on restConfig (the block that calls
httpObservabilityMiddleware with o.HARCollector) with a call to the shared
appendWrapTransport helper (e.g., kubernetes.AppendWrapTransport(restConfig,
httpObservabilityMiddleware(ctx, "kubernetes", o.HARCollector))) so it composes
with any existing WrapTransport; ensure the helper is exported
(AppendWrapTransport) if not already and pass the middleware result directly to
it.
kubernetes/k8s.go (2)

107-118: 💤 Low value

Composition is correct; consider exporting for reuse.

appendWrapTransport correctly chains atop any existing WrapTransport (including the one set by trace). connection/gke.go (Lines 110-114) duplicates this pattern by directly assigning restConfig.WrapTransport; exporting this helper (e.g., AppendWrapTransport) and calling it there would centralize the composition logic and avoid the inconsistency where the GKE path overwrites instead of composes.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@kubernetes/k8s.go` around lines 107 - 118, The helper that composes
rest.Config.WrapTransport should be exported and used by the GKE path: rename
and export appendWrapTransport to AppendWrapTransport (preserving signature func
AppendWrapTransport(config *rest.Config, middleware middlewares.Middleware)) and
replace the direct assignment to restConfig.WrapTransport in the GKE connection
code with a call to AppendWrapTransport so the new middleware composes with any
existing WrapTransport (e.g., the one set by trace) instead of overwriting it;
update any call sites to the new exported name.

160-171: 💤 Low value

Subtle: heuristic only triggers on a " prefix; document the assumption.

normalizeKubeconfigOrPath decodes JSON-encoded kubeconfig only when the trimmed input starts with ". That works because YAML kubeconfigs realistically never start with a JSON string-quote, but it's an undocumented assumption. A short comment on the function explaining the discriminator — and noting that on any decode failure the original (untrimmed) value is returned to avoid silently stripping whitespace from a real path — would help future readers.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@kubernetes/k8s.go` around lines 160 - 171, Add a short comment above
normalizeKubeconfigOrPath explaining the heuristic: it treats inputs that, after
TrimSpace, start with a double-quote as JSON-string-encoded kubeconfigs (since
real YAML kubeconfigs won't start with `"`), attempts json.Unmarshal into a
string, and on any decode failure returns the original untrimmed value to avoid
silently stripping whitespace from a real path; reference the function name
normalizeKubeconfigOrPath and the variables trimmed/value in the comment so
readers know why the discriminator and the fallback behavior are chosen.
kubernetes/utils_test.go (1)

48-48: 💤 Low value

Passing nil logger is fragile.

NewClientFromPathOrConfigWithMiddleware happens to never deref the logger on this test's code path (the input is decoded as kubeconfig content, so the path branch in NewClientWithMiddleware that calls log.Infof isn't hit). But any future change that adds a log call before/after normalizeKubeconfigOrPath would crash this test with an NPE. Prefer logger.GetLogger("test") (or the package's null-logger equivalent) to keep the test resilient.

♻️ Use a real (silent) logger
-	client, restConfig, err := NewClientFromPathOrConfigWithMiddleware(nil, string(encoded), nil)
+	client, restConfig, err := NewClientFromPathOrConfigWithMiddleware(logger.GetLogger("test"), string(encoded), nil)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@kubernetes/utils_test.go` at line 48, The test passes a nil logger into
NewClientFromPathOrConfigWithMiddleware which is fragile; change the test to
pass a real (silent) logger instead of nil (e.g., use logger.GetLogger("test")
or the package's null-logger) when calling
NewClientFromPathOrConfigWithMiddleware so future additions that call logging
(e.g., in NewClientWithMiddleware or normalizeKubeconfigOrPath) won't cause a
nil-pointer panic; update the client, restConfig, err :=
NewClientFromPathOrConfigWithMiddleware(...) invocation to supply that logger.
views/038_config_access.sql (1)

16-38: 💤 Low value

Active-membership CTE correctly excludes soft-deleted memberships from grant fan-out.

Materialising active_external_user_groups as a DISTINCT non-deleted view and using it in both the inner-join branch (line 36) and the NOT EXISTS guard (line 57-58) ensures soft-deleted memberships no longer produce per-user fan-out rows nor mask group-only rows.

Minor stylistic nit (pre-existing, not introduced here): config_access.deleted_at IS NULL and config_access.external_group_id IS NOT NULL belong in a WHERE rather than the ON clause; their position in the join predicate is functionally fine for an INNER JOIN but reads awkwardly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@views/038_config_access.sql` around lines 16 - 38, The JOIN currently places
non-join predicates (config_access.deleted_at IS NULL and
config_access.external_group_id IS NOT NULL) inside the ON clause of the INNER
JOIN between config_access and the CTE active_external_user_groups; move those
two predicates out of the ON clause and into a WHERE clause (keeping the INNER
JOIN only for matching active_external_user_groups.external_group_id =
config_access.external_group_id) so the intent is clearer and the join predicate
only expresses the join key for config_access and active_external_user_groups.
tests/config_access_test.go (1)

71-132: 💤 Low value

No cleanup of test data inserted into a shared DB.

deletedGroup, deletedUser, and deletedMembership are inserted but never deleted; they will persist for the rest of the suite run. While the random UUIDs avoid collisions with other tests, this leaks rows into external_groups/external_users/external_user_groups that subsequent suite-wide queries (e.g., other counts in the same Ordered block) could be sensitive to. Consider an AfterEach/DeferCleanup to remove them.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/config_access_test.go` around lines 71 - 132, Test inserts
deletedGroup, deletedUser and deletedMembership but never removes them; add
cleanup to delete those rows after the spec runs (use Ginkgo's DeferCleanup or
AfterEach) by invoking DefaultContext.DB().Delete on models.ExternalGroup,
models.ExternalUser and models.ExternalUserGroup using the inserted IDs
(deletedGroup.ID, deletedUser.ID and the composite keys from deletedMembership)
so the temporary rows are removed and do not leak into subsequent tests.
tests/config_relationship_test.go (1)

440-502: 💤 Low value

Consider using an existing scraper fixture to future-proof the test.

The config_relationships.scraper_id column currently has no foreign key constraint, so the test works fine with uuid.New(). However, if a FK is added later, ownedRel insertion could fail. Swapping scraperID to reference an existing fixture (e.g., from the dummy data) would make the test more resilient to future schema changes with minimal effort.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/config_relationship_test.go` around lines 440 - 502, The test creates a
random scraperID (scraperID) and assigns it to ownedRel, which could break if a
foreign key is later added; change the initializer in the BeforeAll so scraperID
(and ownedRel.ScraperID) uses an existing dummy fixture scraper UUID (e.g.,
dummy.Scraper.ID or the appropriate existing dummy scraper field) instead of
uuid.New(), keeping the same scraperID variable and ownedRel construction so the
rest of the test and UpdateIsPushed call remain unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@connection/git_logging_test.go`:
- Around line 98-138: The helper makeBareRepo invokes the git executable without
checking availability, causing test failures on systems without Git; modify
makeBareRepo to call exec.LookPath("git") at the start and call t.Skipf(...) if
git is not found so the test is skipped gracefully, and apply the same
availability check/skip to any other test helpers in this file that shell out to
git (the other helper used around the later block that also runs
exec.Command("git", ...)) so tests do not fail on minimal CI images.

In `@connection/git.go`:
- Line 29: The global mutex gitHTTPTransportMu and the configureGitHTTPTransport
function currently install wrapped http/https protocols via
gitClient.InstallProtocol during each GitClient.Clone and hold the lock for the
entire clone, which serializes all clones process-wide; change this by (1)
adding an explanatory comment near gitHTTPTransportMu and
configureGitHTTPTransport/GitClient.Clone describing this serialization and
visibility to all go-git users, and (2) refactor to install the wrapped protocol
clients once at startup (when HTTP observability/logging is enabled) instead of
per-clone — or alternatively implement and install a single lightweight
RoundTripper that dispatches behavior based on http.Request.Context() so
per-call context is respected without re-registering protocols; ensure
references to gitClient.InstallProtocol, configureGitHTTPTransport,
restoreGitTransport, and GitClient.Clone are updated to reflect the new
startup/one-time registration approach and remove per-clone lock usage.

In `@context/template.go`:
- Around line 18-25: The current logging branches in context/template.go leak
secrets by printing the full env and rendered template (see l.V(3).Infof using
logger.Pretty(env) and the high-verbosity rendered output); replace these prints
with non-sensitive summaries: log only env keys or a count (use lo.Keys(env) or
len(env)) and never log the full rendered template/value. Update the
l.V(3).Infof and the analogous branch at lines 58-65 to call a new helper like
sanitizeEnvKeysOrMask(env) (or simply use lo.Keys(env)/len(env)) and remove any
logger.Pretty(env)/rendered value outputs so secrets are not emitted.

In `@Makefile`:
- Around line 18-35: Add the targets "test" and "test-concurrent" to the
Makefile's .PHONY declarations so Make always treats them as phony targets;
update the existing .PHONY line (or add a new .PHONY line) to include test and
test-concurrent alongside the other phony targets to match the surrounding
targets like test-e2e and test-e2e-blobs.

In `@PROPERTIES.md`:
- Around line 9-10: The README contains an absolute, developer-local link to
PROPERTIES.schema.json; update the link target to a relative path (e.g.,
./PROPERTIES.schema.json or PROPERTIES.schema.json) so the reference in
PROPERTIES.md points to the repository file rather than /Users/moshe/…; edit the
line that currently links to
/Users/moshe/go/src/github.com/flanksource/duty/PROPERTIES.schema.json and
replace it with the relative path to PROPERTIES.schema.json.

In `@query/resource_selector.go`:
- Around line 272-279: searchSetID only matches the literal "id", missing
table-specific ids like "config_id" or "component_id"; update the searchSetID
computation (the slice ContainsFunc over flatFields that uses qm.Aliases) to
consider both canonical alias mapping and table-specific id patterns—e.g., after
resolving alias (field := strings.ToLower(s); if alias, field = alias), treat
the field as matching if field == "id" OR strings.HasSuffix(field, "_id") (or
match a regex like ^[a-z]+_id$), so queries explicitly on config_id/component_id
will set searchSetID=true and avoid the auto-agent filter being applied.

In `@schema/config_access.hcl`:
- Around line 127-131: The schema currently sets a steady-state default
"00000000-0000-0000-0000-000000000000" for column "scraper_id" on table
external_user_groups which can hide missed propagation bugs; remove the default
from the production HCL declaration so the column has no default, and instead
ensure any one-off migration/backfill step sets that nil-UUID only during
backfill (e.g., in the migration script that updates external_user_groups) and
then drops/does not persist the default in the steady-state schema (update the
column "scraper_id" block to remove the default and keep null=false/type=uuid).

In `@tests/config_access_test.go`:
- Around line 117-131: The test silently passes when rows are missing because
byID[...] yields a zero-value externalGroupSummary; ensure presence before
asserting counts by first asserting len(summaries) (or expected number of rows)
and then checking map membership for each expected ID (e.g.
dummy.MissionControlAdminsGroup.ID, dummy.MissionControlReadersGroup.ID,
dummy.MissionControlEmptyGroup.ID, deletedGroup.ID) using a presence check (e.g.
_, ok := byID[id] / Expect(ok).To(BeTrue()) or HaveKey) before asserting
MembersCount and PermissionsCount on the retrieved externalGroupSummary values.

In `@tests/setup/common.go`:
- Around line 186-188: The current code replaces "/postgres" by string
manipulation which can corrupt credentials or query params; instead parse the
DSN with net/url (or the appropriate driver DSN parser), set the path to the new
database name (dbName) and re-encode it before assigning PgUrl; update the logic
around postgresDBUrl, dbName and PgUrl so you construct PgUrl by parsing
postgresDBUrl, replacing only the URL.Path (or the DSN's database field) with
"/"+dbName, and serializing back to a string.

---

Outside diff comments:
In `@connection/http.go`:
- Around line 351-410: TransportWithContext currently constructs base as
&netHTTP.Transport{} via applyHTTPObservability without applying
HTTPConnection.TLS, so TLS settings are ignored and the later rt.TLS = conn.TLS
in httpConnectionRoundTripper.RoundTrip is a no-op; update TransportWithContext
(the Transport and TransportWithContext functions) to take conn := h (or access
h.TLS) and apply conn.TLS into the underlying netHTTP.Transport (CA, Cert/Key,
InsecureSkipVerify, HandshakeTimeout etc.) before calling
applyHTTPObservability/creating base, mirroring how
CreateHTTPClient/client.TLSConfig applies TLS, and remove the dead assignment in
httpConnectionRoundTripper.RoundTrip that sets rt.TLS = conn.TLS.

In `@query/resource_selector.go`:
- Around line 293-322: The agentID declared at the top of the block is being
shadowed by the inner short-declaration (agentID, err := getAgentID(...)) inside
the qm.HasAgents branch, so the outer agentID remains nil and the subsequent
getScopeID(ctx, resourceSelector.Scope, table, agentID) call is unscoped; change
the inner declaration to assign to the outer variable (use agentID, err =
getAgentID(...)) so the resolved agent is passed into getScopeID, and ensure
error handling remains the same; check the qm.HasAgents, searchSetAgent, and
searchSetID logic around resourceSelector.Scope and table
(checks/config_items/components/config_changes/catalog_changes) to confirm
scoping behavior is preserved.

In `@tests/setup/common.go`:
- Around line 278-299: The DB creation/drop SQL interpolates caller-supplied
newName directly (seen in the ctx.DB().Exec("CREATE DATABASE %s", newName) and
DROP DATABASE ... (FORCE) calls), reintroducing SQL-injection risk; instead
either validate newName against a strict identifier regex (e.g.
^[A-Za-z_][A-Za-z0-9_]*$) before use or quote it safely with a proper
identifier-quoting helper (e.g. pq.QuoteIdentifier or your project's equivalent)
and use that quoted value in the Exec calls; update the code paths around
newName, the CREATE DATABASE and DROP DATABASE Exec invocations, to use the
validated or quoted identifier.

---

Nitpick comments:
In `@connection/gke.go`:
- Around line 110-114: The code duplicates transport-wrapping logic instead of
using the shared helper: replace the manual WrapTransport assignment on
restConfig (the block that calls httpObservabilityMiddleware with
o.HARCollector) with a call to the shared appendWrapTransport helper (e.g.,
kubernetes.AppendWrapTransport(restConfig, httpObservabilityMiddleware(ctx,
"kubernetes", o.HARCollector))) so it composes with any existing WrapTransport;
ensure the helper is exported (AppendWrapTransport) if not already and pass the
middleware result directly to it.

In `@connection/http.go`:
- Around line 355-364: Change TransportWithContext's first parameter from ctx
any to a typed context (preferably ctx context.Context) to avoid accepting
arbitrary types; update TransportWithContext(ctx context.Context, opts
...types.ClientOption) in the HTTPConnection receiver and adjust callers; then
modify applyHTTPObservability and harTokenTransport to accept context.Context
(or provide an adapter that extracts a ConnectionContext from context.Context)
so observability and token transport get a well-typed context; alternatively, if
you want two overloads, add TransportWithConnectionContext(connCtx
ConnectionContext, opts...) in addition to Transport() so callers must pass
either no-context or a strongly-typed context, and update
httpConnectionRoundTripper construction in TransportWithContext to use the new
typed APIs.

In `@kubernetes/k8s.go`:
- Around line 107-118: The helper that composes rest.Config.WrapTransport should
be exported and used by the GKE path: rename and export appendWrapTransport to
AppendWrapTransport (preserving signature func AppendWrapTransport(config
*rest.Config, middleware middlewares.Middleware)) and replace the direct
assignment to restConfig.WrapTransport in the GKE connection code with a call to
AppendWrapTransport so the new middleware composes with any existing
WrapTransport (e.g., the one set by trace) instead of overwriting it; update any
call sites to the new exported name.
- Around line 160-171: Add a short comment above normalizeKubeconfigOrPath
explaining the heuristic: it treats inputs that, after TrimSpace, start with a
double-quote as JSON-string-encoded kubeconfigs (since real YAML kubeconfigs
won't start with `"`), attempts json.Unmarshal into a string, and on any decode
failure returns the original untrimmed value to avoid silently stripping
whitespace from a real path; reference the function name
normalizeKubeconfigOrPath and the variables trimmed/value in the comment so
readers know why the discriminator and the fallback behavior are chosen.

In `@kubernetes/utils_test.go`:
- Line 48: The test passes a nil logger into
NewClientFromPathOrConfigWithMiddleware which is fragile; change the test to
pass a real (silent) logger instead of nil (e.g., use logger.GetLogger("test")
or the package's null-logger) when calling
NewClientFromPathOrConfigWithMiddleware so future additions that call logging
(e.g., in NewClientWithMiddleware or normalizeKubeconfigOrPath) won't cause a
nil-pointer panic; update the client, restConfig, err :=
NewClientFromPathOrConfigWithMiddleware(...) invocation to supply that logger.

In `@Makefile`:
- Around line 94-101: The Makefile target PROPERTIES.md invokes the external
claude CLI without checking availability, which causes a confusing "command not
found" for fresh checkouts; update the PROPERTIES.md target (the rule that pipes
the claude command into captain) to first check for the claude binary (e.g., use
a shell check like command -v claude) and print a clear error and non-zero exit
if missing, or gate the target with a phony prerequisite that validates claude
presence and emits a helpful install/documentation message before attempting to
run claude.

In `@models/common.go`:
- Around line 91-95: The HealthUnknown case in the badge switch duplicates the
default branch in the function that returns api.Badge (located in
models/common.go), so remove the explicit "case HealthUnknown:" branch to rely
on the default branch, or if you want a distinct presentation keep the case but
change its styling to differ from default (e.g., use the same styling as
Pretty() or a unique bg/text class) and update the api.Badge call in that branch
accordingly.

In `@query/config_tree.go`:
- Around line 35-98: Update the MergeConfigTrees docstring to explicitly state
that the returned "forest" can be a DAG with shared *ConfigTreeNode pointers
across roots (because cloneConfigTree returns existing nodes from byID), so
callers should not assume disjoint trees; note that mutations to a shared node
will affect all roots that reference it, JSON serializers may duplicate the
shared subtree under each parent, and walkers should use a visited set to avoid
repeated processing (refer to MergeConfigTrees, cloneConfigTree, and the
behavior exercised by TestMergeConfigTreesSharedInternalNode).
- Around line 81-98: mergeConfigTreeInto currently only updates EdgeType and can
silently drop a non-empty Relation from src when dst.Relation is empty; change
the merge logic in mergeConfigTreeInto so that when visiting the same node
(function mergeConfigTreeInto) you also merge Relation with explicit precedence:
if dst.Relation == "" and src.Relation != "" then set dst.Relation =
src.Relation (but do not overwrite a non-empty dst.Relation), and ensure this
applies both when promoting EdgeType to "target" and when appending cloned
children (cloneConfigTree) so the preserved relation survives merges and is not
lost due to processing order.
- Around line 225-238: getExistingConfigsByIDs silently skips missing IDs
(errors.Is(err, gorm.ErrRecordNotFound)) which changes behavior; keep the same
skip but emit a debug-level log when ConfigItemFromCache returns
gorm.ErrRecordNotFound so stale-cache or cross-shard misses are visible without
noise. Modify getExistingConfigsByIDs to, inside the errors.Is(err,
gorm.ErrRecordNotFound) branch, call the project's debug logger (consistent with
existing logging usage in this package) to log the missing id (id.String()) and
brief context, then continue; leave other error handling and return values
unchanged.

In `@tests/config_access_test.go`:
- Around line 71-132: Test inserts deletedGroup, deletedUser and
deletedMembership but never removes them; add cleanup to delete those rows after
the spec runs (use Ginkgo's DeferCleanup or AfterEach) by invoking
DefaultContext.DB().Delete on models.ExternalGroup, models.ExternalUser and
models.ExternalUserGroup using the inserted IDs (deletedGroup.ID, deletedUser.ID
and the composite keys from deletedMembership) so the temporary rows are removed
and do not leak into subsequent tests.

In `@tests/config_relationship_test.go`:
- Around line 440-502: The test creates a random scraperID (scraperID) and
assigns it to ownedRel, which could break if a foreign key is later added;
change the initializer in the BeforeAll so scraperID (and ownedRel.ScraperID)
uses an existing dummy fixture scraper UUID (e.g., dummy.Scraper.ID or the
appropriate existing dummy scraper field) instead of uuid.New(), keeping the
same scraperID variable and ownedRel construction so the rest of the test and
UpdateIsPushed call remain unchanged.

In `@views/038_config_access.sql`:
- Around line 16-38: The JOIN currently places non-join predicates
(config_access.deleted_at IS NULL and config_access.external_group_id IS NOT
NULL) inside the ON clause of the INNER JOIN between config_access and the CTE
active_external_user_groups; move those two predicates out of the ON clause and
into a WHERE clause (keeping the INNER JOIN only for matching
active_external_user_groups.external_group_id = config_access.external_group_id)
so the intent is clearer and the join predicate only expresses the join key for
config_access and active_external_user_groups.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 95dcc6e2-e5c0-40f5-9d41-d2ea45e3c396

📥 Commits

Reviewing files that changed from the base of the PR and between 0fd35df and 7de634a.

📒 Files selected for processing (44)
  • .gitignore
  • Makefile
  • PROPERTIES.md
  • PROPERTIES.schema.json
  • Taskfile.yaml
  • connection/aws.go
  • connection/cnrm.go
  • connection/common.go
  • connection/eks.go
  • connection/gcs.go
  • connection/git.go
  • connection/git_logging_test.go
  • connection/gke.go
  • connection/http.go
  • connection/kubernetes.go
  • connection/observability_test.go
  • connection/opensearch.go
  • connection/prometheus.go
  • context/properties.go
  • context/template.go
  • kubernetes/k8s.go
  • kubernetes/utils_test.go
  • models/common.go
  • models/config.go
  • models/config_access.go
  • query/config_tree.go
  • query/config_tree_test.go
  • query/resource_selector.go
  • rbac/objects.go
  • schema/config.hcl
  • schema/config_access.hcl
  • start.go
  • start_test.go
  • suite_test.go
  • tests/config_access_test.go
  • tests/config_relationship_test.go
  • tests/e2e-blobs/containers.go
  • tests/fixtures/dummy/all.go
  • tests/setup/common.go
  • types/client_options.go
  • views/006_config_views.sql
  • views/038_config_access.sql
  • views/045_merge_external_entities.sql
  • views/9998_rls_enable.sql

Comment on lines +98 to +138
func TestGitCloneTraceLogging(t *testing.T) {
g := gomega.NewWithT(t)

root := t.TempDir()
remote := makeBareRepo(t, root)

logger.UseSlog()

readOut, stop := captureLoggerOutput(t)
defer stop()

setGitLogLevel(t, "trace")

dst := filepath.Join(root, "checkout")
gitClient := &GitClient{URL: remote, Branch: "main", Depth: 1}

ctx := context.NewContext(gocontext.TODO())
_, err := gitClient.Clone(ctx, dst)
g.Expect(err).ToNot(gomega.HaveOccurred())

// Simulate a transport progress line going through the same writer
// Clone hands to go-git, to prove the path is live. A real network
// clone emits lines like this via sideband; local/file clones do not.
fmt.Fprintln(logger.GetLogger("git").V(2), "Counting objects: 3, done.")

stop()
out := readOut()

want := []string{
"clone url=",
"branch=main",
"depth=1",
"dir=" + dst,
"checked out ",
"Counting objects: 3, done.",
"(git)",
}
for _, w := range want {
g.Expect(out).To(gomega.ContainSubstring(w), "trace output missing %q", w)
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check whether commons logger exposes a way to save/restore the active backend.
rg -nP -C3 '\bUseSlog\b|\bSetLogger\b|\bGetLogger\b\s*\(\s*\)' --type=go

Repository: flanksource/duty

Length of output: 2522


🏁 Script executed:

#!/bin/bash
# 1. Find the logger package to check available functions
fd -e '.go' | xargs grep -l 'package logger' | head -5

Repository: flanksource/duty

Length of output: 42


🏁 Script executed:

#!/bin/bash
# 2. Check makeBareRepo implementation
sed -n '1,/^func makeBareRepo/p' connection/git_logging_test.go | tail -30

Repository: flanksource/duty

Length of output: 540


🏁 Script executed:

#!/bin/bash
# 3. Read makeBareRepo full function
rg -A 30 'func makeBareRepo\(' connection/git_logging_test.go

Repository: flanksource/duty

Length of output: 1162


🏁 Script executed:

#!/bin/bash
# 4. Check if there's a way to restore logger in commons
rg -nP 'GetOutput|SetOutput|RestoreLogger|SaveLogger' --type=go | head -20

Repository: flanksource/duty

Length of output: 500


🏁 Script executed:

#!/bin/bash
# Find logger package definition
find . -name "*.go" -path "*/commons/*" | xargs grep -l 'func UseSlog' | head -3

Repository: flanksource/duty

Length of output: 42


🏁 Script executed:

#!/bin/bash
# If commons is external, check in vendor or go.mod
rg 'github\.com/flanksource/commons' go.mod | head -5

Repository: flanksource/duty

Length of output: 156


🏁 Script executed:

#!/bin/bash
# Search for any logger implementation getter/setter for the backend itself
rg -nP '\bGetLogger\b\s*\(\s*\)|\bSetLogger\b\s*\(\s*logger' --type=go | head -15

Repository: flanksource/duty

Length of output: 301


🏁 Script executed:

#!/bin/bash
# Check the complete git_logging_test.go to see all test functions and cleanup patterns
wc -l connection/git_logging_test.go && sed -n '1,50p' connection/git_logging_test.go

Repository: flanksource/duty

Length of output: 1544


🏁 Script executed:

#!/bin/bash
# Read the second test function mentioned (lines 143-174)
sed -n '143,174p' connection/git_logging_test.go

Repository: flanksource/duty

Length of output: 1001


🏁 Script executed:

#!/bin/bash
# Check for any cleanup/defer patterns with logger in the file
rg -n 'defer|t\.Cleanup|t\.TempDir' connection/git_logging_test.go

Repository: flanksource/duty

Length of output: 256


🏁 Script executed:

#!/bin/bash
# Search for other test files that call UseSlog to see if they restore it
rg -l 'UseSlog' --type=go | xargs -I {} sh -c 'echo "=== {} ===" && grep -A 5 -B 2 "UseSlog" {}'

Repository: flanksource/duty

Length of output: 626


🏁 Script executed:

#!/bin/bash
# Check commons logger package documentation/API in go.mod or vendor
find . -path "*vendor/github.com/flanksource/commons/logger*" -name "*.go" 2>/dev/null | head -3

Repository: flanksource/duty

Length of output: 42


🏁 Script executed:

#!/bin/bash
# Read captureLoggerOutput to understand the complete logger setup/teardown
sed -n '46,90p' connection/git_logging_test.go

Repository: flanksource/duty

Length of output: 1322


🏁 Script executed:

#!/bin/bash
# Check if there are other test files that might depend on non-slog logger behavior
find . -name '*_test.go' -type f | xargs grep -l 'logger\.' | wc -l

Repository: flanksource/duty

Length of output: 62


🏁 Script executed:

#!/bin/bash
# Verify if UseSlog is truly idempotent by checking if multiple calls cause issues
rg -B 5 -A 10 'func UseSlog' --type=go | head -20

Repository: flanksource/duty

Length of output: 42


🏁 Script executed:

#!/bin/bash
# Check if there's a GetLogger() or similar that returns the current implementation
rg -nP 'func.*Get.*ogger\b.*\(\)' --type=go

Repository: flanksource/duty

Length of output: 42


🏁 Script executed:

#!/bin/bash
# Check if commons has this function by looking for any pattern that might help
rg 'SetLogger|GetLogger|RestoreLogger' --type=go -B 2 -A 2 | grep -E '(func|logger)' | head -20

Repository: flanksource/duty

Length of output: 1599


🏁 Script executed:

#!/bin/bash
# Verify the complete setGitLogLevel function to confirm cleanup pattern
sed -n '87,95p' connection/git_logging_test.go

Repository: flanksource/duty

Length of output: 413


🏁 Script executed:

#!/bin/bash
# Check if UseSlog() is called elsewhere and if tests following it depend on non-slog behavior
rg -B 3 -A 8 'func Test' connection/git_logging_test.go | grep -E '(func Test|UseSlog|Cleanup)'

Repository: flanksource/duty

Length of output: 195


Add git availability check to makeBareRepo to skip gracefully when unavailable.

The test helper shells out to git via exec.Command without checking if the binary exists. On environments without git installed (minimal CI runners, etc.), the test fails rather than skips. Use exec.LookPath("git") and t.Skip(...) to degrade gracefully.

♻️ Skip when git is unavailable
 func makeBareRepo(t *testing.T, root string) string {
 	t.Helper()
 	g := gomega.NewWithT(t)
+	if _, err := exec.LookPath("git"); err != nil {
+		t.Skipf("git binary not available: %v", err)
+	}

 	work := filepath.Join(root, "work")

Also applies to: 143-174

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@connection/git_logging_test.go` around lines 98 - 138, The helper
makeBareRepo invokes the git executable without checking availability, causing
test failures on systems without Git; modify makeBareRepo to call
exec.LookPath("git") at the start and call t.Skipf(...) if git is not found so
the test is skipped gracefully, and apply the same availability check/skip to
any other test helpers in this file that shell out to git (the other helper used
around the later block that also runs exec.Command("git", ...)) so tests do not
fail on minimal CI images.

Comment thread connection/git.go
"github.com/flanksource/duty/types"
)

var gitHTTPTransportMu sync.Mutex
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚖️ Poor tradeoff

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Locate any other callers of go-git's http/transport client registration that could collide.
rg -nP -C3 'plumbing/transport/client' --type=go
rg -nP -C3 'InstallProtocol\s*\(' --type=go
# Find other Clone/Fetch call sites that could run concurrently.
rg -nP -C3 '\bgit\.PlainClone(Context)?\s*\(|\bFetchContext\s*\(' --type=go

Repository: flanksource/duty

Length of output: 1942


🏁 Script executed:

#!/bin/bash
# Read the Clone function to see where configureGitHTTPTransport is called and deferred
sed -n '80,200p' connection/git.go | cat -n

Repository: flanksource/duty

Length of output: 4939


🏁 Script executed:

#!/bin/bash
# Get the complete configureGitHTTPTransport function and surrounding context
sed -n '375,410p' connection/git.go | cat -n

Repository: flanksource/duty

Length of output: 949


🏁 Script executed:

#!/bin/bash
# Search for all Clone method definitions and FetchContext usage in this file
rg -n 'func.*Clone|defer.*configureGitHTTPTransport|FetchContext' connection/git.go

Repository: flanksource/duty

Length of output: 214


🏁 Script executed:

#!/bin/bash
# Search for other callers of GitClient.Clone or direct go-git usage
rg -n 'GitClient.*Clone|git\.PlainClone|go-git' --type=go -i | head -30

Repository: flanksource/duty

Length of output: 1348


🏁 Script executed:

#!/bin/bash
# Search for goroutine patterns that might call Clone
rg -n 'go\s+.*Clone|Clone.*goroutine' --type=go -i

Repository: flanksource/duty

Length of output: 42


🏁 Script executed:

#!/bin/bash
# Check if InstallProtocol is called anywhere else
rg -rn 'InstallProtocol' --type=go

Repository: flanksource/duty

Length of output: 282


🏁 Script executed:

#!/bin/bash
# Find all callers of .Clone()
rg -n '\.Clone\(' --type=go | grep -v test | grep -v 'func.*Clone'

Repository: flanksource/duty

Length of output: 1483


🏁 Script executed:

#!/bin/bash
# Search for direct uses of git package operations outside of GitClient
rg -n 'git\.Plain|git\.Open' --type=go | grep -v test | grep -v connection/git.go

Repository: flanksource/duty

Length of output: 42


🏁 Script executed:

#!/bin/bash
# Look at the Clone call in shell.go and surrounding context
sed -n '310,330p' shell/shell.go | cat -n

Repository: flanksource/duty

Length of output: 662


🏁 Script executed:

#!/bin/bash
# Search for callers of the function containing the Clone call
rg -B20 'client\.Clone\(ctx, result\.mountPoint\)' shell/shell.go | head -40

Repository: flanksource/duty

Length of output: 849


Global mutex serializes concurrent git operations process-wide when HTTP observability is enabled.

configureGitHTTPTransport mutates go-git's package-global protocol registry via gitClient.InstallProtocol (lines 391–392), and gitHTTPTransportMu is held from Lock (line 389) until the deferred cleanup runs (line 26 in the Clone function, via defer restoreGitTransport()). Consequently:

  1. When HAR/logging is enabled for "git", every Clone call across all goroutines is fully serialized on this mutex for the duration of the clone (potentially many seconds for large repos).
  2. The mutated http/https protocol clients are visible to any go-git user in the same process for the lock window — not just calls flowing through GitClient.Clone — which is broader than intended.

The per-mountpoint lock in shell/shell.go (line 320) protects individual checkout operations, but the global mutex still serializes all concurrent Clone operations across different repositories.

go-git does not expose a per-CloneOptions HTTP client, so the global registration is a known constraint. Consider mitigations:

  • Document this serialization behavior explicitly in code comments so future maintainers understand the performance cost.
  • Install the wrapped clients once at startup (when observability is configured) rather than per-clone, eliminating per-operation lock contention. The middleware closure can read per-call context via http.Request.Context().
  • If per-call context varies by operation, use a lightweight RoundTripper that dispatches based on a context value, installed once globally.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@connection/git.go` at line 29, The global mutex gitHTTPTransportMu and the
configureGitHTTPTransport function currently install wrapped http/https
protocols via gitClient.InstallProtocol during each GitClient.Clone and hold the
lock for the entire clone, which serializes all clones process-wide; change this
by (1) adding an explanatory comment near gitHTTPTransportMu and
configureGitHTTPTransport/GitClient.Clone describing this serialization and
visibility to all go-git users, and (2) refactor to install the wrapped protocol
clients once at startup (when HTTP observability/logging is enabled) instead of
per-clone — or alternatively implement and install a single lightweight
RoundTripper that dispatches behavior based on http.Request.Context() so
per-call context is respected without re-registering protocols; ensure
references to gitClient.InstallProtocol, configureGitHTTPTransport,
restoreGitTransport, and GitClient.Clone are updated to reflect the new
startup/one-time registration approach and remove per-clone lock usage.

Comment thread context/template.go
Comment on lines +18 to +25
l := k.Logger.Named("template")
if l.V(3).Enabled() {
l.V(3).Infof("Running template: %s with environment: %v", t.String(), logger.Pretty(env))
} else if l.IsLevelEnabled(logger.Trace) {
l.V(2).Infof("Running template: %s with environment keys: %v", t.String(), lo.Keys(env))
} else {
l.V(1).Infof("Running template: %s", t.String())
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don't dump raw template inputs and outputs to the logger.

These new branches log the full env map and the fully rendered value. Template environments regularly carry secrets, so turning verbosity up will now spill credentials and tokens into logs.

🔒 Suggested direction
-	if l.V(3).Enabled() {
-		l.V(3).Infof("Running template: %s with environment: %v", t.String(), logger.Pretty(env))
-	} else if l.IsLevelEnabled(logger.Trace) {
+	if l.IsLevelEnabled(logger.Trace) {
 		l.V(2).Infof("Running template: %s with environment keys: %v", t.String(), lo.Keys(env))
 	} else {
 		l.V(1).Infof("Running template: %s", t.String())
 	}
@@
-			} else if t.Template != val {
-				if l.V(2).Enabled() {
-					l.V(2).Infof("%s = %s", t.String(), val)
-				} else if l.V(1).Enabled() {
-					l.V(1).Infof("templated %s = changed", t.String())
-				}
+			} else if t.Template != val {
+				l.V(1).Infof("templated %s = changed", t.String())
 			}

Also applies to: 58-65

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@context/template.go` around lines 18 - 25, The current logging branches in
context/template.go leak secrets by printing the full env and rendered template
(see l.V(3).Infof using logger.Pretty(env) and the high-verbosity rendered
output); replace these prints with non-sensitive summaries: log only env keys or
a count (use lo.Keys(env) or len(env)) and never log the full rendered
template/value. Update the l.V(3).Infof and the analogous branch at lines 58-65
to call a new helper like sanitizeEnvKeysOrMask(env) (or simply use
lo.Keys(env)/len(env)) and remove any logger.Pretty(env)/rendered value outputs
so secrets are not emitted.

Comment thread Makefile Outdated
Comment thread PROPERTIES.md Outdated
Comment on lines +272 to +279
searchSetID = slices.ContainsFunc(flatFields, func(s string) bool {
field := strings.ToLower(s)
if alias, ok := qm.Aliases[field]; ok {
field = alias
}
return field == "id"
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect Aliases definitions for the query models to confirm id-related aliases resolve to "id".
rg -nP --type=go -C5 '\bAliases\s*:\s*map\[string\]string' 
ast-grep --pattern 'Aliases: map[string]string{ $$$ }'

Repository: flanksource/duty

Length of output: 4042


🏁 Script executed:

#!/bin/bash
# Check how searchSetID is used and what its impact is
rg -n 'searchSetID' query/resource_selector.go -B2 -A2

# Also check if there are any references to config_id, component_id as primary identifiers
rg -n 'config_id|component_id' query/resource_selector.go -C2

Repository: flanksource/duty

Length of output: 1031


searchSetID only recognizes literal "id" field, but tables use specific id variants like config_id and component_id without alias mappings.

The id detection currently checks only for the literal string "id", but the codebase uses table-specific identifiers (e.g., config_id for config-related tables, component_id for components). None of the QueryModel alias maps contain entries mapping these back to "id". This means queries using config_id or component_id will not set searchSetID=true and the auto-agent filter at line 294 will still apply even when users explicitly query by id—a silent correctness bug.

Either:

  1. Add alias entries ("config_id": "id", "component_id": "id", etc.) to the respective QueryModels, or
  2. Expand the searchSetID check to recognize table-specific id field patterns.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@query/resource_selector.go` around lines 272 - 279, searchSetID only matches
the literal "id", missing table-specific ids like "config_id" or "component_id";
update the searchSetID computation (the slice ContainsFunc over flatFields that
uses qm.Aliases) to consider both canonical alias mapping and table-specific id
patterns—e.g., after resolving alias (field := strings.ToLower(s); if alias,
field = alias), treat the field as matching if field == "id" OR
strings.HasSuffix(field, "_id") (or match a regex like ^[a-z]+_id$), so queries
explicitly on config_id/component_id will set searchSetID=true and avoid the
auto-agent filter being applied.

Comment thread schema/config_access.hcl
Comment on lines +127 to +131
column "scraper_id" {
type = uuid
null = false
default = "00000000-0000-0000-0000-000000000000"
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Drop the steady-state default for external_user_groups.scraper_id after backfill.

Now that scraper_id is part of the primary key, leaving the nil-UUID default here turns missed propagation bugs into silent legacy rows instead of failing fast. That makes it easy to split memberships across 00000000-0000-0000-0000-000000000000 and the real scraper owner.

🧭 Suggested direction
   column "scraper_id" {
     type    = uuid
     null    = false
-    default = "00000000-0000-0000-0000-000000000000"
   }

If the default is only needed to migrate existing rows, keep it in the migration/backfill step and drop it from the steady-state schema.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
column "scraper_id" {
type = uuid
null = false
default = "00000000-0000-0000-0000-000000000000"
}
column "scraper_id" {
type = uuid
null = false
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@schema/config_access.hcl` around lines 127 - 131, The schema currently sets a
steady-state default "00000000-0000-0000-0000-000000000000" for column
"scraper_id" on table external_user_groups which can hide missed propagation
bugs; remove the default from the production HCL declaration so the column has
no default, and instead ensure any one-off migration/backfill step sets that
nil-UUID only during backfill (e.g., in the migration script that updates
external_user_groups) and then drops/does not persist the default in the
steady-state schema (update the column "scraper_id" block to remove the default
and keep null=false/type=uuid).

Comment thread tests/config_access_test.go
Comment thread tests/setup/common.go
Comment on lines 186 to +188
postgresDBUrl = url
dbName = randomDatabaseName("duty_test")

var err error
PgUrl, err = databaseURL(url, dbName)
if err != nil {
return context.Context{}, err
}

quotedDBName := quoteIdentifier(dbName)
_ = execPostgres(postgresDBUrl, fmt.Sprintf("DROP DATABASE IF EXISTS %s (FORCE)", quotedDBName))
if err := execPostgres(postgresDBUrl, fmt.Sprintf("CREATE DATABASE %s", quotedDBName)); err != nil {
dbName = fmt.Sprintf("duty_gingko%d", port)
PgUrl = strings.Replace(url, "/postgres", "/"+dbName, 1)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Parse the DSN and replace only the database path.

Both of these rewrites operate on the full URL string. That will fail whenever DUTY_DB_URL is not pointing at /postgres, and ReplaceAll can also rewrite matching substrings in credentials, query params, or the host.

🛠️ Suggested fix
+func withDatabase(rawURL, dbName string) (string, error) {
+	u, err := url.Parse(rawURL)
+	if err != nil {
+		return "", err
+	}
+	u.Path = "/" + dbName
+	return u.String(), nil
+}
+
 	postgresDBUrl = url
 	dbName = fmt.Sprintf("duty_gingko%d", port)
-	PgUrl = strings.Replace(url, "/postgres", "/"+dbName, 1)
+	PgUrl, err = withDatabase(url, dbName)
+	if err != nil {
+		return context.Context{}, err
+	}
@@
-	config := api.NewConfig(strings.ReplaceAll(pgUrl, pgDbName, newName))
+	nextURL, err := withDatabase(pgUrl, newName)
+	if err != nil {
+		return nil, nil, err
+	}
+	config := api.NewConfig(nextURL)

Also applies to: 282-282

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/setup/common.go` around lines 186 - 188, The current code replaces
"/postgres" by string manipulation which can corrupt credentials or query
params; instead parse the DSN with net/url (or the appropriate driver DSN
parser), set the path to the new database name (dbName) and re-encode it before
assigning PgUrl; update the logic around postgresDBUrl, dbName and PgUrl so you
construct PgUrl by parsing postgresDBUrl, replacing only the URL.Path (or the
DSN's database field) with "/"+dbName, and serializing back to a string.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
connection/git_logging_test.go (1)

25-27: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Skip test when git is unavailable instead of failing hard.

At Line 26, this uses t.Fatalf(...) when git is not installed. For portability in minimal CI environments, this should t.Skipf(...) so the test degrades gracefully instead of reporting a failure.

Suggested change
 	if _, err := exec.LookPath("git"); err != nil {
-		t.Fatalf("git executable not found in PATH: %v", err)
+		t.Skipf("git executable not found in PATH: %v", err)
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@connection/git_logging_test.go` around lines 25 - 27, Replace the hard
failure when git isn't available with a test skip: in
connection/git_logging_test.go change the exec.LookPath("git") error branch to
call t.Skipf(...) instead of t.Fatalf(...), so the test function (the test using
exec.LookPath("git")) will be skipped gracefully when the git executable is not
found in PATH and will include the error in the skip message for debugging.
🧹 Nitpick comments (1)
PROPERTIES.md (1)

169-169: 💤 Low value

Minor grammar: compound adjectives should be hyphenated.

📝 Proposed fixes
-| `--otel-service-name` | caller supplied | OpenTelemetry service name. |
+| `--otel-service-name` | caller-supplied | OpenTelemetry service name. |

And at line 200:

-| `log.level.http.har` | string | unset | Enables HAR capture for HTTP at `debug`; includes full body capture at `trace`. |
+| `log.level.http.har` | string | unset | Enables HAR capture for HTTP at `debug`; includes full-body capture at `trace`. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@PROPERTIES.md` at line 169, The table entry for `--otel-service-name` uses
the unhyphenated compound adjective "caller supplied"; update the text to use a
hyphenated form ("caller-supplied") so the cell reads "`--otel-service-name` |
caller-supplied | OpenTelemetry service name." and ensure any other occurrences
of "caller supplied" in PROPERTIES.md are also hyphenated consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Makefile`:
- Around line 101-105: The Makefile target PROPERTIES.md invokes the claude CLI
without verifying it exists; add a pre-check (like using `command -v claude` or
an equivalent guard) before the claude invocation in the PROPERTIES.md recipe so
it prints a clear, actionable error and exits if claude is missing; update the
PROPERTIES.md target (the recipe that runs claude -p ... | captain) to perform
this validation and fail fast with a helpful message when claude is not found.

In `@PROPERTIES.md`:
- Around line 241-242: The `<event>.debug` and `<event>.trace` entries are
contradictory (they claim enabling when set to `off`/`false`); first verify
whether the async event consumer code interprets these flags inverted—if the
consumer currently enables logging when the flag is false, change the consumer
logic so that true/on enables debug/trace (and update any flag parsing/negation
in the consumer where these flags are read); otherwise, if the code is correct,
update PROPERTIES.md to state that setting `<event>.debug` and `<event>.trace`
to `true`/`on` enables logging (default `false`) and remove the “off/false”
wording. Ensure both the code (if changed) and the docs consistently reflect
that true/on enables the respective logging.

---

Duplicate comments:
In `@connection/git_logging_test.go`:
- Around line 25-27: Replace the hard failure when git isn't available with a
test skip: in connection/git_logging_test.go change the exec.LookPath("git")
error branch to call t.Skipf(...) instead of t.Fatalf(...), so the test function
(the test using exec.LookPath("git")) will be skipped gracefully when the git
executable is not found in PATH and will include the error in the skip message
for debugging.

---

Nitpick comments:
In `@PROPERTIES.md`:
- Line 169: The table entry for `--otel-service-name` uses the unhyphenated
compound adjective "caller supplied"; update the text to use a hyphenated form
("caller-supplied") so the cell reads "`--otel-service-name` | caller-supplied |
OpenTelemetry service name." and ensure any other occurrences of "caller
supplied" in PROPERTIES.md are also hyphenated consistently.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 874a5ce9-f7e0-41c7-b588-7f1469899e62

📥 Commits

Reviewing files that changed from the base of the PR and between 7de634a and 7ff338f.

📒 Files selected for processing (7)
  • Makefile
  • PROPERTIES.md
  • Taskfile.yaml
  • connection/git_logging_test.go
  • query/config_tree.go
  • query/resource_selector.go
  • tests/config_access_test.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • Taskfile.yaml
  • tests/config_access_test.go
  • query/config_tree.go

Comment thread Makefile
Comment on lines +101 to +105
.PHONY: PROPERTIES.md
PROPERTIES.md: captain
claude -p --permission-mode acceptEdits --verbose --output-format stream-json --model sonnet \
"/properties-doc Refresh PROPERTIES.md and PROPERTIES.schema.json from the current source tree. Cross-reference ../incident-commander, ../config-db, ../canary-checker, ../flanksource-ui, and ../commons when present. Run in update mode and preserve hand-written prose sections." \
| captain
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

claude CLI is not validated before use.

captain gets an install guard (command -v), but the claude CLI invoked on line 103 has no equivalent check. A missing claude binary will fail with an unhelpful command not found error instead of an actionable message.

🛡️ Proposed fix
 .PHONY: PROPERTIES.md
 PROPERTIES.md: captain
+	`@command` -v claude >/dev/null || (echo "Claude CLI is required. Install it from https://docs.anthropic.com/en/docs/claude-code/cli-usage" && exit 1)
 	claude -p --permission-mode acceptEdits --verbose --output-format stream-json --model sonnet \
 		"/properties-doc Refresh PROPERTIES.md and PROPERTIES.schema.json from the current source tree. Cross-reference ../incident-commander, ../config-db, ../canary-checker, ../flanksource-ui, and ../commons when present. Run in update mode and preserve hand-written prose sections." \
 		| captain
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
.PHONY: PROPERTIES.md
PROPERTIES.md: captain
claude -p --permission-mode acceptEdits --verbose --output-format stream-json --model sonnet \
"/properties-doc Refresh PROPERTIES.md and PROPERTIES.schema.json from the current source tree. Cross-reference ../incident-commander, ../config-db, ../canary-checker, ../flanksource-ui, and ../commons when present. Run in update mode and preserve hand-written prose sections." \
| captain
.PHONY: PROPERTIES.md
PROPERTIES.md: captain
`@command` -v claude >/dev/null || (echo "Claude CLI is required. Install it from https://docs.anthropic.com/en/docs/claude-code/cli-usage" && exit 1)
claude -p --permission-mode acceptEdits --verbose --output-format stream-json --model sonnet \
"/properties-doc Refresh PROPERTIES.md and PROPERTIES.schema.json from the current source tree. Cross-reference ../incident-commander, ../config-db, ../canary-checker, ../flanksource-ui, and ../commons when present. Run in update mode and preserve hand-written prose sections." \
| captain
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Makefile` around lines 101 - 105, The Makefile target PROPERTIES.md invokes
the claude CLI without verifying it exists; add a pre-check (like using `command
-v claude` or an equivalent guard) before the claude invocation in the
PROPERTIES.md recipe so it prints a clear, actionable error and exits if claude
is missing; update the PROPERTIES.md target (the recipe that runs claude -p ...
| captain) to perform this validation and fail fast with a helpful message when
claude is not found.

Comment thread PROPERTIES.md
Comment on lines +241 to +242
| `<event>.debug` | bool/off switch | `false` | Enables debug logging for a named async event consumer when set to `off`/`false` by current code. |
| `<event>.trace` | bool/off switch | `false` | Enables trace logging for a named async event consumer when set to `off`/`false` by current code. |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Contradictory property descriptions for <event>.debug and <event>.trace.

Both rows say "Enables … logging … when set to off/false by current code" — enabling a feature when set to its off value is self-contradictory. Either the documentation should say true/on, or this is documenting an inverted-logic bug in the consumer code that should be fixed rather than just noted.

Please clarify which is the case and correct accordingly.

📝 Proposed fix (if the description is simply inverted)
-| `<event>.debug` | bool/off switch | `false` | Enables debug logging for a named async event consumer when set to `off`/`false` by current code. |
-| `<event>.trace` | bool/off switch | `false` | Enables trace logging for a named async event consumer when set to `off`/`false` by current code. |
+| `<event>.debug` | bool/off switch | `false` | Enables debug logging for a named async event consumer when set to `true`/`on`. |
+| `<event>.trace` | bool/off switch | `false` | Enables trace logging for a named async event consumer when set to `true`/`on`. |
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| `<event>.debug` | bool/off switch | `false` | Enables debug logging for a named async event consumer when set to `off`/`false` by current code. |
| `<event>.trace` | bool/off switch | `false` | Enables trace logging for a named async event consumer when set to `off`/`false` by current code. |
| `<event>.debug` | bool/off switch | `false` | Enables debug logging for a named async event consumer when set to `true`/`on`. |
| `<event>.trace` | bool/off switch | `false` | Enables trace logging for a named async event consumer when set to `true`/`on`. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@PROPERTIES.md` around lines 241 - 242, The `<event>.debug` and
`<event>.trace` entries are contradictory (they claim enabling when set to
`off`/`false`); first verify whether the async event consumer code interprets
these flags inverted—if the consumer currently enables logging when the flag is
false, change the consumer logic so that true/on enables debug/trace (and update
any flag parsing/negation in the consumer where these flags are read);
otherwise, if the code is correct, update PROPERTIES.md to state that setting
`<event>.debug` and `<event>.trace` to `true`/`on` enables logging (default
`false`) and remove the “off/false” wording. Ensure both the code (if changed)
and the docs consistently reflect that true/on enables the respective logging.

@moshloop moshloop merged commit a2b1286 into main May 7, 2026
19 of 20 checks passed
@moshloop moshloop deleted the pr/har-logging branch May 7, 2026 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants