Skip to content

CapabilityRegulatoryTriage + CapabilityRegulatoryAnalysis in analyst.go #94

@scttfrdmn

Description

@scttfrdmn

Summary

Add two-stage regulatory intelligence AI pipeline to internal/ai/analyst.go.

Two-Stage Design

Stage 1 — Triage (Haiku, ~$0.001/notice): Is this notice relevant to research computing compliance? Filters ~90% of Federal Register items cheaply. Input: title + abstract only.

Stage 2 — Analysis (Sonnet): Deep analysis for relevant notices. Full document text (12KB truncated). Returns structured JSON with issue title, body, affected frameworks, action required.

New Capabilities

CapabilityRegulatoryTriage   Capability = "regulatory_triage"    // Haiku
CapabilityRegulatoryAnalysis Capability = "regulatory_analysis"  // Sonnet

New Method

func (a *Analyst) AnalyzeRegulatoryNotice(ctx context.Context, n regulatory.Notice) (*regulatory.RelevanceResult, error)

Internally: triage first, return early if not relevant, then deep analysis.

Prompt Structure (Analysis stage)

System prompt uses existing buildSystemPrompt() for SRE/framework context. Notice content wrapped in XML delimiters with closing-tag escaping (same pattern as AnalyzeAnomalies). Response parsed as JSON into RelevanceResult.

type RelevanceResult struct {
    IsRelevant         bool
    AffectedFrameworks []string
    Impact             string   // "new-control"|"control-update"|"new-framework"|"conflict-update"|"info-only"
    ActionRequired     string
    IssueTitle         string
    IssueBody          string   // full GitHub issue markdown
    Labels             []string
}

Acceptance Criteria

  • CapabilityRegulatoryTriage uses Haiku; CapabilityRegulatoryAnalysis uses Sonnet (selectModel updated)
  • Triage prompt: title + abstract only, returns simple JSON {is_relevant: bool}
  • Analysis prompt: sanitized notice text in delimiters, returns full RelevanceResult JSON
  • NOT-OD-25-081 test: returns AffectedFrameworks: ["nih-gds"], Impact: "new-control"
  • Federal Register generic rule test: returns IsRelevant: false (should NOT create an issue)
  • All notice content sanitized via sanitizePromptField() before embedding

Blocked by

#93 (regulatory package — Notice type)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions