docs: Restructure AGENTS.md for better agent guidance and progressive disclosure

[bensonwong]

Summary

Complete rewrite of AGENTS.md to address 33 feedback items, making it more effective for AI code agents implementing DeepCitation.

Key Changes

Structure & Progressive Disclosure

• Added "Agent Behavior Guidelines" at the very top with explicit DO/DON'T list
• Added "Is DeepCitation Right for You?" quick-check section
• Moved Critical Warnings (<<<CITATION_DATA>>> block) to prominent position with visual before/after examples
• Moved complete Quick Start example to near-top (was buried at end)
• Added ASCII Core Workflow diagram with "Why Each Step Exists" explanations
• Reorganized content by user journey (basics → advanced)

Preventing Agent Fabrication

• Created Appendix A: Real URLs (complete list of all legitimate DeepCitation URLs)
• Created Appendix B: What DeepCitation Doesn't Do (explicit limitations)
• Added warnings against fabricating URLs or inventing undocumented methods

Technical Completeness

• Added prepareUrl() method documentation (was missing)
• Added URL processing time warning (30s for URLs vs <1s for images)
• Added unsafeFastUrlOutput security warning for untrusted URLs
• Added error handling patterns with try/catch examples
• Expanded multiple documents pattern with explicit fileDataParts mapping
• Added UI guidance to verification status table

Improved Troubleshooting

• Expanded each issue with: Symptoms, Causes, Solutions, DON'T sections
• Separated troubleshooting from agent guidelines

Collapsible Sections

• Used <details> for LLM provider alternatives (Anthropic, Gemini, Vercel AI SDK)
• Used <details> for React component variants reference
• Used <details> for unsafe URL mode documentation

Reference Improvements

• Added descriptions to Key Imports Reference
• Added Production Checklist in prominent location
• Moved Next.js API routes under Framework Integration section

Feedback Items Addressed

All 33 feedback items from the review have been addressed:

• fix: correct package exports to point to lib/ instead of src #1-5: Structure & progressive disclosure
• Patch/client cleanup #6-7, Rename display props to hide* pattern and disable default click behavior when onClick provided (Vibe Kanban) #18-19: Preventing fabrication with explicit guidelines
• Update examples to use fast/cheap models and add Gemini support #8, Replace diff dependency with custom implementation for Firebase Functions compatibility (Vibe Kanban) #27: Error handling patterns
• Feature/clearer naming #10, Improve system prompt reliability with CoT ordering and RE2 wrap strategy (Vibe Kanban) #32: Expanded troubleshooting
• Patch/rename #11-13: Critical warnings made prominent
• Feature/keyspan support #14-17: Technical completeness (async verification, multiple docs, etc.)
• Rename fileId to attachmentId across the codebase (Vibe Kanban) #20-25: Reference section improvements
• Optimize npm package build with tsup bundler #29-31: Reorganized by user journey
• Migrate VerificationTabs and DiffDisplay to Tailwind CSS with CI test workflow (Vibe Kanban) #33: Added ASCII data flow diagram

Test Plan

• Read through document top-to-bottom as an agent would
• Verify all URLs in Appendix A are legitimate
• Check all code examples compile/are syntactically correct
• Confirm <details> sections render correctly in GitHub markdown
• Verify Quick Start example is complete and runnable

[github-actions]

✅ Playwright Test Report

Status: Tests passed

📊 View Full Report
📸 View Visual Snapshots Gallery

What's in the Visual Snapshots

The gallery includes visual snapshots for:

• 🖥️ Desktop showcase (all variants × all states)
• 📱 Mobile showcase (iPhone SE viewport)
• 📟 Tablet showcase (iPad viewport)
• 🔍 Popover states (verified, partial, not found)
• 🔗 URL citation variants

---

Run ID: 21539096352 | Download Artifacts

[claude]

Pull Request Review: AGENTS.md Improvements

Summary

This PR significantly improves AGENTS.md by restructuring it to be more agent-friendly and addressing 33 specific feedback items. The changes focus on preventing AI agents from fabricating URLs/methods, improving progressive disclosure, and providing clearer examples.

Strengths

1. Excellent Agent Guardrails ✅

The new "Agent Behavior Guidelines" section at the top directly addresses the fabrication problem mentioned in the PR description.

2. Critical Warning Prominence ✅

The <<<CITATION_DATA>>> warning is now prominent with visual examples (lines 54-79). Previously buried, now impossible to miss.

3. Progressive Disclosure ✅

Much better structure: Is this right for you? → Critical warnings → Quick start → Step-by-step → Advanced patterns

4. Complete Quick Start Example ✅

Lines 84-152 provide a fully runnable example upfront, much better than the previous scattered approach.

5. Visual Data Flow Diagram ✅

Lines 162-206 show the workflow with ASCII art.

Critical Issues

1. Streaming Example Bug 🐛 (Lines 808-815)

Issue: The code streams ALL content including the citation block, contradicting the critical warning.

// Current code - WRONG
process.stdout.write(content);  // Streams <<<CITATION_DATA>>> to users!

Fix: Buffer everything, then extract visible text after streaming completes:

// Don't stream raw content - buffer it
fullResponse += content;

// After streaming completes:
const visibleText = extractVisibleText(fullResponse);
// Now send visibleText to client

2. Model Name Verification ⚠️ (Lines 121, 363, 817)

Uses gpt-5-mini which doesn't exist as of January 2025. Either:

• Use current model (gpt-4-turbo)
• Add comment: // Example model - use current OpenAI model

Important Issues

3. Missing Rate Limiting Guidance

PR feedback item #26 not fully addressed. Add:

### Rate Limiting & Caching
- attachmentId valid for 24 hours
- Cache attachmentId to avoid re-uploads
- Check docs for API rate limits

4. API Route Input Validation (Lines 703-705)

Add validation example:

if (!attachmentId || typeof attachmentId !== 'string') {
  return NextResponse.json({ error: "Invalid attachmentId" }, { status: 400 });
}

5. Error Handling (Lines 723-749)

Add comment about string matching fragility:

// Note: Error detection relies on message strings
// TODO: Use typed errors if/when available

Security Concerns

SSRF Warning Missing

Add warning for user-provided URLs (lines 268-276):

**Security**: Validate user-provided URLs to prevent SSRF attacks.
Never allow internal URLs (localhost, 192.168.*, etc.)

Minor Issues

6. Position options table (lines 332-341): Add token cost/performance context
7. TypeScript types (lines 1065-1100): Add property descriptions or docs link
8. Inconsistent code fence labels: Standardize language tags
9. Table alignment: Multiple tables have inconsistent formatting
10. Capitalization: "When shown" → "When Shown" (line 854)

Completeness Check

Addressed: 32/33 feedback items (97%) ✅

Only partially addressed:

• Rate limiting guidance (item Fix TypeScript build errors for CitationComponent and Radix UI types (Vibe Kanban) #26)
• Performance implications details (item Refactor CitationComponent: simplify variants and add displayBrackets prop (Vibe Kanban) #15)

Overall Assessment

Quality: 8.5/10
Completeness: 97%
Agent-Friendliness: 9/10
Structure: 9.5/10

Recommendation

✅ Approve with Changes

Must Fix Before Merge

1. ❌ Fix streaming example (lines 808-815) - critical bug
2. ⚠️ Verify/clarify model name (lines 121, 363, 817)

Should Fix

3. Add rate limiting guidance
4. Add input validation example
5. Add SSRF security warning

Nice to Have

6-10. Minor documentation enhancements listed above

---

Great work addressing the extensive feedback! This will significantly reduce agent fabrication and improve UX. Just need to fix those 2 critical code examples before merge. 🎉

Files Changed: 1 (AGENTS.md)
Lines: +585 / -297

[claude]

PR Review: AGENTS.md Restructure

Thank you for this comprehensive rewrite of AGENTS.md! The restructuring significantly improves the document's usefulness for AI agents. Here's my detailed review:

---

Strengths ✅

1. Excellent Progressive Disclosure

• The "Agent Behavior Guidelines" at the top is perfect - clear DO/DON'T list
• "Is DeepCitation Right for You?" quick-check section helps agents avoid misapplication
• Critical warnings about <<<CITATION_DATA>>> block are now prominent with visual examples

2. Prevention of Agent Fabrication

• Appendix A (Real URLs) is brilliant - prevents agents from inventing endpoints
• Appendix B (What DeepCitation Doesn't Do) sets clear boundaries
• Explicit warnings against fabricating URLs/methods

3. Improved Technical Completeness

• Added prepareUrl() documentation (was missing)
• Processing time warnings (30s for URLs vs <1s for images) are critical
• Security warning for unsafeFastUrlOutput and SSRF prevention
• Error handling patterns with try/catch examples
• Input validation in verify API route

4. Better Organization

• Core workflow ASCII diagram with "Why Each Step Exists" is excellent
• Quick Start moved to top (was buried)
• Collapsible sections reduce cognitive overload
• User journey structure (basics → advanced)

---

Issues & Recommendations 🔍

High Priority

1. Streaming Example Needs Clarification (AGENTS.md:846-873)

The streaming example says "Don't stream raw content directly to users!" but doesn't clearly explain when to use Option A vs Option B:

// IMPORTANT: Don't stream raw content directly to users!
// The <<<CITATION_DATA>>> block arrives at the end and must be stripped.
//
// Option A: Buffer everything, send after complete (simpler)
// Option B: Stream chunks but filter out the citation block (complex)

Recommendation: Add a decision tree or more explicit guidance:

// RECOMMENDED: Option A for most use cases
// Buffer the full response server-side, then extract visible text

// ADVANCED: Option B only if you need progressive streaming
// Requires detecting "<<<CITATION_DATA>>>" and stopping stream

2. Model Name Comment Ambiguity (AGENTS.md:121, 836)

The comment "Use your preferred model" appears multiple times but might confuse agents:

model: "gpt-5-mini",  // Use your preferred model

Recommendation: Be more explicit:

model: "gpt-5-mini",  // Replace with your model (e.g., "gpt-4o", "gpt-4-turbo")

3. Missing File Hash Implementation (AGENTS.md:1081)

The caching example uses fileHash but doesn't show how to generate it:

async function getAttachmentId(fileHash: string, file: Buffer, filename: string) {
  const cached = attachmentCache.get(fileHash);

Recommendation: Add hash generation example:

import { createHash } from 'crypto';

function getFileHash(file: Buffer): string {
  return createHash('sha256').update(file).digest('hex');
}

async function getAttachmentId(file: Buffer, filename: string) {
  const fileHash = getFileHash(file);
  // ... rest of implementation
}

Medium Priority

4. Status Table Inconsistency (AGENTS.md:904-913)

The "Detailed Status Values" table adds "UI Guidance" column, which is helpful, but some entries are vague:

• "found_phrase_missed_anchor_text" → "Show as verified" (what does this mean visually?)
• "first_word_found" → "Consider not showing" (contradicts showing with ⚠️)

Recommendation: Make UI guidance more actionable:

| Status | UI Guidance |
|--------|-------------|
| "found_phrase_missed_anchor_text" | Show green checkmark, proof image may highlight broader area |
| "first_word_found" | Show amber warning, consider treating as unverified in strict mode |

5. Error Handling Could Be More Robust (AGENTS.md:504-521)

The error handling example is good but doesn't cover rate limiting (429 errors):

Recommendation: Add rate limiting case:

} catch (error: any) {
  const message = error?.message || "Unknown error";
  
  if (message.includes("rate limit") || error.status === 429) {
    // Rate limit exceeded - implement exponential backoff
    console.error("Rate limit exceeded. Retry after delay.");
  } else if (message.includes("Invalid or expired API key")) {
    // ...
  }
}

6. SSRF Warning Placement (AGENTS.md:279-283)

The SSRF warning appears in a quote block after the URL example. This critical security concern could be more prominent.

Recommendation: Move it above the code example or use a dedicated warning section:

### ⚠️ Security: URL Validation

When accepting user-provided URLs, **always validate** to prevent SSRF attacks:
- Block internal IPs: `localhost`, `127.0.0.1`, `192.168.*`, etc.
- Validate URL scheme is `http` or `https` only

```typescript
// Example validation...

### Low Priority

#### 7. **Multiple Documents Pattern** (AGENTS.md:303-318)
The comment about mapping is helpful but could be clearer:

```typescript
// fileDataParts[0].attachmentId → report.pdf
// fileDataParts[1].attachmentId → chart.png

Recommendation: Show how to actually use this mapping:

// Verify citation against specific document
const reportAttachmentId = fileDataParts[0].attachmentId;
await deepcitation.verifyAttachment(reportAttachmentId, citations);

8. Verification Status Flags Table (AGENTS.md:928-939)

Added isPending column which is great, but it's not used in any code examples.

Recommendation: Add a usage example in the "Check verification status" section (AGENTS.md:502).

9. Collapsible Sections Accessibility

Using <details> tags for "Other LLM Providers" (line 373) might hide important information from agents that scan linearly.

Recommendation: Consider using headers instead, or add a note at the top:

<details>
<summary>Other LLM Providers (Claude, Gemini, Vercel AI SDK)</summary>

**Note**: These examples follow the same pattern as OpenAI - replace the client and model.

---

Code Quality

Positive

• All TypeScript examples appear syntactically correct
• Error handling patterns follow best practices
• Input validation in API routes is comprehensive
• Comments are clear and actionable

Suggestions

• The Quick Start example (line 100) is excellent but lacks comments explaining why each step happens
• Consider adding // Step X:... comments for consistency with the narrative flow

---

Security Considerations

Well Handled ✅

• SSRF prevention guidance for user URLs
• unsafeFastUrlOutput warnings
• API key environment variable usage
• Input validation examples

Could Improve

• Add guidance on rate limiting user requests (not just API rate limits)
• Mention XSS risks when rendering visibleText in web apps (should be sanitized)
• Consider adding CSP recommendations for React components

---

Test Coverage

The PR doesn't include tests for the documentation itself. Consider:

• Validating all code examples compile (could use a test script)
• Checking all URLs in Appendix A are reachable
• Verifying <details> tags render correctly on GitHub

---

Performance Considerations

Good

• Processing time warnings (30s for URLs)
• Caching strategy with 24-hour TTL
• Mentions async verification

Could Add

• Guidance on when to cache deepTextPromptPortion (doesn't expire like attachmentId)
• Recommend debouncing verification requests in real-time applications
• Suggest batch verification for multiple citations

---

Overall Assessment

This is a substantial improvement over the previous version. The document is now:

• ✅ Agent-friendly with clear boundaries
• ✅ Progressive (quick start → advanced patterns)
• ✅ Security-conscious
• ✅ Well-organized with visual aids

Recommendation: Approve with minor revisions (address High Priority issues)

---

Suggested Next Steps

1. Fix streaming example clarity (High Priority fix: correct package exports to point to lib/ instead of src #1)
2. Add file hash implementation (High Priority Fix/npm build #3)
3. Enhance error handling with rate limit case (Medium Priority Feature/key span #5)
4. Move SSRF warning to more prominent position (Medium Priority Patch/client cleanup #6)
5. Run code examples through linter/compiler to verify
6. Consider adding a "Migration Guide" for users of the old AGENTS.md

---

Great work addressing all 33 feedback items! This will significantly improve agent implementations. 🎉

[claude]

Comprehensive PR Review for AGENTS.md Restructuring

OVERVIEW:
This is an excellent rewrite that significantly improves the agent experience. The restructuring addresses all 33 feedback items with thoughtful progressive disclosure, better organization, and critical safety guardrails.

STRENGTHS:

1. Progressive Disclosure - Agent Behavior Guidelines at top prevents hallucination
2. Safety Features - Appendix A and B prevent URL and feature fabrication
3. Technical Completeness - Added prepareUrl(), error handling, streaming guidance
4. Better Organization - Collapsible sections, user journey flow
5. Production Ready - Checklist, troubleshooting, caching strategies

CRITICAL ISSUES (Must Fix):

1. Array Bounds Check Missing (Line 712)

   • messages[messages.length - 1] will throw if messages is empty
   • Add validation: if (!messages || messages.length === 0) return error

2. SSRF Validation (Lines 277-282)

   • Warns about SSRF but no implementation provided
   • Need complete isUrlSafe() function blocking private IPs

3. Input Validation Missing

   • Upload route lacks file validation
   • Add: if (!file || !(file instanceof File)) return error

MEDIUM PRIORITY:

1. Type Safety (Line 707) - Add Array.isArray() check before casting
2. Buffer Overflow (Line 847) - Add MAX_RESPONSE_SIZE limit for streaming
3. Filename Sanitization (Line 638) - Use path.basename() to prevent traversal
4. Streaming Example Incomplete - Show Option B implementation

LOW PRIORITY:

1. Performance - Use reduce for verification summary
2. Documentation - Clarify cache TTL timing
3. Add debugging section for agents

TEST COVERAGE:
No test changes included. Consider adding:

• Syntax validation tests for code examples
• Link validation for Appendix A
• Integration test for Quick Start

OVERALL: Approve with Minor Improvements

Files Changed: 1 file, +649/-295 lines
Risk Level: Low (documentation only)
Breaking Changes: None

Great work on this comprehensive rewrite!