Skip to content

Commit 97352d1

Browse files
andy-stark-redisandy-stark-redis
authored
Merge pull request #2466 from redis/DOC-5991-data-type-choices
DOC-5991 page of advice on choosing data types
2 parents 61c5f4e + c21445c commit 97352d1

File tree

11 files changed

+1290
-5
lines changed

11 files changed

+1290
-5
lines changed

‎assets/css/index.css‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,10 @@ section.prose {
9494
padding: 1rem;
9595
}
9696

97+
.prose pre.decision-tree-source {
98+
display: none;
99+
}
100+
97101
.prose pre > code {
98102
@apply bg-none font-monogeist;
99103
}

‎build/render_hook_docs/AI_RENDER_HOOK_LESSONS.md‎

Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -409,23 +409,208 @@ if ((metaValue.startsWith('"') && metaValue.endsWith('"'))) {
409409

410410
---
411411

412+
## 21. Server-Side Metadata Extraction for AI Agents
413+
414+
**Pattern**: Extract metadata from structured content (YAML, JSON) in the Hugo render hook and embed it as JSON in the HTML output for AI agents that don't execute JavaScript.
415+
416+
**Implementation**:
417+
```html
418+
{{- /* Extract top-level fields only (no indentation) */ -}}
419+
{{- $lines := split .Inner "\n" -}}
420+
{{- $id := "" -}}
421+
{{- range $lines -}}
422+
{{- /* Check if line starts without whitespace (32=space, 9=tab) */ -}}
423+
{{- if and (gt (len .) 0) (ne (index . 0) 32) (ne (index . 0) 9) -}}
424+
{{- $trimmed := strings.TrimSpace . -}}
425+
{{- if strings.HasPrefix $trimmed "id:" -}}
426+
{{- $afterPrefix := strings.Replace $trimmed "id:" "" 1 -}}
427+
{{- $id = strings.TrimSpace $afterPrefix -}}
428+
{{- end -}}
429+
{{- end -}}
430+
{{- end -}}
431+
432+
{{- /* Embed as JSON for AI agents */ -}}
433+
{{- $metadata := dict "type" "decision-tree" "id" $id -}}
434+
{{ $jsonMetadata := $metadata | jsonify (dict "indent" " ") }}
435+
{{ printf "<script type=\"application/json\" data-redis-metadata=\"decision-tree\">\n%s\n</script>" $jsonMetadata | safeHTML }}
436+
```
437+
438+
**Key Considerations**:
439+
- **Indentation detection**: When parsing nested structures, only extract top-level fields by checking if the line starts with whitespace
440+
- **String manipulation**: Use `strings.Replace` instead of `strings.TrimPrefix` for more reliable extraction
441+
- **Character codes**: Use ASCII codes (32 for space, 9 for tab) to detect indentation reliably
442+
- **Metadata format**: Use simple JSON (not JSON-LD) for clarity and ease of parsing
443+
- **Data attributes**: Use `data-*` attributes to mark metadata elements for AI agents
444+
445+
**Why This Matters**:
446+
- AI agents typically don't execute JavaScript, so metadata must be in static HTML
447+
- Server-side extraction ensures metadata is available even if JavaScript fails
448+
- Structured metadata helps AI agents understand the purpose and scope of components
449+
- Separating metadata from content improves maintainability
450+
451+
**Lesson**: Always provide metadata in static HTML for AI agents. Use server-side extraction to ensure accuracy and avoid relying on JavaScript parsing.
452+
453+
---
454+
455+
## 22. Handling Nested Structures in Hugo Templates
456+
457+
**Pattern**: When extracting data from nested YAML/JSON structures, distinguish between top-level and nested fields using indentation detection.
458+
459+
**Problem**: If you extract all occurrences of a field (e.g., `id:`), you'll get nested occurrences too, leading to incorrect values.
460+
461+
**Solution**: Check indentation before processing:
462+
```html
463+
{{- if and (gt (len .) 0) (ne (index . 0) 32) (ne (index . 0) 9) -}}
464+
{{- /* Process only top-level lines */ -}}
465+
{{- end -}}
466+
```
467+
468+
**Why This Works**:
469+
- YAML indentation is significant and indicates nesting level
470+
- Top-level fields have no leading whitespace
471+
- Nested fields have leading spaces or tabs
472+
- Character code 32 = space, 9 = tab
473+
474+
**Lesson**: When parsing nested structures in Hugo templates, use indentation detection to distinguish between levels. This prevents extracting nested values when you only want top-level ones.
475+
476+
---
477+
478+
## 23. Progressive Enhancement with Metadata
479+
480+
**Pattern**: Combine progressive enhancement with metadata embedding to serve both humans and AI agents from a single source.
481+
482+
**Architecture**:
483+
1. **Server-side (Hugo)**:
484+
- Extract metadata from source content
485+
- Embed metadata as JSON in HTML
486+
- Preserve raw source in `<pre>` element
487+
488+
2. **Client-side (JavaScript)**:
489+
- Parse raw source for rendering
490+
- Use metadata for context/identification
491+
- Enhance with interactivity
492+
493+
3. **AI agents**:
494+
- Read static JSON metadata
495+
- Parse raw source from `<pre>` element
496+
- No JavaScript execution needed
497+
498+
**Benefits**:
499+
- Single source of truth (the YAML/JSON in the Markdown)
500+
- Metadata available to all consumers (humans, AI agents, JavaScript)
501+
- Graceful degradation if JavaScript fails
502+
- AI-friendly without extra work
503+
504+
**Lesson**: Design render hooks to serve multiple audiences simultaneously. Metadata should be available in static HTML, not just in JavaScript.
505+
506+
---
507+
508+
## 24. Text Wrapping and Box Sizing in SVG Diagrams
509+
510+
**Pattern**: When rendering text in SVG boxes, calculate dimensions based on character width and implement text wrapping to fit within maximum width.
511+
512+
**Implementation**:
513+
```javascript
514+
const charWidth = 8; // Space Mono at 14px
515+
const maxBoxWidth = 420;
516+
const maxCharsPerLine = Math.floor(maxBoxWidth / charWidth);
517+
518+
function wrapText(text, maxChars) {
519+
const words = text.split(' ');
520+
const lines = [];
521+
let currentLine = '';
522+
523+
for (const word of words) {
524+
if ((currentLine + ' ' + word).length > maxChars) {
525+
if (currentLine) lines.push(currentLine);
526+
currentLine = word;
527+
} else {
528+
currentLine = currentLine ? currentLine + ' ' + word : word;
529+
}
530+
}
531+
if (currentLine) lines.push(currentLine);
532+
return lines;
533+
}
534+
```
535+
536+
**Considerations**:
537+
- **Font metrics**: Different fonts have different character widths
538+
- **Padding**: Account for box padding when calculating available width
539+
- **Line height**: Multiply number of lines by line height for total box height
540+
- **Dynamic sizing**: Calculate SVG dimensions based on content, not fixed values
541+
542+
**Common Pitfall**: Hardcoding SVG width can cause content to be cut off. Instead:
543+
```javascript
544+
const svgWidth = leftMargin + (maxDepth + 1) * indentWidth + maxBoxWidth + 40;
545+
```
546+
547+
**Lesson**: Calculate SVG dimensions dynamically based on content. Account for all visual elements (padding, margins, decorations) when sizing boxes and containers.
548+
549+
---
550+
551+
## 25. Scope and Context Metadata for Component Discovery
552+
553+
**Pattern**: Add `scope` or `category` metadata to components to help AI agents understand their purpose and applicability.
554+
555+
**Implementation**:
556+
```yaml
557+
id: documents-tree
558+
scope: documents
559+
rootQuestion: root
560+
questions:
561+
# ...
562+
```
563+
564+
**Benefits**:
565+
- **Discoverability**: AI agents can filter components by scope
566+
- **Context awareness**: Agents know which problem domain each component addresses
567+
- **Prevents misapplication**: Agents won't use a "collections" tree to recommend document storage
568+
- **Relationship mapping**: Enables linking related components
569+
570+
**Use Cases**:
571+
- Filtering decision trees by data type category
572+
- Finding all components related to a specific feature
573+
- Organizing components hierarchically
574+
- Providing context in search results
575+
576+
**Lesson**: Add semantic metadata (scope, category, type) to components. This helps AI agents understand purpose and applicability, enabling better recommendations and filtering.
577+
578+
---
579+
412580
## Quick Checklist for Future Render Hooks
413581

582+
### Core Patterns
414583
- [ ] Preserve source content in a `<pre>` or similar element
415584
- [ ] Use page store pattern to avoid duplicate resource loading
416585
- [ ] Place static JavaScript in `static/js/`, not `assets/js/`
417586
- [ ] Avoid `innerHTML` with dynamic content; use safe DOM methods
418587
- [ ] Use `data-*` attributes to pass server data to JavaScript
588+
589+
### Testing & Accessibility
419590
- [ ] Test with multiple instances on the same page
420591
- [ ] Consider state persistence if needed
421592
- [ ] Use semantic HTML and proper accessibility attributes
422593
- [ ] Document the Markdown format clearly
423594
- [ ] Provide sensible defaults for optional parameters
595+
596+
### Hugo-Specific
424597
- [ ] Remember Hugo converts attribute names to lowercase
598+
- [ ] Use indentation detection when parsing nested structures
599+
- [ ] Extract top-level metadata in render hook (not JavaScript)
600+
- [ ] Use `strings.Replace` for reliable string manipulation in templates
601+
602+
### Advanced Features
425603
- [ ] Use SVG for complex visual structures
426604
- [ ] Track processed lines when parsing nested structures
427605
- [ ] Account for all visual elements in dimension calculations
428606
- [ ] Use type attributes for conditional features
429607
- [ ] Handle string unescaping during parsing, not rendering
430608
- [ ] Create comprehensive format documentation
431609

610+
### AI Agent Compatibility
611+
- [ ] Embed metadata as JSON in static HTML (not just JavaScript)
612+
- [ ] Add `scope` or `category` metadata for component discovery
613+
- [ ] Use `data-*` attributes to mark metadata elements
614+
- [ ] Ensure metadata is available without JavaScript execution
615+
- [ ] Preserve raw source content for AI parsing
616+

‎build/render_hook_docs/DECISION_TREE_FORMAT.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# Decision Tree Format Specification
2+
3+
## Overview
4+
5+
Decision trees are structured YAML documents that guide users through a series of questions to reach a recommendation. They are rendered as interactive SVG diagrams with boxes, connecting lines, and Yes/No branch labels.
6+
7+
## Basic Structure
8+
9+
```yaml
10+
```decision-tree {id="documents-tree"}
11+
id: documents-tree
12+
scope: documents
13+
rootQuestion: root
14+
questions:
15+
root:
16+
text: "Your question here?"
17+
whyAsk: "Explanation of why this question matters"
18+
answers:
19+
yes:
20+
value: "Yes"
21+
outcome:
22+
label: "Recommendation"
23+
id: outcomeId
24+
no:
25+
value: "No"
26+
nextQuestion: nextQuestionId
27+
nextQuestionId:
28+
text: "Follow-up question?"
29+
whyAsk: "Why this matters"
30+
answers:
31+
yes:
32+
value: "Yes"
33+
outcome:
34+
label: "Recommendation"
35+
id: outcomeId
36+
no:
37+
value: "No"
38+
outcome:
39+
label: "Alternative recommendation"
40+
id: altOutcomeId
41+
```
42+
```
43+
44+
## Fields
45+
46+
### Top-level
47+
48+
- **`id`** (required): Unique identifier for this decision tree (e.g., `documents-tree`, `collections-tree`). Used for discovery and referencing by AI agents.
49+
- **`scope`** (required): Category or domain this tree applies to (e.g., `documents`, `collections`, `sequences`). Helps AI agents understand the tree's purpose and applicability.
50+
- **`rootQuestion`** (required): The ID of the starting question
51+
- **`questions`** (required): Object containing all questions, keyed by ID
52+
53+
### Question Object
54+
55+
- **`text`** (required): The question text. Can span multiple lines using YAML's `|` literal block syntax
56+
- **`whyAsk`** (required): Explanation of why this question matters. Helps AI agents understand the decision logic
57+
- **`answers`** (required): Object with `yes` and `no` keys
58+
59+
### Answer Object
60+
61+
Each answer (`yes` or `no`) contains:
62+
63+
- **`value`** (required): Display text ("Yes" or "No")
64+
- **`outcome`** (optional): Terminal recommendation
65+
- `label`: Text to display (e.g., "Use JSON")
66+
- `id`: Unique identifier for this outcome
67+
- **`nextQuestion`** (optional): ID of the next question to ask
68+
69+
**Note**: Each answer must have either `outcome` or `nextQuestion`, not both.
70+
71+
### Outcome Object
72+
73+
- **`label`** (required): The recommendation text
74+
- **`id`** (required): Unique identifier (e.g., `jsonOutcome`, `hashOutcome`)
75+
76+
## Multi-line Text
77+
78+
Use YAML's literal block syntax (`|`) for multi-line text:
79+
80+
```yaml
81+
text: |
82+
Do you need nested data structures
83+
(fields and arrays) or geospatial
84+
index/query with Redis query engine?
85+
whyAsk: |
86+
JSON is the only document type that supports
87+
deeply nested structures and integrates with
88+
the query engine for those structures
89+
```
90+
91+
## Code Block Attributes
92+
93+
The code block fence supports the following attributes:
94+
95+
- **`id`** (optional): Unique identifier for the tree. Should match the `id` field in the YAML. Used by Hugo to pass metadata to the render hook.
96+
97+
Example:
98+
```markdown
99+
```decision-tree {id="documents-tree"}
100+
id: documents-tree
101+
scope: documents
102+
# ...
103+
```
104+
```
105+
106+
## Best Practices
107+
108+
1. **Use descriptive IDs**: `root`, `hashQuestion`, `jsonOutcome` are clearer than `q1`, `q2`
109+
2. **Keep questions concise**: Aim for 1-2 lines when possible
110+
3. **Explain the rationale**: The `whyAsk` field helps users and AI agents understand the decision logic
111+
4. **Reuse outcomes**: Multiple paths can lead to the same outcome (same `id`)
112+
5. **Consistent naming**: Use camelCase for IDs, end question IDs with "Question"
113+
6. **Match fence and YAML IDs**: The `id` in the code block fence should match the `id` field in the YAML for consistency
114+
7. **Use meaningful scopes**: Choose scope values that clearly indicate the tree's domain (e.g., `documents`, `collections`, `sequences`)
115+
116+
## Example: Redis Data Type Selection
117+
118+
See `content/develop/data-types/compare-data-types.md` for a complete example.
119+
120+
## Rendering
121+
122+
Decision trees are rendered as:
123+
- **SVG diagram** for humans (with boxes, lines, and labels)
124+
- **Normalized JSON** embedded for AI agents (accessible via `.html.md` URLs)
125+
- **Raw YAML** preserved in `<pre>` element for accessibility
126+
127+
## AI Agent Compatibility
128+
129+
The format is designed to be easily parseable by AI agents:
130+
131+
### Metadata Embedding
132+
- **Server-side JSON**: Each tree is embedded with metadata as `<script type="application/json" data-redis-metadata="decision-tree">` containing `id`, `scope`, and `type` fields
133+
- **No JavaScript required**: Metadata is available in static HTML, accessible to AI agents that don't execute JavaScript
134+
- **Discoverable**: The `id` and `scope` fields enable AI agents to identify and filter trees
135+
136+
### Content Accessibility
137+
- **Raw YAML preserved**: The original YAML source is preserved in a `<pre class="decision-tree-source">` element for AI parsing
138+
- **Clear hierarchical structure**: Explicit question IDs and outcome IDs
139+
- **Reasoning context**: `whyAsk` field explains the decision logic
140+
- **Normalized JSON**: Available for deterministic processing
141+
142+
### Example Metadata
143+
```json
144+
{
145+
"type": "decision-tree",
146+
"id": "documents-tree",
147+
"scope": "documents"
148+
}
149+
```
150+
151+
This metadata helps AI agents:
152+
- Identify the tree's purpose and domain
153+
- Filter trees by scope
154+
- Reference specific trees in recommendations
155+
- Understand the tree's applicability to different problems
156+

0 commit comments

Comments
 (0)