From d7911080680beae942753a5dc89db88785c5d853 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Fri, 21 Nov 2025 10:03:08 +0000 Subject: [PATCH 01/16] DOC-5991 initial data type summary --- .../develop/data-types/compare-data-types.md | 145 ++++++++++++++++++ 1 file changed, 145 insertions(+) create mode 100644 content/develop/data-types/compare-data-types.md diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md new file mode 100644 index 0000000000..a533b8d8d5 --- /dev/null +++ b/content/develop/data-types/compare-data-types.md @@ -0,0 +1,145 @@ +--- +categories: +- docs +- develop +- stack +- oss +- rs +- rc +- oss +- kubernetes +- clients +description: Choose the best Redis data type for your task. +linkTitle: Compare data types +title: Compare data types +weight: 1 +--- + +Redis provides a wide range of data types to store your data. +The following are highly specialized for precise purposes: + +- [Geospatial]({{< relref "/develop/data-types/geospatial" >}}): + store strings with associated coordinates for geospatial queries. +- [Vector sets]({{< relref "/develop/data-types/vector-sets" >}}): + store strings with associated vector data (and optional metadata) + for vector similarity queries. +- [Probabilistic data types]({{< relref "/develop/data-types/probabilistic" >}}): + keep approximate counts and other statistics for large datasets. +- [Time series]({{< relref "/develop/data-types/timeseries" >}}): + store real-valued data points along with the time they were collected. + +The remaining data types are more general-purpose: + +- [Strings]({{< relref "/develop/data-types/strings" >}}): + store text or binary data. +- [Hashes]({{< relref "/develop/data-types/hashes" >}}): + store key-value pairs within a single key. +- [JSON]({{< relref "/develop/data-types/json" >}}): + store structured, hierarchical arrays and key-value objects that match + the popular [JSON](https://www.json.org/json-en.html) text file format. +- [Lists]({{< relref "/develop/data-types/lists" >}}): + store a simple sequence of strings. +- [Sets]({{< relref "/develop/data-types/sets" >}}): + store a collection of unique strings. +- [Sorted sets]({{< relref "/develop/data-types/sorted-sets" >}}): + store a collection of unique strings with associated scores. +- [Streams]({{< relref "/develop/data-types/streams" >}}): + store a sequence of entries, each with a set of field-value pairs. + +The general-purpose data types have some overlap among their features +and indeed, you could probably emulate any of them using just strings +and a little creativity. However, each data type provides different +tradeoffs in terms of performance, memory usage, and functionality. +This guide helps you choose the best data type for your task. + +## Data type features + +The sections below summarize the features of each data type. + +### Strings + +- **Structure**: unstructured text/binary data or simple counters, + bit sets, or integer collections. +- **Operations**: get, set, append, increment, decrement, bitwise operations. +- **Suitable for**: Unstructured documents, counters, flags, bitmaps. + +Strings are mainly useful for storing text or binary data chunks +whose internal structure will be managed by your own application. +However, they also support operations to access ranges of bits +in the string to use as bit sets, integers, or floating-point numbers. + + +### Hashes + +- **Structure**: collection of key-value pairs. +- **Operations**: get, set, delete, increment, decrement. +- **Suitable for**: Simple objects with a small number of fields. + +Hashes are mainly useful for storing objects with a small number of fields +that are not nested or intricately structured. However, there is +no real limit to the number of fields you can store in a hash, so you +can use hashes in many different ways inside your application. +The field values are strings, but hashes provide commands to treat +them as integers or floating-point numbers and perform simple arithmetic +operations on them. You can set expirations on individual hash fields +and you can also index and query hash documents using the Redis +[query engine]({{< relref "/develop/ai/search-and-query" >}}). + +### JSON + +- **Structure**: hierarchical arrays and key-value objects that match + the popular [JSON](https://www.json.org/json-en.html) text file format. +- **Operations**: get, set, update, delete, query. +- **Suitable for**: Complex, nested objects with many fields. + +JSON provides rich data modeling capabilities with nested fields and arrays. +You can use a simple path syntax to access any subset of the data within +a JSON document. JSON also has more powerful and flexible +[query engine]({{< relref "/develop/ai/search-and-query" >}}) +features compared to hashes. + +### Lists + +- **Structure**: simple sequence of strings. +- **Operations**: push, pop, get, set, trim. +- **Suitable for**: Queues, stacks, logs, and other linear data structures. + +Lists store sequences of string values. They are optimized for +adding and removing small numbers of elements at the head or tail, +and so they are very efficient for implementing queues, stacks, +and deques. + +### Sets + +- **Structure**: collection of unique strings. +- **Operations**: add, remove, test membership, intersect, union, difference. +- **Suitable for**: Unique items with no associated data. + +Sets store collections of unique strings. They provide efficient +operations for testing membership, adding and removing elements. +They also support set operations like intersection, union, and difference. + +### Sorted sets + +- **Structure**: collection of unique strings with associated scores. +- **Operations**: add, remove, test membership, range by score or rank. +- **Suitable for**: Unique items with a score, or ordered collections. + +Sorted sets store collections of unique strings with associated scores. +They are optimized for efficient range queries based on the score, +and so they are useful for implementing priority queues and other ordered +collections. + +### Streams + +- **Structure**: sequence of entries, each with a set of field-value pairs. +- **Operations**: add, read, trim. +- **Suitable for**: Log data, time series, and other append-only structures. + +Streams store sequences of entries, each with a set of field-value pairs. +They are optimized for appending new entries and reading them in order, +and so they are useful for implementing log data, time series, and other +append-only data structures. They also have built-in support for consumer groups +to manage multiple readers and ensure at-least-once delivery. + + From 104d2675042793b2f3116a3fd01a7ca521459a6c Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Mon, 24 Nov 2025 13:50:43 +0000 Subject: [PATCH 02/16] DOC-5991 start comparison section --- content/develop/data-types/compare-data-types.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index a533b8d8d5..d340db0752 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -142,4 +142,12 @@ and so they are useful for implementing log data, time series, and other append-only data structures. They also have built-in support for consumer groups to manage multiple readers and ensure at-least-once delivery. +## Choose a data type +The sections below explore the pros and cons of each data type for +particular tasks. + +### Documents + +You would normally store document data using the string, hash, or JSON +types. From e7cc39e48cc3a3f1c2042cf2b7684dd58dbd6f1d Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Mon, 24 Nov 2025 15:59:31 +0000 Subject: [PATCH 03/16] DOC-5991 started document choice section --- content/develop/data-types/compare-data-types.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index d340db0752..3fa858d8f8 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -151,3 +151,9 @@ particular tasks. You would normally store document data using the string, hash, or JSON types. + +- Strings are suitable for unstructured or opaquely structured text and +binary documents that change infrequently or will be processed as a whole +by your application. They provide only limited options for accessing +individual parts bits and bit ranges within the string. + From a761a1ce4ac484ddd8f6b032fa475829ed5553ff Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Wed, 26 Nov 2025 09:30:57 +0000 Subject: [PATCH 04/16] DOC-5991 experimental decision tree start --- .../develop/data-types/compare-data-types.md | 62 +++++++++++++++++-- 1 file changed, 56 insertions(+), 6 deletions(-) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index 3fa858d8f8..960d139395 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -150,10 +150,60 @@ particular tasks. ### Documents You would normally store document data using the string, hash, or JSON -types. - -- Strings are suitable for unstructured or opaquely structured text and -binary documents that change infrequently or will be processed as a whole -by your application. They provide only limited options for accessing -individual parts bits and bit ranges within the string. +types. JSON has the highest requirements for memory and processing, followed +by hashes, and then strings. Consider the questions below as a guide to +choosing the best data type for your task. + +1. Do you need nested fields or arrays? + + If so, use **JSON** since it is the only type that supports these features. + +1. Do you need to index and query using the Redis query engine? + + If so, use **hashes** since they support indexing and querying with lower memory overhead than JSON. + +1. Do you need any of the following features? + - Frequent access to individual fields within the document + - Expiration times on individual pieces of data within the document + + If so, use **hashes** for efficient field-level access and expiration. + +1. Is your data unstructured or opaquely structured? + + If so, use **strings** for simple, unstructured data. Otherwise, use **hashes** for structured data without querying needs. + +## Decision tree + +Use this hierarchy to systematically choose the best data type: + +```hierarchy {type="decision"} +"Choose a data type for documents": + _meta: + description: "Decision tree for selecting between strings, hashes, and JSON" + "Do you need nested fields or arrays?": + _meta: + description: "Only JSON supports nested structures and arrays" + "Yes": "Use JSON" + "No": + "Do you need to index and query using the Redis query engine?": + _meta: + description: "Hashes and JSON both support querying, but hashes are more efficient" + "Yes": "Use hashes" + "No": + "Do you need frequent access to individual fields?": + _meta: + description: "Hashes provide efficient field-level access" + "Yes": + "Do you need expiration times on individual fields?": + _meta: + description: "Only hashes support field-level expiration" + "Yes": "Use hashes" + "No": "Use hashes or strings (hashes recommended for structured data)" + "No": + "Is your data unstructured or opaquely structured?": + _meta: + description: "Strings are ideal for unstructured data" + "Yes": "Use strings" + "No": "Use hashes" +``` From 24ba065e38c46ef4ed2932ce921e4ded61666c92 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Wed, 26 Nov 2025 16:04:25 +0000 Subject: [PATCH 05/16] DOC-5991 rounded out choice lists --- .../develop/data-types/compare-data-types.md | 114 +++++++++++------- 1 file changed, 68 insertions(+), 46 deletions(-) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index 960d139395..7db5c52654 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -150,60 +150,82 @@ particular tasks. ### Documents You would normally store document data using the string, hash, or JSON -types. JSON has the highest requirements for memory and processing, followed -by hashes, and then strings. Consider the questions below as a guide to +types. JSON generally has the highest requirements for memory and processing, +followed by hashes, and then strings. Use the considerations below as a guide to choosing the best data type for your task. -1. Do you need nested fields or arrays? +1. Do you need nested data structures (fields and arrays) or geospatial + index/query with Redis query engine? If so, use **JSON** since it is the only type that supports these features. -1. Do you need to index and query using the Redis query engine? +2. Do you need to index/query using Redis query engine but can live without + nested data structures and geospatial data? - If so, use **hashes** since they support indexing and querying with lower memory overhead than JSON. + If so, use **hashes** since they support indexing and querying with lower memory overhead and faster field access than JSON. -1. Do you need any of the following features? - - Frequent access to individual fields within the document - - Expiration times on individual pieces of data within the document +3. Do you need to set expiration times on individual pieces of data within + the document? If so, use **hashes** for efficient field-level access and expiration. -1. Is your data unstructured or opaquely structured? - - If so, use **strings** for simple, unstructured data. Otherwise, use **hashes** for structured data without querying needs. - -## Decision tree - -Use this hierarchy to systematically choose the best data type: - -```hierarchy {type="decision"} -"Choose a data type for documents": - _meta: - description: "Decision tree for selecting between strings, hashes, and JSON" - "Do you need nested fields or arrays?": - _meta: - description: "Only JSON supports nested structures and arrays" - "Yes": "Use JSON" - "No": - "Do you need to index and query using the Redis query engine?": - _meta: - description: "Hashes and JSON both support querying, but hashes are more efficient" - "Yes": "Use hashes" - "No": - "Do you need frequent access to individual fields?": - _meta: - description: "Hashes provide efficient field-level access" - "Yes": - "Do you need expiration times on individual fields?": - _meta: - description: "Only hashes support field-level expiration" - "Yes": "Use hashes" - "No": "Use hashes or strings (hashes recommended for structured data)" - "No": - "Is your data unstructured or opaquely structured?": - _meta: - description: "Strings are ideal for unstructured data" - "Yes": "Use strings" - "No": "Use hashes" -``` +4. Do you need frequent access to individual data fields within the document? + If the data fields are simple integers or individual bits and you can + easily refer to them by an integer index, use **strings** for efficient + access and minimum memory overhead. Otherwise, use **hashes** for + named fields and support for string and binary field values. + +5. For other simple documents with arbitrary internal structure, use **strings** + for simplicity and minimum memory overhead. + +### Collections + +You would normally store collection data using the set or sorted set +types and for very simple collections, you can even use strings. They all allow +basic membership tests, but have different additional features and tradeoffs. +Sorted sets have the highest memory overhead and processing requirements, followed +by sets, and then strings. +Use the considerations below as a guide to choosing the best data type for your task. +Note that if you need to store extra information for the keys in a set +or sorted set, you can do so with an auxiliary hash or JSON object that has field +names matching the keys in the collection. + +1. Do you need to store and retrieve the keys in an arbitrary order or in + lexicographical order? + + If so, use **sorted sets** since they are the only collection type that supports ordered iteration. + +2. Are the keys always simple integer indices in a known range? + + If so, use the bitmap features of **strings** for minimum memory overhead and efficient random access. String bitmaps also support bitwise operations + that are equivalent to set operations such as union, intersection, and difference. + +3. For arbitrary string or binary keys, use **sets** for efficient membership tests and + set operations. If you *only* need membership tests on the keys, but you + need to store extra information for each key, consider using **hashes** with + the keys as field names. + +## Sequences + +You would normally store sequences of string or binary data using sorted sets, +lists or streams. They each have advantages and disadvantages for particular purposes. +Use the considerations below as a guide to choosing the best data type for your task. + +1. Do you frequently need to do any of the following? + + - Maintain an arbitrary priority order or lexicographical order of the elements + - Access individual elements or ranges of elements by index + - Perform basic set operations on the elements + + If so, use **sorted sets** since they are the only sequence type that supports these + operations directly. + +2. Do you need to store and retrieve elements primarily in timestamp order or + manage multiple consumers reading from the sequence? + + If so, use **streams** since they are the only sequence type that supports these + features natively. + +3. For simple sequences of string or binary data, use **lists** for efficient + push/pop operations at the head or tail. From cada51a79069da02f02f05984c85e6731ce036db Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 11:14:25 +0000 Subject: [PATCH 06/16] DOC-5991 solid decision tree format (finally) --- .../develop/data-types/compare-data-types.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index 7db5c52654..9ce0743678 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -179,6 +179,88 @@ choosing the best data type for your task. 5. For other simple documents with arbitrary internal structure, use **strings** for simplicity and minimum memory overhead. +```decision-tree +rootQuestion: root +questions: + root: + text: | + Do you need nested data structures (fields and arrays) or geospatial + index/query with Redis query engine? + whyAsk: | + JSON is the only document type that supports deeply nested structures and integrates with the query engine for those structures + answers: + yes: + value: "Yes" + outcome: + label: "Use JSON" + id: jsonOutcome + no: + value: "No" + nextQuestion: hashQuestion + hashQuestion: + text: | + Do you need to index/query using Redis query engine but can live + without nested data structures and geospatial indexing? + whyAsk: | + Hashes support indexing and querying with lower memory overhead and faster field access than JSON + answers: + yes: + value: "Yes" + outcome: + label: "Use hashes" + id: hashOutcome + no: + value: "No" + nextQuestion: expirationQuestion + expirationQuestion: + text: | + Do you need to set expiration times on individual pieces of data + within the document? + whyAsk: "Only hashes support efficient field-level access and expiration" + answers: + yes: + value: "Yes" + outcome: + label: "Use hashes" + id: hashOutcome + no: + value: "No" + nextQuestion: fieldAccessQuestion + fieldAccessQuestion: + text: | + Do you need frequent access to individual data fields within the + document, but the fields are simple integers or bits that you can easily + refer to by an integer index? + whyAsk: | + Strings and hashes support efficient field access, but strings are more compact and efficient if you only need bit fields with integer indices + answers: + yes: + value: "Yes" + outcome: + label: "Use strings" + id: stringOutcome + no: + value: "No" + nextQuestion: stringQuestion + stringQuestion: + text: | + Do you need frequent access to individual data fields within the + document that have string or binary data values? + whyAsk: | + Hashes support general field access, but strings are more compact and efficient if you don't need it + answers: + yes: + value: "Yes" + outcome: + label: "Use hashes" + id: hashOutcome + no: + value: "No" + outcome: + label: "Use strings" + id: stringOutcome +``` + ### Collections You would normally store collection data using the set or sorted set From ef9cf2fe77a741ee0131bda0bdea4acff5ee22d4 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 13:15:45 +0000 Subject: [PATCH 07/16] DOC-5991 intial decision tree diagram --- .../render-codeblock-decision-tree.html | 4 + layouts/_default/baseof.html | 5 + static/js/decision-tree.js | 338 ++++++++++++++++++ 3 files changed, 347 insertions(+) create mode 100644 layouts/_default/_markup/render-codeblock-decision-tree.html create mode 100644 static/js/decision-tree.js diff --git a/layouts/_default/_markup/render-codeblock-decision-tree.html b/layouts/_default/_markup/render-codeblock-decision-tree.html new file mode 100644 index 0000000000..6027aa1614 --- /dev/null +++ b/layouts/_default/_markup/render-codeblock-decision-tree.html @@ -0,0 +1,4 @@ +{{- /* Render hook for decision tree code blocks */ -}} +
{{ .Inner | htmlEscape | safeHTML }}
+{{ .Page.Store.Set "hasDecisionTree" true }} + diff --git a/layouts/_default/baseof.html b/layouts/_default/baseof.html index 32ff961bf3..9c8bbe33eb 100644 --- a/layouts/_default/baseof.html +++ b/layouts/_default/baseof.html @@ -108,5 +108,10 @@ {{ if .Page.Store.Get "hasHierarchy" }} {{ end }} + + + {{ if .Page.Store.Get "hasDecisionTree" }} + + {{ end }} diff --git a/static/js/decision-tree.js b/static/js/decision-tree.js new file mode 100644 index 0000000000..c566e216a5 --- /dev/null +++ b/static/js/decision-tree.js @@ -0,0 +1,338 @@ +// Decision tree rendering engine +// Parses YAML decision trees and renders them as tree structure diagrams + +(function() { + 'use strict'; + + // Parse YAML from the pre element + function parseDecisionTreeYAML(yamlText) { + const lines = yamlText.split('\n'); + const root = {}; + const stack = [{ node: root, indent: -1, key: null }]; + + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + if (!line.trim() || line.trim().startsWith('#')) continue; + + const indent = line.search(/\S/); + const content = line.trim(); + + while (stack.length > 1 && stack[stack.length - 1].indent >= indent) { + stack.pop(); + } + + const parent = stack[stack.length - 1].node; + + if (content.includes(':')) { + const colonIndex = content.indexOf(':'); + const key = content.substring(0, colonIndex).trim(); + let value = content.substring(colonIndex + 1).trim(); + + if ((value.startsWith('"') && value.endsWith('"')) || + (value.startsWith("'") && value.endsWith("'"))) { + value = value.slice(1, -1).replace(/\\"/g, '"').replace(/\\\\/g, '\\'); + } + + if (value === '|') { + const multiLineValue = []; + i++; + while (i < lines.length) { + const nextLine = lines[i]; + if (nextLine.trim() === '') { + i++; + continue; + } + const nextIndent = nextLine.search(/\S/); + if (nextIndent <= indent) break; + multiLineValue.push(nextLine.trim()); + i++; + } + i--; + value = multiLineValue.join(' '); + } + + if (value === '') { + const newObj = {}; + parent[key] = newObj; + stack.push({ node: newObj, indent: indent, key: key }); + } else { + parent[key] = value; + } + } + } + + return root; + } + + // Flatten decision tree into a list for tree rendering + function flattenDecisionTree(questions, rootId) { + const items = []; + const visited = new Set(); + + function traverse(nodeId, depth) { + if (visited.has(nodeId)) return; + visited.add(nodeId); + + const question = questions[nodeId]; + if (!question) return; + + items.push({ + id: nodeId, + depth: depth, + type: 'question', + text: question.text || '', + whyAsk: question.whyAsk || '' + }); + + if (question.answers) { + // Process yes answer + if (question.answers.yes) { + if (question.answers.yes.nextQuestion) { + traverse(question.answers.yes.nextQuestion, depth + 1); + } else if (question.answers.yes.outcome) { + items.push({ + id: question.answers.yes.outcome.id, + depth: depth + 1, + type: 'outcome', + text: question.answers.yes.outcome.label || '', + answer: 'Yes' + }); + } + } + + // Process no answer + if (question.answers.no) { + if (question.answers.no.nextQuestion) { + traverse(question.answers.no.nextQuestion, depth + 1); + } else if (question.answers.no.outcome) { + items.push({ + id: question.answers.no.outcome.id, + depth: depth + 1, + type: 'outcome', + text: question.answers.no.outcome.label || '', + answer: 'No' + }); + } + } + } + } + + traverse(rootId, 0); + return items; + } + + // Wrap text to fit within a maximum width + function wrapText(text, maxChars) { + const words = text.split(' '); + const lines = []; + let currentLine = ''; + + words.forEach(word => { + if ((currentLine + ' ' + word).trim().length <= maxChars) { + currentLine = currentLine ? currentLine + ' ' + word : word; + } else { + if (currentLine) lines.push(currentLine); + currentLine = word; + } + }); + if (currentLine) lines.push(currentLine); + return lines; + } + + // Render decision tree as SVG with tree structure and boxes + function renderDecisionTree(container, treeData) { + const rootId = treeData.rootQuestion; + const questions = treeData.questions; + + if (!rootId || !questions[rootId]) { + container.textContent = 'Error: Invalid decision tree structure'; + return; + } + + const items = flattenDecisionTree(questions, rootId); + + const lineHeight = 24; + const charWidth = 8; + const leftMargin = 20; + const topMargin = 10; + const indentWidth = 24; + const boxPadding = 8; + const maxBoxWidth = 280; // Max width for text in box + const maxCharsPerLine = Math.floor(maxBoxWidth / charWidth); + + // Calculate box dimensions for each item + items.forEach(item => { + const lines = wrapText(item.text, maxCharsPerLine); + item.lines = lines; + item.boxHeight = lines.length * lineHeight + boxPadding * 2; + item.boxWidth = Math.min(maxBoxWidth, Math.max(...lines.map(l => l.length)) * charWidth + boxPadding * 2); + }); + + // Calculate SVG dimensions + let maxDepth = 0; + items.forEach(item => { + maxDepth = Math.max(maxDepth, item.depth); + }); + + const svgWidth = leftMargin + (maxDepth + 1) * indentWidth + 320; + const svgHeight = topMargin + items.reduce((sum, item) => sum + item.boxHeight + 20, 0) + 10; + + const svg = document.createElementNS('http://www.w3.org/2000/svg', 'svg'); + svg.setAttribute('width', svgWidth); + svg.setAttribute('height', svgHeight); + svg.setAttribute('class', 'decision-tree-diagram'); + svg.style.marginTop = '1em'; + svg.style.marginBottom = '1em'; + svg.style.fontFamily = '"Space Mono", monospace'; + svg.style.fontSize = '13px'; + + let currentY = topMargin; + + items.forEach((item, index) => { + const x = leftMargin + item.depth * indentWidth; + const y = currentY + item.boxHeight / 2; + + if (item.depth > 0) { + drawTreeLines(svg, item, items, index, leftMargin, topMargin, lineHeight, indentWidth, currentY, item.boxHeight); + } + + // Draw box with different styling for outcomes vs questions + const rect = document.createElementNS('http://www.w3.org/2000/svg', 'rect'); + rect.setAttribute('x', x + 10); + rect.setAttribute('y', currentY); + rect.setAttribute('width', item.boxWidth); + rect.setAttribute('height', item.boxHeight); + + if (item.type === 'outcome') { + // Outcomes: lighter background, dashed border + rect.setAttribute('fill', '#f9f9f9'); + rect.setAttribute('stroke', '#ccc'); + rect.setAttribute('stroke-width', '1'); + rect.setAttribute('stroke-dasharray', '3,3'); + } else { + // Questions: standard styling + rect.setAttribute('fill', '#f5f5f5'); + rect.setAttribute('stroke', '#999'); + rect.setAttribute('stroke-width', '1'); + } + rect.setAttribute('rx', '4'); + svg.appendChild(rect); + + // Draw text lines + item.lines.forEach((line, lineIndex) => { + const text = document.createElementNS('http://www.w3.org/2000/svg', 'text'); + text.setAttribute('x', x + 10 + boxPadding); + text.setAttribute('y', currentY + boxPadding + (lineIndex + 1) * lineHeight - 6); + text.setAttribute('font-family', '"Space Mono", monospace'); + text.setAttribute('font-size', '13'); + text.setAttribute('fill', item.type === 'outcome' ? '#666' : '#333'); + text.textContent = line; + svg.appendChild(text); + }); + + currentY += item.boxHeight + 20; + }); + + const jsonScript = document.createElement('script'); + jsonScript.type = 'application/json'; + jsonScript.className = 'decision-tree-data'; + jsonScript.textContent = JSON.stringify(treeData, null, 2); + + container.appendChild(svg); + container.parentNode.insertBefore(jsonScript, container.nextSibling); + } + + function drawTreeLines(svg, item, items, itemIndex, leftMargin, topMargin, lineHeight, indentWidth, currentY, boxHeight) { + const y = currentY + boxHeight / 2; + const x = leftMargin + item.depth * indentWidth; + const parentX = x - indentWidth; + const connectorX = parentX + 10; // Left edge of parent box (vertical line position) + + // Find parent item and its Y position, and determine the answer (Yes/No) + let parentY = null; + let parentBoxHeight = 0; + let parentCurrentY = null; + let answerLabel = ''; + + for (let i = itemIndex - 1; i >= 0; i--) { + if (items[i].depth === item.depth - 1) { + // Calculate parent's Y position + let calcY = topMargin; + for (let j = 0; j < i; j++) { + calcY += items[j].boxHeight + 20; + } + parentCurrentY = calcY; + parentY = calcY + items[i].boxHeight / 2; + parentBoxHeight = items[i].boxHeight; + + // Determine if this is a Yes or No answer by checking the parent's answers + // Count how many siblings come before this item + let siblingCount = 0; + for (let k = i + 1; k < itemIndex; k++) { + if (items[k].depth === item.depth) { + siblingCount++; + } + } + + // If this is the first child of the parent, it's the "yes" path, otherwise "no" + answerLabel = siblingCount === 0 ? 'Yes' : 'No'; + break; + } + } + + if (parentY !== null) { + // Vertical line starts from bottom of parent box + const verticalStartY = parentCurrentY + parentBoxHeight; + + // Vertical line from parent box bottom + const line = document.createElementNS('http://www.w3.org/2000/svg', 'line'); + line.setAttribute('x1', connectorX); + line.setAttribute('y1', verticalStartY); + line.setAttribute('x2', connectorX); + line.setAttribute('y2', y); + line.setAttribute('stroke', '#999'); + line.setAttribute('stroke-width', '1'); + svg.appendChild(line); + + // Horizontal line to child + const boxX = x + 10; + const hline = document.createElementNS('http://www.w3.org/2000/svg', 'line'); + hline.setAttribute('x1', connectorX); + hline.setAttribute('y1', y); + hline.setAttribute('x2', boxX); + hline.setAttribute('y2', y); + hline.setAttribute('stroke', '#999'); + hline.setAttribute('stroke-width', '1'); + svg.appendChild(hline); + + // Add answer label on the horizontal line, positioned below it to avoid boxes + const labelX = connectorX + (boxX - connectorX) / 2; + const labelY = y + 10; + + const label = document.createElementNS('http://www.w3.org/2000/svg', 'text'); + label.setAttribute('x', labelX); + label.setAttribute('y', labelY); + label.setAttribute('font-family', '"Space Mono", monospace'); + label.setAttribute('font-size', '11'); + label.setAttribute('fill', '#666'); + label.setAttribute('text-anchor', 'middle'); + label.textContent = answerLabel; + svg.appendChild(label); + } + } + + document.addEventListener('DOMContentLoaded', function() { + const sources = document.querySelectorAll('pre.decision-tree-source'); + + sources.forEach(pre => { + const yamlText = pre.textContent; + const treeData = parseDecisionTreeYAML(yamlText); + + const container = document.createElement('div'); + container.className = 'decision-tree-container'; + pre.parentNode.insertBefore(container, pre.nextSibling); + + renderDecisionTree(container, treeData); + }); + }); +})(); From efbab33e494e29e357fd3eed1ea9f66fc2036f35 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 13:36:27 +0000 Subject: [PATCH 08/16] DOC-5991 better tree diagram JS --- static/js/decision-tree.js | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/static/js/decision-tree.js b/static/js/decision-tree.js index c566e216a5..2f7fb55915 100644 --- a/static/js/decision-tree.js +++ b/static/js/decision-tree.js @@ -155,7 +155,7 @@ const charWidth = 8; const leftMargin = 20; const topMargin = 10; - const indentWidth = 24; + const indentWidth = 40; // Increased from 24 for wider indent const boxPadding = 8; const maxBoxWidth = 280; // Max width for text in box const maxCharsPerLine = Math.floor(maxBoxWidth / charWidth); @@ -294,19 +294,20 @@ line.setAttribute('stroke-width', '1'); svg.appendChild(line); - // Horizontal line to child + // Horizontal line to child (extended to match wider indent) const boxX = x + 10; + const hlineExtension = 15; // Extra space for label const hline = document.createElementNS('http://www.w3.org/2000/svg', 'line'); hline.setAttribute('x1', connectorX); hline.setAttribute('y1', y); - hline.setAttribute('x2', boxX); + hline.setAttribute('x2', boxX + hlineExtension); hline.setAttribute('y2', y); hline.setAttribute('stroke', '#999'); hline.setAttribute('stroke-width', '1'); svg.appendChild(hline); // Add answer label on the horizontal line, positioned below it to avoid boxes - const labelX = connectorX + (boxX - connectorX) / 2; + const labelX = connectorX + (boxX + hlineExtension - connectorX) / 2; const labelY = y + 10; const label = document.createElementNS('http://www.w3.org/2000/svg', 'text'); From ebb3e78090633fc91651dab6b008f5230dbdfe38 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 13:50:19 +0000 Subject: [PATCH 09/16] DOC-5991 diagrams basically OK --- static/js/decision-tree.js | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/static/js/decision-tree.js b/static/js/decision-tree.js index 2f7fb55915..d0574cb295 100644 --- a/static/js/decision-tree.js +++ b/static/js/decision-tree.js @@ -156,8 +156,8 @@ const leftMargin = 20; const topMargin = 10; const indentWidth = 40; // Increased from 24 for wider indent - const boxPadding = 8; - const maxBoxWidth = 280; // Max width for text in box + const boxPadding = 12; // Increased from 8 for more padding + const maxBoxWidth = 360; // Increased from 280 for wider boxes const maxCharsPerLine = Math.floor(maxBoxWidth / charWidth); // Calculate box dimensions for each item @@ -204,9 +204,9 @@ rect.setAttribute('height', item.boxHeight); if (item.type === 'outcome') { - // Outcomes: lighter background, dashed border - rect.setAttribute('fill', '#f9f9f9'); - rect.setAttribute('stroke', '#ccc'); + // Outcomes: pale red background, dashed border + rect.setAttribute('fill', '#ffe6e6'); + rect.setAttribute('stroke', '#d9534f'); rect.setAttribute('stroke-width', '1'); rect.setAttribute('stroke-dasharray', '3,3'); } else { From 3465c30533540c305ffe1f39e30b2c19784900c3 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 14:37:51 +0000 Subject: [PATCH 10/16] DOC-5991 initial tree diagram and format now solid --- assets/css/index.css | 4 +++ .../develop/data-types/compare-data-types.md | 33 +++---------------- .../render-codeblock-decision-tree.html | 8 +++++ 3 files changed, 17 insertions(+), 28 deletions(-) diff --git a/assets/css/index.css b/assets/css/index.css index f8500f64f2..1fe681161d 100644 --- a/assets/css/index.css +++ b/assets/css/index.css @@ -94,6 +94,10 @@ section.prose { padding: 1rem; } +.prose pre.decision-tree-source { + display: none; +} + .prose pre > code { @apply bg-none font-monogeist; } diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index 9ce0743678..d343193a3a 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -145,40 +145,17 @@ to manage multiple readers and ensure at-least-once delivery. ## Choose a data type The sections below explore the pros and cons of each data type for -particular tasks. +particular tasks. Note that you should regard +the suggestions as "rules-of-thumb" rather than strict prescriptions, since +there are potentially many subtle reasons to prefer one data type over another. ### Documents You would normally store document data using the string, hash, or JSON types. JSON generally has the highest requirements for memory and processing, -followed by hashes, and then strings. Use the considerations below as a guide to +followed by hashes, and then strings. Use the decision tree below as a guide to choosing the best data type for your task. -1. Do you need nested data structures (fields and arrays) or geospatial - index/query with Redis query engine? - - If so, use **JSON** since it is the only type that supports these features. - -2. Do you need to index/query using Redis query engine but can live without - nested data structures and geospatial data? - - If so, use **hashes** since they support indexing and querying with lower memory overhead and faster field access than JSON. - -3. Do you need to set expiration times on individual pieces of data within - the document? - - If so, use **hashes** for efficient field-level access and expiration. - -4. Do you need frequent access to individual data fields within the document? - - If the data fields are simple integers or individual bits and you can - easily refer to them by an integer index, use **strings** for efficient - access and minimum memory overhead. Otherwise, use **hashes** for - named fields and support for string and binary field values. - -5. For other simple documents with arbitrary internal structure, use **strings** - for simplicity and minimum memory overhead. - ```decision-tree rootQuestion: root questions: @@ -288,7 +265,7 @@ names matching the keys in the collection. need to store extra information for each key, consider using **hashes** with the keys as field names. -## Sequences +### Sequences You would normally store sequences of string or binary data using sorted sets, lists or streams. They each have advantages and disadvantages for particular purposes. diff --git a/layouts/_default/_markup/render-codeblock-decision-tree.html b/layouts/_default/_markup/render-codeblock-decision-tree.html index 6027aa1614..62547d5a84 100644 --- a/layouts/_default/_markup/render-codeblock-decision-tree.html +++ b/layouts/_default/_markup/render-codeblock-decision-tree.html @@ -1,4 +1,12 @@ {{- /* Render hook for decision tree code blocks */ -}} +{{- /* Preserve YAML source for visual display and JavaScript parsing */ -}} 
{{ .Inner | htmlEscape | safeHTML }}
+ +{{- /* Embed metadata as custom JSON for AI agents */ -}} +{{- $metadata := dict "type" "decision-tree" -}} + +{{ $jsonMetadata := $metadata | jsonify (dict "indent" " ") }} +{{ printf "" $jsonMetadata | safeHTML }} + {{ .Page.Store.Set "hasDecisionTree" true }} From 3cb6879d79f9528dc546d2ccc0d001155c1d0c5a Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 15:08:49 +0000 Subject: [PATCH 11/16] DOC-5991 added Collections tree --- .../develop/data-types/compare-data-types.md | 56 ++++++++++++++++++- static/js/decision-tree.js | 4 +- 2 files changed, 57 insertions(+), 3 deletions(-) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index d343193a3a..2b71a4ea08 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -260,11 +260,65 @@ names matching the keys in the collection. If so, use the bitmap features of **strings** for minimum memory overhead and efficient random access. String bitmaps also support bitwise operations that are equivalent to set operations such as union, intersection, and difference. -3. For arbitrary string or binary keys, use **sets** for efficient membership tests and +3. For arbitrary string or binary keys, use **sets** for efficient membership tests and set operations. If you *only* need membership tests on the keys, but you need to store extra information for each key, consider using **hashes** with the keys as field names. +```decision-tree +rootQuestion: root +questions: + root: + text: | + Do you need to store and retrieve the keys in an arbitrary order or in + lexicographical order? + whyAsk: | + Sorted sets are the only collection type that supports ordered iteration, + which is essential if you need to access elements in a specific order + answers: + yes: + value: "Yes" + outcome: + label: "Use sorted sets" + id: sortedSetsOutcome + no: + value: "No" + nextQuestion: extraInfo + extraInfo: + text: | + Do you need to store extra information for each key in the collection, + and do you NOT need set operations (union, intersection, difference)? + whyAsk: | + Hashes allow you to associate data with each key, but they don't support + set operations. If you need both extra data and set operations, sets are not suitable + answers: + yes: + value: "Yes" + outcome: + label: "Use hashes" + id: hashesOutcome + no: + value: "No" + nextQuestion: integerIndices + integerIndices: + text: | + Are the keys always simple integer indices in a known range? + whyAsk: | + String bitmaps provide minimum memory overhead and efficient random access + for integer indices, with bitwise operations equivalent to set operations + answers: + yes: + value: "Yes" + outcome: + label: "Use strings (bitmaps)" + id: stringsOutcome + no: + value: "No" + outcome: + label: "Use sets" + id: setsOutcome +``` + ### Sequences You would normally store sequences of string or binary data using sorted sets, diff --git a/static/js/decision-tree.js b/static/js/decision-tree.js index d0574cb295..18578abece 100644 --- a/static/js/decision-tree.js +++ b/static/js/decision-tree.js @@ -157,7 +157,7 @@ const topMargin = 10; const indentWidth = 40; // Increased from 24 for wider indent const boxPadding = 12; // Increased from 8 for more padding - const maxBoxWidth = 360; // Increased from 280 for wider boxes + const maxBoxWidth = 420; // Increased to accommodate longer questions const maxCharsPerLine = Math.floor(maxBoxWidth / charWidth); // Calculate box dimensions for each item @@ -174,7 +174,7 @@ maxDepth = Math.max(maxDepth, item.depth); }); - const svgWidth = leftMargin + (maxDepth + 1) * indentWidth + 320; + const svgWidth = leftMargin + (maxDepth + 1) * indentWidth + maxBoxWidth + 40; const svgHeight = topMargin + items.reduce((sum, item) => sum + item.boxHeight + 20, 0) + 10; const svg = document.createElementNS('http://www.w3.org/2000/svg', 'svg'); From 1aaa4ef56fd30209e041a9e6ad0d7adbe1d5eb40 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 15:31:22 +0000 Subject: [PATCH 12/16] DOC-5991 rounded out whole page --- .../develop/data-types/compare-data-types.md | 79 ++++++++++--------- 1 file changed, 43 insertions(+), 36 deletions(-) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index 2b71a4ea08..31e1fcc39f 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -245,26 +245,11 @@ types and for very simple collections, you can even use strings. They all allow basic membership tests, but have different additional features and tradeoffs. Sorted sets have the highest memory overhead and processing requirements, followed by sets, and then strings. -Use the considerations below as a guide to choosing the best data type for your task. +Use the decision tree below as a guide to choosing the best data type for your task. Note that if you need to store extra information for the keys in a set or sorted set, you can do so with an auxiliary hash or JSON object that has field names matching the keys in the collection. -1. Do you need to store and retrieve the keys in an arbitrary order or in - lexicographical order? - - If so, use **sorted sets** since they are the only collection type that supports ordered iteration. - -2. Are the keys always simple integer indices in a known range? - - If so, use the bitmap features of **strings** for minimum memory overhead and efficient random access. String bitmaps also support bitwise operations - that are equivalent to set operations such as union, intersection, and difference. - -3. For arbitrary string or binary keys, use **sets** for efficient membership tests and - set operations. If you *only* need membership tests on the keys, but you - need to store extra information for each key, consider using **hashes** with - the keys as field names. - ```decision-tree rootQuestion: root questions: @@ -286,8 +271,8 @@ questions: nextQuestion: extraInfo extraInfo: text: | - Do you need to store extra information for each key in the collection, - and do you NOT need set operations (union, intersection, difference)? + Do you need to store extra information for each key AND you don't need + set operations (union, intersection, difference)? whyAsk: | Hashes allow you to associate data with each key, but they don't support set operations. If you need both extra data and set operations, sets are not suitable @@ -323,22 +308,44 @@ questions: You would normally store sequences of string or binary data using sorted sets, lists or streams. They each have advantages and disadvantages for particular purposes. -Use the considerations below as a guide to choosing the best data type for your task. - -1. Do you frequently need to do any of the following? - - - Maintain an arbitrary priority order or lexicographical order of the elements - - Access individual elements or ranges of elements by index - - Perform basic set operations on the elements - - If so, use **sorted sets** since they are the only sequence type that supports these - operations directly. - -2. Do you need to store and retrieve elements primarily in timestamp order or - manage multiple consumers reading from the sequence? - - If so, use **streams** since they are the only sequence type that supports these - features natively. +Use the decision tree below as a guide to choosing the best data type for your task. -3. For simple sequences of string or binary data, use **lists** for efficient - push/pop operations at the head or tail. +```decision-tree +rootQuestion: root +questions: + root: + text: | + Do you need to maintain an arbitrary priority order, lexicographical order, + frequently access elements by index, or perform set operations? + whyAsk: | + Sorted sets are the only sequence type that supports both ordering and set operations. + While lists also support indexing, it is O(n) for lists but O(log n) for sorted sets, + so sorted sets are more efficient if you need frequent index access + answers: + yes: + value: "Yes" + outcome: + label: "Use sorted sets" + id: sortedSetsOutcome + no: + value: "No" + nextQuestion: timestampOrder + timestampOrder: + text: | + Do you need to store and retrieve elements primarily in timestamp order + or manage multiple consumers reading from the sequence? + whyAsk: | + Streams are the only sequence type that supports timestamp-based ordering + and consumer groups for managing multiple readers with at-least-once delivery + answers: + yes: + value: "Yes" + outcome: + label: "Use streams" + id: streamsOutcome + no: + value: "No" + outcome: + label: "Use lists" + id: listsOutcome +``` From 866d1dd84444f58c880c2ebfbcbd3f79b4a2305c Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 16:02:28 +0000 Subject: [PATCH 13/16] DOC-5991 all trees solid --- .../develop/data-types/compare-data-types.md | 12 ++++++++--- .../render-codeblock-decision-tree.html | 21 ++++++++++++++++++- 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/content/develop/data-types/compare-data-types.md b/content/develop/data-types/compare-data-types.md index 31e1fcc39f..2e11dac94b 100644 --- a/content/develop/data-types/compare-data-types.md +++ b/content/develop/data-types/compare-data-types.md @@ -156,7 +156,9 @@ types. JSON generally has the highest requirements for memory and processing, followed by hashes, and then strings. Use the decision tree below as a guide to choosing the best data type for your task. -```decision-tree +```decision-tree {id="documents-tree"} +id: documents-tree +scope: documents rootQuestion: root questions: root: @@ -250,7 +252,9 @@ Note that if you need to store extra information for the keys in a set or sorted set, you can do so with an auxiliary hash or JSON object that has field names matching the keys in the collection. -```decision-tree +```decision-tree {id="collections-tree"} +id: collections-tree +scope: collections rootQuestion: root questions: root: @@ -310,7 +314,9 @@ You would normally store sequences of string or binary data using sorted sets, lists or streams. They each have advantages and disadvantages for particular purposes. Use the decision tree below as a guide to choosing the best data type for your task. -```decision-tree +```decision-tree {id="sequences-tree"} +id: sequences-tree +scope: sequences rootQuestion: root questions: root: diff --git a/layouts/_default/_markup/render-codeblock-decision-tree.html b/layouts/_default/_markup/render-codeblock-decision-tree.html index 62547d5a84..d943bfe58e 100644 --- a/layouts/_default/_markup/render-codeblock-decision-tree.html +++ b/layouts/_default/_markup/render-codeblock-decision-tree.html @@ -2,8 +2,27 @@ {{- /* Preserve YAML source for visual display and JavaScript parsing */ -}} 
{{ .Inner | htmlEscape | safeHTML }}
+{{- /* Extract id and scope from YAML for metadata (only top-level fields) */ -}} +{{- $lines := split .Inner "\n" -}} +{{- $id := "" -}} +{{- $scope := "" -}} +{{- range $lines -}} + {{- /* Only process lines that don't start with whitespace (top-level fields) */ -}} + {{- if and (gt (len .) 0) (ne (index . 0) 32) (ne (index . 0) 9) -}} + {{- $trimmed := strings.TrimSpace . -}} + {{- if strings.HasPrefix $trimmed "id:" -}} + {{- $afterPrefix := strings.Replace $trimmed "id:" "" 1 -}} + {{- $id = strings.TrimSpace $afterPrefix -}} + {{- end -}} + {{- if strings.HasPrefix $trimmed "scope:" -}} + {{- $afterPrefix := strings.Replace $trimmed "scope:" "" 1 -}} + {{- $scope = strings.TrimSpace $afterPrefix -}} + {{- end -}} + {{- end -}} +{{- end -}} + {{- /* Embed metadata as custom JSON for AI agents */ -}} -{{- $metadata := dict "type" "decision-tree" -}} +{{- $metadata := dict "type" "decision-tree" "id" $id "scope" $scope -}} {{ $jsonMetadata := $metadata | jsonify (dict "indent" " ") }} {{ printf "" $jsonMetadata | safeHTML }} From 292f96838c9dc0295d0931b5a68a1f0b5df801a8 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Nov 2025 16:11:44 +0000 Subject: [PATCH 14/16] DOC-5991 final polish and lessons learned --- .../AI_RENDER_HOOK_LESSONS.md | 185 ++++++++++++++++++ .../render_hook_docs/DECISION_TREE_FORMAT.md | 156 +++++++++++++++ .../DECISION_TREE_IMPLEMENTATION_NOTES.md | 142 ++++++++++++++ build/render_hook_docs/README.md | 64 +++++- content/ai-agent-resources.md | 9 + content/develop/data-types/_index.md | 3 + 6 files changed, 554 insertions(+), 5 deletions(-) create mode 100644 build/render_hook_docs/DECISION_TREE_FORMAT.md create mode 100644 build/render_hook_docs/DECISION_TREE_IMPLEMENTATION_NOTES.md diff --git a/build/render_hook_docs/AI_RENDER_HOOK_LESSONS.md b/build/render_hook_docs/AI_RENDER_HOOK_LESSONS.md index 49fddd83b4..26bad369bd 100644 --- a/build/render_hook_docs/AI_RENDER_HOOK_LESSONS.md +++ b/build/render_hook_docs/AI_RENDER_HOOK_LESSONS.md @@ -409,19 +409,197 @@ if ((metaValue.startsWith('"') && metaValue.endsWith('"'))) { --- +## 21. Server-Side Metadata Extraction for AI Agents + +**Pattern**: Extract metadata from structured content (YAML, JSON) in the Hugo render hook and embed it as JSON in the HTML output for AI agents that don't execute JavaScript. + +**Implementation**: +```html +{{- /* Extract top-level fields only (no indentation) */ -}} +{{- $lines := split .Inner "\n" -}} +{{- $id := "" -}} +{{- range $lines -}} + {{- /* Check if line starts without whitespace (32=space, 9=tab) */ -}} + {{- if and (gt (len .) 0) (ne (index . 0) 32) (ne (index . 0) 9) -}} + {{- $trimmed := strings.TrimSpace . -}} + {{- if strings.HasPrefix $trimmed "id:" -}} + {{- $afterPrefix := strings.Replace $trimmed "id:" "" 1 -}} + {{- $id = strings.TrimSpace $afterPrefix -}} + {{- end -}} + {{- end -}} +{{- end -}} + +{{- /* Embed as JSON for AI agents */ -}} +{{- $metadata := dict "type" "decision-tree" "id" $id -}} +{{ $jsonMetadata := $metadata | jsonify (dict "indent" " ") }} +{{ printf "" $jsonMetadata | safeHTML }} +``` + +**Key Considerations**: +- **Indentation detection**: When parsing nested structures, only extract top-level fields by checking if the line starts with whitespace +- **String manipulation**: Use `strings.Replace` instead of `strings.TrimPrefix` for more reliable extraction +- **Character codes**: Use ASCII codes (32 for space, 9 for tab) to detect indentation reliably +- **Metadata format**: Use simple JSON (not JSON-LD) for clarity and ease of parsing +- **Data attributes**: Use `data-*` attributes to mark metadata elements for AI agents + +**Why This Matters**: +- AI agents typically don't execute JavaScript, so metadata must be in static HTML +- Server-side extraction ensures metadata is available even if JavaScript fails +- Structured metadata helps AI agents understand the purpose and scope of components +- Separating metadata from content improves maintainability + +**Lesson**: Always provide metadata in static HTML for AI agents. Use server-side extraction to ensure accuracy and avoid relying on JavaScript parsing. + +--- + +## 22. Handling Nested Structures in Hugo Templates + +**Pattern**: When extracting data from nested YAML/JSON structures, distinguish between top-level and nested fields using indentation detection. + +**Problem**: If you extract all occurrences of a field (e.g., `id:`), you'll get nested occurrences too, leading to incorrect values. + +**Solution**: Check indentation before processing: +```html +{{- if and (gt (len .) 0) (ne (index . 0) 32) (ne (index . 0) 9) -}} + {{- /* Process only top-level lines */ -}} +{{- end -}} +``` + +**Why This Works**: +- YAML indentation is significant and indicates nesting level +- Top-level fields have no leading whitespace +- Nested fields have leading spaces or tabs +- Character code 32 = space, 9 = tab + +**Lesson**: When parsing nested structures in Hugo templates, use indentation detection to distinguish between levels. This prevents extracting nested values when you only want top-level ones. + +--- + +## 23. Progressive Enhancement with Metadata + +**Pattern**: Combine progressive enhancement with metadata embedding to serve both humans and AI agents from a single source. + +**Architecture**: +1. **Server-side (Hugo)**: + - Extract metadata from source content + - Embed metadata as JSON in HTML + - Preserve raw source in `
` element
+
+2. **Client-side (JavaScript)**:
+   - Parse raw source for rendering
+   - Use metadata for context/identification
+   - Enhance with interactivity
+
+3. **AI agents**:
+   - Read static JSON metadata
+   - Parse raw source from `
` element
+   - No JavaScript execution needed
+
+**Benefits**:
+- Single source of truth (the YAML/JSON in the Markdown)
+- Metadata available to all consumers (humans, AI agents, JavaScript)
+- Graceful degradation if JavaScript fails
+- AI-friendly without extra work
+
+**Lesson**: Design render hooks to serve multiple audiences simultaneously. Metadata should be available in static HTML, not just in JavaScript.
+
+---
+
+## 24. Text Wrapping and Box Sizing in SVG Diagrams
+
+**Pattern**: When rendering text in SVG boxes, calculate dimensions based on character width and implement text wrapping to fit within maximum width.
+
+**Implementation**:
+```javascript
+const charWidth = 8; // Space Mono at 14px
+const maxBoxWidth = 420;
+const maxCharsPerLine = Math.floor(maxBoxWidth / charWidth);
+
+function wrapText(text, maxChars) {
+  const words = text.split(' ');
+  const lines = [];
+  let currentLine = '';
+
+  for (const word of words) {
+    if ((currentLine + ' ' + word).length > maxChars) {
+      if (currentLine) lines.push(currentLine);
+      currentLine = word;
+    } else {
+      currentLine = currentLine ? currentLine + ' ' + word : word;
+    }
+  }
+  if (currentLine) lines.push(currentLine);
+  return lines;
+}
+```
+
+**Considerations**:
+- **Font metrics**: Different fonts have different character widths
+- **Padding**: Account for box padding when calculating available width
+- **Line height**: Multiply number of lines by line height for total box height
+- **Dynamic sizing**: Calculate SVG dimensions based on content, not fixed values
+
+**Common Pitfall**: Hardcoding SVG width can cause content to be cut off. Instead:
+```javascript
+const svgWidth = leftMargin + (maxDepth + 1) * indentWidth + maxBoxWidth + 40;
+```
+
+**Lesson**: Calculate SVG dimensions dynamically based on content. Account for all visual elements (padding, margins, decorations) when sizing boxes and containers.
+
+---
+
+## 25. Scope and Context Metadata for Component Discovery
+
+**Pattern**: Add `scope` or `category` metadata to components to help AI agents understand their purpose and applicability.
+
+**Implementation**:
+```yaml
+id: documents-tree
+scope: documents
+rootQuestion: root
+questions:
+  # ...
+```
+
+**Benefits**:
+- **Discoverability**: AI agents can filter components by scope
+- **Context awareness**: Agents know which problem domain each component addresses
+- **Prevents misapplication**: Agents won't use a "collections" tree to recommend document storage
+- **Relationship mapping**: Enables linking related components
+
+**Use Cases**:
+- Filtering decision trees by data type category
+- Finding all components related to a specific feature
+- Organizing components hierarchically
+- Providing context in search results
+
+**Lesson**: Add semantic metadata (scope, category, type) to components. This helps AI agents understand purpose and applicability, enabling better recommendations and filtering.
+
+---
+
 ## Quick Checklist for Future Render Hooks
 
+### Core Patterns
 - [ ] Preserve source content in a `
` or similar element
 - [ ] Use page store pattern to avoid duplicate resource loading
 - [ ] Place static JavaScript in `static/js/`, not `assets/js/`
 - [ ] Avoid `innerHTML` with dynamic content; use safe DOM methods
 - [ ] Use `data-*` attributes to pass server data to JavaScript
+
+### Testing & Accessibility
 - [ ] Test with multiple instances on the same page
 - [ ] Consider state persistence if needed
 - [ ] Use semantic HTML and proper accessibility attributes
 - [ ] Document the Markdown format clearly
 - [ ] Provide sensible defaults for optional parameters
+
+### Hugo-Specific
 - [ ] Remember Hugo converts attribute names to lowercase
+- [ ] Use indentation detection when parsing nested structures
+- [ ] Extract top-level metadata in render hook (not JavaScript)
+- [ ] Use `strings.Replace` for reliable string manipulation in templates
+
+### Advanced Features
 - [ ] Use SVG for complex visual structures
 - [ ] Track processed lines when parsing nested structures
 - [ ] Account for all visual elements in dimension calculations
@@ -429,3 +607,10 @@ if ((metaValue.startsWith('"') && metaValue.endsWith('"'))) {
 - [ ] Handle string unescaping during parsing, not rendering
 - [ ] Create comprehensive format documentation
 
+### AI Agent Compatibility
+- [ ] Embed metadata as JSON in static HTML (not just JavaScript)
+- [ ] Add `scope` or `category` metadata for component discovery
+- [ ] Use `data-*` attributes to mark metadata elements
+- [ ] Ensure metadata is available without JavaScript execution
+- [ ] Preserve raw source content for AI parsing
+
diff --git a/build/render_hook_docs/DECISION_TREE_FORMAT.md b/build/render_hook_docs/DECISION_TREE_FORMAT.md
new file mode 100644
index 0000000000..9ee33ba524
--- /dev/null
+++ b/build/render_hook_docs/DECISION_TREE_FORMAT.md
@@ -0,0 +1,156 @@
+# Decision Tree Format Specification
+
+## Overview
+
+Decision trees are structured YAML documents that guide users through a series of questions to reach a recommendation. They are rendered as interactive SVG diagrams with boxes, connecting lines, and Yes/No branch labels.
+
+## Basic Structure
+
+```yaml
+```decision-tree {id="documents-tree"}
+id: documents-tree
+scope: documents
+rootQuestion: root
+questions:
+    root:
+        text: "Your question here?"
+        whyAsk: "Explanation of why this question matters"
+        answers:
+            yes:
+                value: "Yes"
+                outcome:
+                    label: "Recommendation"
+                    id: outcomeId
+            no:
+                value: "No"
+                nextQuestion: nextQuestionId
+    nextQuestionId:
+        text: "Follow-up question?"
+        whyAsk: "Why this matters"
+        answers:
+            yes:
+                value: "Yes"
+                outcome:
+                    label: "Recommendation"
+                    id: outcomeId
+            no:
+                value: "No"
+                outcome:
+                    label: "Alternative recommendation"
+                    id: altOutcomeId
+```
+```
+
+## Fields
+
+### Top-level
+
+- **`id`** (required): Unique identifier for this decision tree (e.g., `documents-tree`, `collections-tree`). Used for discovery and referencing by AI agents.
+- **`scope`** (required): Category or domain this tree applies to (e.g., `documents`, `collections`, `sequences`). Helps AI agents understand the tree's purpose and applicability.
+- **`rootQuestion`** (required): The ID of the starting question
+- **`questions`** (required): Object containing all questions, keyed by ID
+
+### Question Object
+
+- **`text`** (required): The question text. Can span multiple lines using YAML's `|` literal block syntax
+- **`whyAsk`** (required): Explanation of why this question matters. Helps AI agents understand the decision logic
+- **`answers`** (required): Object with `yes` and `no` keys
+
+### Answer Object
+
+Each answer (`yes` or `no`) contains:
+
+- **`value`** (required): Display text ("Yes" or "No")
+- **`outcome`** (optional): Terminal recommendation
+  - `label`: Text to display (e.g., "Use JSON")
+  - `id`: Unique identifier for this outcome
+- **`nextQuestion`** (optional): ID of the next question to ask
+
+**Note**: Each answer must have either `outcome` or `nextQuestion`, not both.
+
+### Outcome Object
+
+- **`label`** (required): The recommendation text
+- **`id`** (required): Unique identifier (e.g., `jsonOutcome`, `hashOutcome`)
+
+## Multi-line Text
+
+Use YAML's literal block syntax (`|`) for multi-line text:
+
+```yaml
+text: |
+    Do you need nested data structures
+    (fields and arrays) or geospatial
+    index/query with Redis query engine?
+whyAsk: |
+    JSON is the only document type that supports
+    deeply nested structures and integrates with
+    the query engine for those structures
+```
+
+## Code Block Attributes
+
+The code block fence supports the following attributes:
+
+- **`id`** (optional): Unique identifier for the tree. Should match the `id` field in the YAML. Used by Hugo to pass metadata to the render hook.
+
+Example:
+```markdown
+```decision-tree {id="documents-tree"}
+id: documents-tree
+scope: documents
+# ...
+```
+```
+
+## Best Practices
+
+1. **Use descriptive IDs**: `root`, `hashQuestion`, `jsonOutcome` are clearer than `q1`, `q2`
+2. **Keep questions concise**: Aim for 1-2 lines when possible
+3. **Explain the rationale**: The `whyAsk` field helps users and AI agents understand the decision logic
+4. **Reuse outcomes**: Multiple paths can lead to the same outcome (same `id`)
+5. **Consistent naming**: Use camelCase for IDs, end question IDs with "Question"
+6. **Match fence and YAML IDs**: The `id` in the code block fence should match the `id` field in the YAML for consistency
+7. **Use meaningful scopes**: Choose scope values that clearly indicate the tree's domain (e.g., `documents`, `collections`, `sequences`)
+
+## Example: Redis Data Type Selection
+
+See `content/develop/data-types/compare-data-types.md` for a complete example.
+
+## Rendering
+
+Decision trees are rendered as:
+- **SVG diagram** for humans (with boxes, lines, and labels)
+- **Normalized JSON** embedded for AI agents (accessible via `.html.md` URLs)
+- **Raw YAML** preserved in `
` element for accessibility
+
+## AI Agent Compatibility
+
+The format is designed to be easily parseable by AI agents:
+
+### Metadata Embedding
+- **Server-side JSON**: Each tree is embedded with metadata as `