Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build/components/example.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
'java-async': '//',
'java-reactive': '//',
'go': '//',
'c': '//',
'c#': '//',
'c#-sync': '//',
'c#-async': '//',
Expand Down
6 changes: 6 additions & 0 deletions build/local_examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
'.py': 'python',
'.js': 'node.js',
'.go': 'go',
'.c': 'c',
'.h': 'c',
'.cs': 'c#',
'.java': 'java',
'.php': 'php',
Expand All @@ -34,6 +36,7 @@
'python': 'Python',
'node.js': 'Node.js',
'go': 'Go',
'c': 'C',
'c#': 'C#-Sync',
'java': 'Java-Sync', # Default to sync, could be overridden
'php': 'PHP',
Expand All @@ -57,12 +60,15 @@ def get_client_name_from_language_and_path(language: str, path: str) -> str:
"""Get client name from language with path-based overrides.

For Java (.java) files, override based on path substrings:
- If 'lettuce-sync' in path -> Lettuce-Sync
- If 'lettuce-async' in path -> Java-Async
- If 'lettuce-reactive' in path -> Java-Reactive

Substring checks are case-sensitive and can appear anywhere in the path.
"""
if language == 'java':
if 'lettuce-sync' in path:
return 'Lettuce-Sync'
if 'lettuce-async' in path:
return 'Java-Async'
if 'lettuce-reactive' in path:
Expand Down
42 changes: 32 additions & 10 deletions build/tcedocs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ There are two sections that need to updated when new languages are added.
1. In the `[params]` section:

```toml
clientsExamples = ["Python", "Node.js", "Java-Sync", "Java-Async", "Java-Reactive", "Go", "C#", "RedisVL", "PHP"]
clientsExamples = ["Python", "Node.js", "Java-Sync", "Lettuce-Sync", "Java-Async", "Java-Reactive", "Go", "C", "C#-Sync", "C#-Async", "RedisVL", "PHP", "Rust-Sync", "Rust-Async"]
```

The order of the `clientsExamples` list matters: it's the order in which the language tabs are presented for each code example.
Expand All @@ -18,13 +18,18 @@ There are two sections that need to updated when new languages are added.
[params.clientsConfig]
"Python"={quickstartSlug="redis-py"}
"Node.js"={quickstartSlug="nodejs"}
"Java-sync"={quickstartSlug="jedis"}
"Java-async"={quickstartSlug="lettuce"}
"Java-reactive"={quickstartSlug="lettuce"}
"Java-Sync"={quickstartSlug="jedis"}
"Lettuce-Sync"={quickstartSlug="lettuce"}
"Java-Async"={quickstartSlug="lettuce"}
"Java-Reactive"={quickstartSlug="lettuce"}
"Go"={quickstartSlug="go"}
"C#"={quickstartSlug="dotnet"}
"C"={quickstartSlug="hiredis"}
"C#-Sync"={quickstartSlug="dotnet"}
"C#-Async"={quickstartSlug="dotnet"}
"RedisVL"={quickstartSlug="redis-vl"}
"PHP"={quickstartSlug="php"}
"Rust-Sync"={quickstartSlug="rust"}
"Rust-Async"={quickstartSlug="rust"}
```

This configuration, along with the configuration steps below, is used to control the behavior of the Hugo shortcode that was developed to show tabbed code examples.
Expand All @@ -36,7 +41,7 @@ A shortcode is a simple snippet inside a content file that Hugo will render usin

The folder `data/components` contains one component configuration file for each supported language. These files contain information about the GitHub repos that house the code examples.

Here is the configuration file for Python, `redis_py.json`:
Here is the configuration file for Python, `redis_py.json`:

```json
{
Expand Down Expand Up @@ -65,15 +70,20 @@ Register your component file by adding it to the `clients` array in the `index.j
Here is an example:
```json
"clients": [
"nredisstack",
"nredisstack_sync",
"nredisstack_async",
"go_redis",
"node_redis",
"php",
"redis_py",
"jedis",
"lettuce_sync",
"lettuce_async",
"lettuce_reactive",
"redis_vl"
"redis_vl",
"redis_rs_sync",
"redis_rs_async",
"hi_redis"
]
```

Expand All @@ -99,14 +109,18 @@ PREFIXES = {
'java-async': '//',
'java-reactive': '//',
'go': '//',
'c': '//',
'c#': '//',
'redisvl': '#',
'php': '//'
'php': '//',
'rust': '//'
}
```

The `TEST_MARKER` dictionary maps programming languages to test framework annotations, which allows the parser to filter such source code lines out. The `PREFIXES` dictionary maps each language to its comment prefix. Python, for example, uses a hashtag (`#`) to start a comment.

⚠️ **CRITICAL**: The `PREFIXES` dictionary is **essential** for the example parser to work. If you add a new language, you **must** add an entry to this dictionary, or examples will fail to process with an "Unknown language" error. This is the most commonly missed step when adding a new language.

## Understand special comments in the example source code files

Each code example uses special comments, such as `HIDE_START` and `REMOVE_START`, to control how the examples are displayed. The following list gives an explanation:
Expand All @@ -130,10 +144,12 @@ Add a source code file to an appropriate client repo. Consult the /data/componen

| Programming Language | GitHub Repo | Default directory |
|----------------------|-----------------------------------------------------|---------------------------------------------------|
| C | [hiredis](https://github.com/redis/hiredis) | `examples` |
| C# | [NRedisStack](https://github.com/redis/NRedisStack) | `tests/Doc` |
| Go | [go-redis](https://github.com/redis/go-redis) | `doctests` |
| Java | [jedis](https://github.com/redis/jedis) | `src/test/java/io/redis/examples` |
| | [Lettuce](https://github.com/redis/lettuce) | `src/test/java/io/redis/examples/async` or |
| | [Lettuce](https://github.com/redis/lettuce) | `src/test/java/io/redis/examples/sync`, |
| | | `src/test/java/io/redis/examples/async`, or |
| | | `src/test/java/io/redis/examples/reactive` |
| Node.js | [node-redis](https://github.com/redis/node-redis) | `doctests` |
| PHP | [Predis](https://github.com/predis/predis) | Examples, for now, are stored in `local_examples` |
Expand All @@ -148,9 +164,15 @@ At times, it can take quite a while to get new or updated examples through the r
local_examples
├── client-specific
│   ├── go
│ ├── c
│ │ ...

│   │   ...
│   ├── jedis
│   │   ...
│ ├── lettuce-sync
│ │ ...

│   ├── lettuce-async
│   │   ...
│   ├── lettuce-reactive
Expand Down
137 changes: 133 additions & 4 deletions build/tcedocs/SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,15 +245,16 @@ The system operates in three distinct phases:

Some languages have multiple client implementations (sync/async, different libraries). The system uses directory path to determine which variant:

- Java files in `lettuce-async/` → `Java-Async` (Lettuce async client)
- Java files in `lettuce-sync/` → `Lettuce-Sync` (Lettuce synchronous client)
- Java files in `lettuce-async/` → `Java-Async` (Lettuce asynchronous client)
- Java files in `lettuce-reactive/` → `Java-Reactive` (Lettuce reactive client)
- Java files elsewhere → `Java-Sync` (Jedis synchronous client)
- Rust files in `rust-async/` → `Rust-Async`
- Rust files in `rust-sync/` → `Rust-Sync`
- C# files in `async/` → `C#-Async`
- C# files in `sync/` → `C#-Sync`

This allows the same language to appear multiple times in the tab interface with different implementations.
This allows the same language to appear multiple times in the tab interface with different implementations. The order of checks matters: more specific paths (e.g., `lettuce-sync`) should be checked before generic ones (e.g., `Java-Sync`).

**Outputs**:
- Copies files to `examples/{example_id}/local_{filename}`
Expand Down Expand Up @@ -1126,7 +1127,7 @@ def main():
**Client Examples Order**:
```toml
[params]
clientsExamples = ["Python", "Node.js", "Java-Sync", "Java-Async", "Java-Reactive", "Go", "C#-Sync", "C#-Async", "RedisVL", "PHP", "Rust-Sync", "Rust-Async"]
clientsExamples = ["Python", "Node.js", "Java-Sync", "Lettuce-Sync", "Java-Async", "Java-Reactive", "Go", "C", "C#-Sync", "C#-Async", "RedisVL", "PHP", "Rust-Sync", "Rust-Async"]
```

This controls:
Expand Down Expand Up @@ -1489,9 +1490,10 @@ See [Appendix: Adding a Language](#adding-a-language) for complete step-by-step
1. ✅ Update `config.toml` (clientsExamples, clientsConfig)
2. ✅ Create component config in `data/components/`
3. ✅ Register in `data/components/index.json`
4. ✅ Add language to `PREFIXES` in `build/components/example.py`
4. ✅ Add language to `PREFIXES` in `build/components/example.py` ⚠️ **CRITICAL - DO NOT SKIP**
5. ✅ Add extension mapping in `build/local_examples.py`
6. ✅ Add test markers if needed
7. ⚠️ Check if Jupyter notebook support is needed (update `build/jupyterize/` if applicable)

### Customizing the UI

Expand Down Expand Up @@ -2138,3 +2140,130 @@ OK
{{< /clients-example >}}
```


## Lessons Learned: Adding the C (hiredis) Client

### Critical Discovery: The PREFIXES Dictionary

When adding the C client, a critical step was initially missed: **adding the language to the `PREFIXES` dictionary in `build/components/example.py`**.

**Why this matters**: The `PREFIXES` dictionary maps each language to its comment prefix character(s). This is used by the example parser to:
- Identify special markers like `EXAMPLE:`, `STEP_START`, `HIDE_START`, etc.
- Parse metadata from source files
- Process example files correctly

**What happens if you skip this step**:
- The example parser will fail with an error: `Unknown language "c" for example {path}`
- Examples won't be processed
- The build system will silently skip C examples
- No error message will appear in the build output (just a debug log)

**The fix**:
```python
# In build/components/example.py, add to PREFIXES dictionary:
PREFIXES = {
...
'c': '//', # C uses // for comments
...
}
```

### Complete Checklist for Adding a New Language

The original checklist was incomplete. Here's the comprehensive version:

**Configuration Files**:
1. ✅ `config.toml` - Add to `clientsExamples` list and `clientsConfig` section
2. ✅ `data/components/{language}.json` - Create component configuration
3. ✅ `data/components/index.json` - Register the component

**Build System**:
4. ✅ `build/components/example.py` - **CRITICAL**: Add to `PREFIXES` dictionary
5. ✅ `build/components/example.py` - Add to `TEST_MARKER` dictionary (if language has test annotations)
6. ✅ `build/local_examples.py` - Add file extension mapping to `EXTENSION_TO_LANGUAGE`
7. ✅ `build/local_examples.py` - Add language to `LANGUAGE_TO_CLIENT` mapping

**Optional (if Jupyter notebook support is needed)**:
8. ⚠️ `build/jupyterize/jupyterize.py` - Add to `KERNEL_SPECS` dictionary
9. ⚠️ `build/jupyterize/jupyterize_config.json` - Add language-specific boilerplate and unwrap patterns

**Documentation**:
10. ✅ `build/tcedocs/SPECIFICATION.md` - Update examples and checklist
11. ✅ `build/tcedocs/README.md` - Update tables and examples

### Pre-existing Examples

**Important**: Before adding a new language, check if examples already exist in the repository:
- Look in `local_examples/client-specific/{language}/` for local examples
- Check the client repository for remote examples
- Verify the component configuration points to the correct example directory

For C (hiredis), there was already a `landing.c` example in `local_examples/client-specific/c/` that was ready to be processed once the language was properly configured.

### Language-Specific Comment Prefixes

Different languages use different comment styles. When adding a language, ensure the correct prefix is used:

| Language | Prefix | Example |
|----------|--------|---------|
| Python | `#` | `# EXAMPLE: my_example` |
| C | `//` | `// EXAMPLE: my_example` |
| Java | `//` | `// EXAMPLE: my_example` |
| Go | `//` | `// EXAMPLE: my_example` |
| C# | `//` | `// EXAMPLE: my_example` |
| PHP | `//` | `// EXAMPLE: my_example` |
| Rust | `//` | `// EXAMPLE: my_example` |
| Node.js | `//` | `// EXAMPLE: my_example` |

**Critical**: The `PREFIXES` dictionary uses **lowercase** language names as keys, but the `Example` class converts the language to lowercase before accessing it (line 57 in `example.py`).

### Verification Steps

After adding a new language, verify the integration:

```bash
# 1. Check that the language is recognized
grep -r "c" build/components/example.py # Should find 'c': '//' in PREFIXES

# 2. Process examples
python3 build/local_examples.py

# 3. Verify examples were processed
grep -i "landing" data/examples.json | grep -i "c"

# 4. Check for errors in the build output
python3 build/make.py 2>&1 | grep -i "error\|unknown language"

# 5. Build and serve
hugo serve
```

### Common Mistakes to Avoid

1. **Forgetting the PREFIXES entry**: This is the most common mistake. The build will appear to succeed but examples won't be processed.

2. **Case sensitivity**: Language names in `PREFIXES` must be lowercase, but `clientsExamples` in `config.toml` uses proper case (e.g., `"C"` not `"c"`).

3. **Inconsistent naming**: Ensure the language name is consistent across:
- `config.toml` clientsExamples (proper case, e.g., `"C"`)
- `config.toml` clientsConfig keys (proper case, e.g., `"C"`)
- `build/local_examples.py` LANGUAGE_TO_CLIENT values (proper case, e.g., `'C'`)
- `build/components/example.py` PREFIXES keys (lowercase, e.g., `'c'`)

4. **Missing component registration**: If the component isn't registered in `data/components/index.json`, remote examples won't be fetched.

5. **Wrong file extension mapping**: Ensure the file extension correctly maps to the language name in `EXTENSION_TO_LANGUAGE`.

### Single-Variant vs Multi-Variant Languages

**Single-variant languages** (Python, Go, PHP, C):
- One client implementation per language
- No path-based client name overrides needed
- File extension mapping is straightforward

**Multi-variant languages** (Java, Rust, C#):
- Multiple client implementations (e.g., Sync, Async, Reactive)
- Require path-based client name overrides in `get_client_name_from_language_and_path()`
- More complex configuration

C is a single-variant language, so it doesn't require path-based overrides.
14 changes: 8 additions & 6 deletions config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ tagManagerId = "GTM-TKZ6J9R"
gitHubRepo = "https://github.com/redis/docs"

# Display and sort order for client examples
clientsExamples = ["Python", "Node.js", "Java-Sync", "Java-Async", "Java-Reactive", "Go", "C#-Sync", "C#-Async", "RedisVL", "PHP", "Rust-Sync", "Rust-Async"]
clientsExamples = ["Python", "Node.js", "Java-Sync", "Lettuce-Sync", "Java-Async", "Java-Reactive", "Go", "C", "C#-Sync", "C#-Async", "RedisVL", "PHP", "Rust-Sync", "Rust-Async"]
searchService = "/convai/api/search-service"
ratingsService = "/docusight/api/rate/docs"

Expand All @@ -60,16 +60,18 @@ rdi_current_version = "1.15.0"
[params.clientsConfig]
"Python"={quickstartSlug="redis-py"}
"Node.js"={quickstartSlug="nodejs"}
"Java-sync"={quickstartSlug="jedis"}
"Java-async"={quickstartSlug="lettuce"}
"Java-reactive"={quickstartSlug="lettuce"}
"Java-Sync"={quickstartSlug="jedis"}
"Lettuce-Sync"={quickstartSlug="lettuce"}
"Java-Async"={quickstartSlug="lettuce"}
"Java-Reactive"={quickstartSlug="lettuce"}
"Go"={quickstartSlug="go"}
"C"={quickstartSlug="hiredis"}
"C#-Sync"={quickstartSlug="dotnet"}
"C#-Async"={quickstartSlug="dotnet"}
"RedisVL"={quickstartSlug="redis-vl"}
"PHP"={quickstartSlug="php"}
"Rust-sync"={quickstartSlug="rust"}
"Rust-async"={quickstartSlug="rust"}
"Rust-Sync"={quickstartSlug="rust"}
"Rust-Async"={quickstartSlug="rust"}

# Mount directories for duplicate content
[module]
Expand Down
Loading