docs: restructure sidebar — split Observability from Evaluation, move LLM-as-a-Judge to How-To

[ajbozarth]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Fixes Publish the Telemetry sidebar entry #838

Restructures the docs sidebar in two ways:

1. Flatten Telemetry sub-group: the nested "Telemetry" dropdown is removed; the four observability pages (Telemetry, Tracing, Metrics, Logging) are now listed flat. Also removes the redundant sidebarTitle: "Overview" override from telemetry.md.

2. Split "Evaluation and Observability": moves evaluate-with-llm-as-a-judge into the How-To section (it was already tagged diataxis: how-to) and renames the remaining group to "Observability". Renames the folder from evaluation-and-observability/ to observability/ to match, with redirects for all old URLs. Adds a contextual link to the evaluate page from the Requirements System concepts page.

Before (closed):

Before (open):

After — Observability section:

After — How-To section:

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[ajbozarth]

As detailed in #838 (comment) I am not a fan of this change, but this is what @abrahamdaniels has requested

Edit: after discussion over slack I misunderstood the goal, I fully support this updated implementation

[ajbozarth]

After discussion with @abrahamdaniels over Slack we reworked the intent of this PR. I updated the description to match the new intent and this is ready for review again

[abrahamdaniels]

Looks good !

[planetf1]

The removal of the "Evaluate with LLM-as-a-Judge" cross-links from tracing.md and telemetry.md silently drops documentation of real, observable behaviour.

The LLM-as-a-judge call in Requirement.validate() goes through the same backend.generate_from_context() path as the primary generation — which means it produces an OTel span with mellea.action_type = "Requirement", distinguishable in your trace tree from the main generation span. This is worth surfacing explicitly on the evaluate page rather than leaving users to discover it from the span attributes.

Suggested addition to evaluate-with-llm-as-a-judge.md (e.g. at the end of the "How it works" section):

Observability: If OpenTelemetry tracing is enabled, judge calls appear as child spans with mellea.action_type = "Requirement", making them distinguishable from the main generation span. See Tracing.

This keeps the two sections decoupled in the sidebar while preserving the connection for users who care about it.

[planetf1]

Minor pre-existing point worth a check while this area is being touched: the pages now in observability/ have mixed diataxis tags — telemetry.md, tracing.md, and metrics.md are tagged how-to, while logging.md is tagged reference — but all four sit in a feature-grouped sidebar section rather than the How-To or Reference sections.

Not introduced by this PR, and feature-grouping over diataxis-grouping is a defensible choice for discoverability. Just flagging in case you want to align the tags with actual placement (or vice versa) as a follow-up.