docs: architecture.md refresh + docs/ideas untrack + showcase Part A expected output#23
Merged
Merged
Conversation
architecture.md — updated to reflect the current class layout. Additions reflect code that landed across recent releases but was never added to the diagram: - Step::RetryPolicy (DSL-level config object, distinct from RetryExecutor) - Pipeline::Result (per-step outputs + aggregated trace) - Eval::AggregatedReport, Eval::BaselineDiff, Eval::PromptDiffComparator, Eval::EvalHistory, Eval::RetryOptimizer - Eval::ExpectationEvaluator (replaces the outdated Eval::TraitEvaluator entry — that class no longer exists) - OptimizeRakeTask alongside RakeTask docs/ideas/ is already in .gitignore but docs/ideas/10_eval_defaults.md was committed before the ignore rule landed, so git kept tracking it. Untracked via `git rm --cached` — file stays on disk locally, no future idea notes will be committed by accident. No version bump. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
examples/README.md showed only Part B (the "after" trace with retry success) but omitted Part A (schema-only pain point). Readers scrolling the examples index on GitHub saw the fix but not the problem the fix solves, which is the whole reason Part A exists in the script. Both sections now verified byte-identical to `ruby examples/11_fallback_showcase.rb`. No version bump. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
justi
changed the title
There was a problem hiding this comment.
Pull request overview
Refreshes documentation to reflect the current ruby_llm-contract class layout and removes an accidentally-tracked “ideas” note from version control.
Changes:
- Updated
docs/architecture.mdclass tree to include newer pipeline/step/eval/rake components and remove stale entries. - Removed
docs/ideas/10_eval_defaults.mdfrom git tracking (sodocs/ideas/remains ignored going forward).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| docs/ideas/10_eval_defaults.md | Deleted previously-tracked idea note so the ignored docs/ideas/ directory stays untracked. |
| docs/architecture.md | Updated architecture/class tree to match current codebase structure and tooling. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
19
to
22
| ├── Contract::Definition # parse strategy + validates | ||
| │ ├── Contract::Parser # :json / :text (auto-inferred from output type) | ||
| │ ├── Contract::Validator # runs parse + schema + validates | ||
| │ ├── Contract::Validator # runs parse + schema + validates + observations | ||
| │ └── Contract::SchemaValidator # JSON Schema validation (nested) |
There was a problem hiding this comment.
The architecture tree lists Contract::Definition, Contract::Parser, Contract::Validator, and Contract::SchemaValidator, but these constants don’t exist under a Contract:: namespace. In code they are RubyLLM::Contract::Definition, RubyLLM::Contract::Parser, RubyLLM::Contract::Validator, and RubyLLM::Contract::SchemaValidator. Updating the names (either fully-qualify them or drop the Contract:: prefix) would avoid misleading readers.
| ├── Eval::Report # score, pass_rate, per-case results | ||
| ├── Eval::AggregatedReport # merged reports across models or runs | ||
| ├── Eval::CaseResult # value object (name, passed?, output, expected, mismatches, cost) | ||
| ├── Eval::ExpectationEvaluator # expected / expected_traits / evaluator proc |
There was a problem hiding this comment.
Eval::ExpectationEvaluator is described as handling expected_traits, but trait evaluation is implemented separately in RubyLLM::Contract::Eval::TraitEvaluator (and ExpectationEvaluator only selects evaluators for expected values like Hash/Regexp/exact). Consider updating this description and/or adding Eval::TraitEvaluator to the tree so the responsibilities match the current code.
Suggested change
| ├── Eval::ExpectationEvaluator # expected / expected_traits / evaluator proc | |
| ├── Eval::ExpectationEvaluator # expected matching (Hash / Regexp / exact) | |
| ├── Eval::TraitEvaluator # expected_traits evaluation |
This was referenced Apr 23, 2026
justi
added a commit
that referenced
this pull request
Apr 24, 2026
Adoption-friction release. No runtime behavior changes — every delta is in `docs/`, `examples/`, or `spec/integration/` (plus version.rb / Gemfile.lock bumps). Upgrading from 0.7.2 picks up the expanded guide set, the consolidated runnable showcases, and one extra integration spec. Consolidates 7 merged PRs (#21–#27) into one release: - #21 Guide rewrite + adoption friction (why.md, "Do I need this?", outcome labels, TL;DR boxes) - #22 Runnable aha-moment showcases (fallback + retry variants) - #23 architecture.md refresh + docs/ideas untracked - #24 Schema pitfall fix (5 example files) + expected output coverage - #25 Examples consolidation — drop Reddit, renumber 00-06, restore pipeline + real-LLM minimal - #26 Rails integration FAQ guide (7 pre-emptive questions) - #27 Pipeline-level run_eval coverage — closes the "09 STEP 5" known issue from 0.7.2 Copilot review of the CHANGELOG itself flagged two inaccuracies before merge: - "No gem-level code changes" replaced with "No runtime behavior changes" so version.rb / Gemfile.lock bumps are not misrepresented. - Stale `examples/09_eval_dataset.rb` reference updated to current `05_eval_dataset.rb` after the renumber. Verification: 1287 specs pass, 6/6 test-adapter examples run clean, bundle install resolves 0.7.3. Full changelog entry on main in CHANGELOG.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three small documentation cleanups.
1. architecture.md refresh
The class tree in
docs/architecture.mdwas stale. Updated to match the current code:Step::RetryPolicy(DSL-level config; distinct fromStep::RetryExecutor).Pipeline::Result.Eval::AggregatedReport,Eval::BaselineDiff,Eval::PromptDiffComparator,Eval::EvalHistory,Eval::RetryOptimizer.Eval::TraitEvaluatorentry withEval::ExpectationEvaluator(the actual class name).OptimizeRakeTaskalongsideRakeTask.2. docs/ideas/ untracked
Already in
.gitignore(line 14), butdocs/ideas/10_eval_defaults.mdwas committed before the ignore rule landed, so git kept tracking it.git rm --cachedremoves it from version control without deleting it on disk. The other 11 idea files were already ignored.3. Showcase 11 Part A expected output in examples/README.md
examples/README.mdshowed only Part B (the "after" trace with retry success) but omitted Part A (schema-only pain point). Readers scrolling the examples index on GitHub saw the fix but not the problem the fix solves, which is the whole reason Part A exists in the script. Both sections now verified byte-identical toruby examples/11_fallback_showcase.rb.No version bump
Docs-only change; gem stays at 0.7.2.