-
Notifications
You must be signed in to change notification settings - Fork 1
Roadmap
Aludel's roadmap follows the current GitHub milestones and open issue assignments. The aim is to strengthen the workbench in layers: first smooth the core user experience, then harden execution and architecture, then expand evaluation depth and optimization.
The Cauldron Awakens
Due: April 13, 2026
Focus: immediate workflow cleanup and provider UX polish.
Open issues:
-
#59 extract suite editor workflow out of
SuiteLive.Show - #58 apply prompt search and tag filters before pagination
- #30 implement custom select dropdown component
- #11 auto-populate model dropdown based on selected provider
Expected outcome:
- cleaner suite and prompt management flows
- less LiveView complexity in the editor surface
- lower-friction provider configuration
- a more polished first-use experience
Roots of the Ancient Grove
Focus: architecture, supervision, run execution, and platform reliability.
Open issues:
- #61 architecture: introduce application-service boundaries and thin LiveViews
- #60 move run execution orchestration into a supervised application service
- #62 model explicit run execution states and transitions
- #51 optimize LiveView run updates with selective fetching
- #50 make LLM request concurrency configurable
- #44 retry individual test cases
- #31 default pricing with override support
- #72 add Gemini provider support
- #28 custom storage backends for test case documents
Expected outcome:
- clearer application boundaries
- supervised and explicit execution ownership
- better UI feedback for long-running work
- expanded core provider coverage
- stronger production-readiness for pricing and document storage
The Transmutation Begins
Focus: richer evaluation fidelity.
Open issues:
- #69 embedded app callback execution for high-fidelity evals
- #43 deep comparison scoring for structured output assertions
Expected outcome:
- high-fidelity evals against real host-app execution paths
- better scoring for structured outputs
- stronger signal for prompt and provider comparison
The Great Work is Done
Focus: first-class workflow evaluation.
Open issues:
- #42 workflow evaluation support
Expected outcome:
- multi-step workflow test cases
- workflow-level cost and latency tracking
- richer comparison of real-world LLM application behavior
What Was Hidden, Speaks
Focus: optimization and prompt intelligence after workflow/eval primitives are in place.
Open issues:
- #12 integrate GEPA for prompt evolution and optimization
Expected outcome:
- prompt evolution intelligence
- optimization-oriented workflows on top of existing evaluation history
- a clearer path from testing to systematic improvement
The following milestones exist but currently have no open issues assigned:
These are good candidates for:
- polish and consolidation
- platform scale features
- advanced workflow orchestration
- long-horizon optimization and recommendation systems
The roadmap now reads as a coherent progression:
- Refine the immediate UX and trim workflow-heavy LiveViews.
- Strengthen execution ownership and platform reliability.
- Improve evaluation fidelity and scoring.
- Expand into workflow evaluation.
- Build optimization intelligence on top of stronger evaluation primitives.
- Prompts
- Providers
- Runs and Execution
- Evaluation Suites
- Regex Assertions
- Metric Context
- Evaluator Execution Details
- Rubric Judges
- Judge Catalog
- Repeated Sampling
- Quality Policies
- ExUnit Evaluations
- File-Based Suites
- Evaluation Reporters
- Datasets
- Red-Team Datasets
- Generated Red-Team Cases
- Analytics and Prompt Evolution
- Exports and CI
- Documents and Storage
- Embedding and Access