Skip to content

Roadmap

Cristiano Carvalho edited this page Apr 6, 2026 · 13 revisions

Aludel's roadmap follows the current GitHub milestones and open issue assignments. The aim is to strengthen the workbench in layers: first smooth the core user experience, then harden execution and architecture, then expand evaluation depth, analytics, and optimization.

image

Milestone Overview

The Cauldron Awakens
Due: April 13, 2026

Focus: immediate workflow cleanup and provider UX polish.

Open issues:

  • #59 extract suite editor workflow out of SuiteLive.Show
  • #58 apply prompt search and tag filters before pagination
  • #30 implement custom select dropdown component
  • #11 auto-populate model dropdown based on selected provider

Expected outcome:

  • cleaner suite and prompt management flows
  • less LiveView complexity in the editor surface
  • lower-friction provider configuration
  • a more polished first-use experience

Roots of the Ancient Grove

Focus: architecture, supervision, run execution, and platform reliability.

Open issues:

  • #61 architecture: introduce application-service boundaries and thin LiveViews
  • #60 move run execution orchestration into a supervised application service
  • #62 model explicit run execution states and transitions
  • #51 optimize LiveView run updates with selective fetching
  • #50 make LLM request concurrency configurable
  • #44 retry individual test cases
  • #31 default pricing with override support
  • #72 add Gemini provider support
  • #28 custom storage backends for test case documents

Expected outcome:

  • clearer application boundaries
  • supervised and explicit execution ownership
  • better UI feedback for long-running work
  • expanded core provider coverage
  • stronger production-readiness for pricing and document storage

The Transmutation Begins

Focus: richer evaluation fidelity and better result inspection.

Open issues:

  • #69 embedded app callback execution for high-fidelity evals
  • #43 deep comparison scoring for structured output assertions
  • #73 add copy actions for run and suite results
  • #77 export raw run and suite results as JSON
  • #75 support nullable metrics and callback metadata in result views

Expected outcome:

  • high-fidelity evals against real host-app execution paths
  • better scoring for structured outputs
  • easier debugging and result reuse
  • stronger result views for callback-mode and structured-output workflows

The Great Work is Done

Focus: first-class workflow evaluation.

Open issues:

  • #42 workflow evaluation support

Expected outcome:

  • multi-step workflow test cases
  • workflow-level cost and latency tracking
  • richer comparison of real-world LLM application behavior

The Knowing Deepens

Focus: decision-oriented analytics and regression visibility.

Open issues:

  • #78 rework dashboard stats around rolling windows and period deltas
  • #81 add outcome-weighted efficiency metrics
  • #80 add version-over-version deltas to prompt evolution
  • #79 add regression and stability signals to analytics

Expected outcome:

  • analytics that stay useful after long-term use
  • clearer tradeoff metrics tying quality to cost and latency
  • better visibility into regressions and instability
  • stronger prompt and provider decision support

What Was Hidden, Speaks

Focus: richer prompt analysis surfaces after analytics and workflow primitives are in place.

Open issues:

  • #76 export prompt evolution metrics
  • #74 add prompt version diff view

Expected outcome:

  • clearer prompt history analysis and comparison
  • exportable metrics for external analysis
  • better visibility into how prompt iterations change outcomes

The Sacred Patterns Revealed

Focus: optimization and recommendation systems built on top of mature evaluation and analytics.

Open issues:

  • #12 integrate GEPA for prompt evolution and optimization

Expected outcome:

  • prompt evolution intelligence
  • optimization-oriented workflows on top of trusted evaluation history
  • a clearer path from measurement to automated improvement

Unassigned Future Milestones

The following milestones exist but currently have no open issues assigned:

These are good candidates for:

  • polish and consolidation
  • platform scale features
  • advanced workflow orchestration
  • long-horizon optimization and recommendation systems

Strategic Summary

The roadmap now reads as a coherent progression:

  1. Refine the immediate UX and trim workflow-heavy LiveViews.
  2. Strengthen execution ownership and platform reliability.
  3. Improve evaluation fidelity, structured scoring, and result inspection.
  4. Expand into workflow evaluation.
  5. Make analytics decision-oriented rather than cumulative.
  6. Improve prompt analysis and comparison surfaces.
  7. Build optimization intelligence on top of mature evaluation primitives.

Clone this wiki locally