Skip to content

Add one evidence-driven debrief and a course-start passport #71

Description

@rosshd

Parent: #58
Depends on: #68, #69, #70.
Related: #59, #62, #63, #64.

Problem

The current placement asks complexity and follow-up questions for every completed implementation, even when the additional prose cannot change the course decision.
It then prints provisional rubric and profile details that are technically accurate but difficult for a learner to turn into a next action.

Goal

Use execution and communication evidence to select zero or one targeted debrief, then return an actionable course-start passport instead of a broad proficiency verdict or internal rubric dump.

Debrief policy

  • Start from durable clarification, plan, source snapshot, public/hidden runner outcome, and assistance evidence.
  • Ask no debrief when the available evidence already implies the same safe starting route under all recorded uncertainties.
  • Otherwise choose exactly one bounded question or editor microtask that can change the recommended first unit or next verification.
  • Initial deterministic probes may cover complexity, a failed edge class, approach tradeoff, or a bounded follow-up constraint.
  • Record why the probe was selected and which routing uncertainty it was intended to resolve.
  • Do not claim calibrated information gain until the problem bank and evidence model support it.
  • Model-generated wording may not alter the selected probe contract or score.

Course-start passport

Show, in this order:

  1. Recommended starting unit or activity.
  2. Evidence-backed strengths demonstrated in this attempt.
  3. One highest-priority practice area.
  4. One uncertainty that normal practice will verify.
  5. The immediate next action, defaulting to start the first lesson.

Keep raw lifecycle, rubric, evidence IDs, and detailed profile data behind explicit status/details commands.
Explain the no-mastery rule in learner language: placement chooses a starting route but marks no skill mastered.

Likely surfaces:

  • src/openlearn/interview_prep.py
  • src/openlearn/cli.py
  • interview skill and selection helpers
  • tests/test_interview_prep.py
  • tests/test_cli.py
  • tests/workflows/test_interview_journey.py
  • README and manual journey documentation

Acceptance criteria

  • A decisive implementation can finish with no debrief question.
  • An ambiguous implementation receives exactly one probe tied to a named routing uncertainty.
  • The same durable evidence deterministically selects the same baseline probe contract.
  • Model phrasing cannot change probe type, expected evidence, or scoring.
  • Skipped or unavailable implementation evidence does not trigger dependent complexity questions.
  • The passport names a starting activity, strengths, priority, uncertainty, and next action.
  • The default completion action begins the first lesson instead of bouncing to an unexplained menu.
  • The learner-facing result contains no raw rubric IDs, evidence counts, or storage vocabulary by default.
  • Placement remains provisional and grants no mastery.
  • Detailed status remains inspectable locally.
  • Tests cover clean pass, partial failure, skipped code, unsupported language, editor/runner failure, provider failure, and interrupted debrief.
  • make check passes.

Out of scope

  • A psychometrically calibrated adaptive test.
  • Multiple sequential follow-up questions during quick placement.
  • Full mock-interview postmortems.
  • Continuous plan changes from later practice evidence.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions