Copy the existing course platform verbatim and establish its characterization baseline

[alexeygrigorev]

Parent epic: #5

Normative spec: 04 — Adoption baseline
Recorded owner decision: #13
Full production-like migration/reconciliation owner: #60

Scope

Adopt the existing course-management platform as code, not as a behavioral reimplementation. Begin from a clean checkout of exact source commit 98a235283904b4ef9ad29e196298540756cf1bcc and use the literal cp command to copy the tracked maintained source into this repository. Integrate only enough project settings/routing/dependencies for the unchanged copied code and characterization suite to run in the new scaffold.

• Record the source repository and exact commit before copying.
• Produce an allowlisted inventory of tracked source to copy, including Django apps, original migrations, tests, fixtures, templates/static assets, management commands, compatibility API, cadmin, and app-label metadata.
• Use literal cp from that clean checkout; do not manually read and recreate files or behavior.
• Exclude only documented local databases, secrets, virtual environments, caches/build/deploy outputs, unrelated notebooks, and temporary/untracked files.
• Record a copied-file manifest with source-relative destination, size, and checksum so omissions and later upstream comparison are mechanical.
• Preserve Django app labels, original migration files, and migration-state identity.
• Reconcile dependency/settings/URL integration mechanically and minimally, documenting every target-owned or modified-file integration patch as a separately reviewable per-file ledger entry.
• First make the unchanged copied characterization suite pass. The baseline expected approximately 760 Django test methods plus 48 end-to-end methods; record actual discovered and passing/skip/xfail counts.
• Copy and verify the complete original migration chain before any later structural change.
• Retain the original migrations in this adoption baseline. The owner permits later Django-supported squashing/collapse but does not require it now.
• If a future change proposes a squash, replacement, or removal, it must first prove original-chain and replacement fresh-install parity plus upgrade/import compatibility on a production-like source snapshot. Preserve schema/state/data operations, dependencies, permissions/content types, app labels, stable keys, and behavior. Originals remain until that evidence passes.
• When no production-like snapshot is available, the required safe result is exactly what this baseline does: keep every original migration and make no squash/removal claim.
• Leave full source→unified production-like data import, count/key reconciliation, reverse/forward migration-window rehearsal, and Course→Cohort migration evidence to Preserve legacy course HTML/API compatibility and rehearse full data migration #60.
• Document every excluded, patched, or intentionally unsupported source path/behavior with rationale and owner approval.

Non-goals

• No Course→Cohort/model refactor, family mapping/backfill, route redesign, authorization redesign, service-layer rewrite, or Studio redesign.
• No migration squash, replacement, or original-migration removal in this baseline.
• No claim that production-like snapshot upgrade/import parity was run when no snapshot was supplied; full migration/reconciliation belongs to Preserve legacy course HTML/API compatibility and rehearse full data migration #60.
• Do not hand-rewrite Django models or business logic as part of any later migration collapse.
• Do not opportunistically refactor scoring, submissions, peer review, leaderboards, certificates, calendars, API serializers, or cadmin behavior.
• Do not silently omit failing, inconvenient, deprecated-looking, or poorly documented business behavior. A known failure must remain visible as characterized behavior or an explicitly approved exception.
• Do not copy secrets, local data, deployment state, or untracked source-worktree changes.

Acceptance criteria

• A clean source checkout is pinned and verified at 98a235283904b4ef9ad29e196298540756cf1bcc before any copy.
• Implementation evidence shows the literal cp command copied the allowlisted source; the platform was not manually reconstructed from reading source files.
• All tracked maintained Django apps, original migrations, tests/fixtures, templates/static, management commands, compatibility APIs, cadmin behavior, and app labels are present unless an explicit reviewed exception names the exact path and behavior.
• A versioned provenance record contains source repository, commit, copy command/allowlist, and per-file manifest/checksums.
• Integration patches are separately reviewable and limited to dependency, settings, namespace, routing, filesystem, and test-environment compatibility; every modified copied file and target-owned compatibility shim has an exact per-file ledger entry and rationale, with no business logic silently changed.
• The unchanged copied characterization suite passes before Course→Cohort or other product/model work begins, with actual Django/E2E collected/pass/skip/xfail counts recorded.
• Every current course HTML route, compatibility API route, management command, and cadmin route is inventoried and smoke-covered from the copied code.
• The complete original migration chain is copied, retains its app labels/dependencies/state/data operations, and applies successfully from a fresh database with no migration drift.
• No squash/replacement/removal is proposed by this baseline. Every original migration remains because no production-like snapshot was supplied; this is the required safe fallback and no snapshot-parity claim is made.
• A documented/tested gate forbids any future squash/replacement/removal unless original-versus-replacement fresh install and production-like snapshot upgrade/import parity first pass with no unexplained schema/state/data/key/behavior differences. Originals must remain until that evidence passes.
• Full production-like source→unified import, count/key reconciliation, reverse/forward migration-window rehearsal, and Course→Cohort migration evidence are explicitly assigned to Preserve legacy course HTML/API compatibility and rehearse full data migration #60 and do not block closure of this literal-copy baseline.
• No Course→Cohort/model refactor or manual business-logic rewrite appears in this issue's diff.
• No source secret, local database, cache/build output, deployment state, or untracked local change is copied.

Test scenarios

Copy/provenance

1. Verify the source checkout is clean and at the pinned SHA; compare Git's tracked file list against the allowlist, destination manifest, and checksums.
2. Delete the adopted destination in an isolated test checkout and repeat the documented literal cp process; the resulting copied-file manifest is identical before integration patches.
3. Detect an intentionally omitted tracked test/template/migration/command/API/cadmin file and fail the provenance check.
4. Compare every modified copied file and every target-owned compatibility shim with the integration-patch ledger; a missing exact path or rationale fails this issue.

Django and migrations

1. Run the copied Django suite unchanged before structural work and record discovered/passed/skipped/xfail tests, including all adopted apps.
2. Create independent fresh databases through the complete unchanged original migration graph; verify expected app labels, numbered migrations, dependencies, schema state, and no migration drift.
3. Assert the adoption tree retains every copied original migration and contains no squash/replacement/removal.
4. Assert the migration safety gate rejects/remains unsatisfied for squash/removal when production-like snapshot parity evidence is absent.
5. If a later issue proposes a squash/removal, require both fresh-install equivalence and production-like snapshot upgrade/import reconciliation before originals may be removed. That conditional test is not invoked by this no-squash baseline.
6. Run the full source→unified production-like import, reconciliation, migration-window, and Course→Cohort rehearsal under Preserve legacy course HTML/API compatibility and rehearse full data migration #60 rather than fabricating evidence in Copy the existing course platform verbatim and establish its characterization baseline #30.

Playwright

1. Run all 48 copied E2E methods (or the actual discovered count) against the integrated copied application and record pass/skip/xfail/failure disposition.
2. Navigate representative public discovery, registration/login, dashboard/calendar, homework, project/peer review, leaderboard, certificate, API-backed, and cadmin workflows at desktop and mobile widths.
3. Capture screenshots for representative learner and cadmin pages and verify they render the copied behavior, not 404/debug/error pages.

Behavior inventory

1. Compare Django URL resolver output and API schemas with the source commit.
2. Run copied scoring, peer-assignment, certificate, communication/outbox, and management-command tests without altering expected values merely to make them pass.
3. For every explicit exception, demonstrate the source behavior, rationale, downstream replacement issue, and owner approval.

Dependencies

• Depends on Bootstrap the runnable Django/uv foundation and engineering workflow #1 for the runnable unified Django/uv scaffold.
• Decision: Record adoption—not rewrite—of the existing course platform #13 records the binding adoption decision.
• This baseline does not depend on a production-like snapshot or on Preserve legacy course HTML/API compatibility and rehearse full data migration #60 because it retains the full original migration chain and performs no squash/removal.
• Preserve legacy course HTML/API compatibility and rehearse full data migration #60 depends on this adopted baseline and owns full source→unified production-like import/reconciliation and later migration rehearsal.
• Blocks Course→Cohort/model and course workflow changes until the literal-copy characterization and integration-patch ledger pass.

[alexeygrigorev]

Software Engineer Report

Implemented the literal Course Management Platform adoption from clean detached source commit
98a235283904b4ef9ad29e196298540756cf1bcc. Changes remain uncommitted for independent tester and
PM review.

Changed surface

• Copied 768 tracked files with literal cp from the verified clean checkout: accounts, api,
  cadmin, course_management, courses, data, e2e, source templates (mounted at
  course_platform_templates), and operational scripts.
• Preserved original app labels and all numbered migrations: 10 accounts, 40 courses, and 5
  data; api and cadmin retain their source no-numbered-migration state.
• Integrated bounded dependencies, Django settings/middleware/templates, root routes, allauth,
  loginas, Unfold, Django Q, target Studio/admin API compatibility, and source-pinned Playwright
  1.58.
• Added checksum/provenance artifacts, an explicit copied-template patch record, a generated
  89-route/13-command behavior inventory, migration squash gate, verification record, and 5
  adoption-contract tests.
• Kept copied Python roots byte-frozen by excluding them from autoformat/static-style rewrites;
  target-owned integration shims remain explicitly linted/formatted/type-checked.

Verification

• uv run python scripts/verify_course_platform_adoption.py — 768 copied destinations verified
  against the clean pinned checkout and explicit integration-patch manifest.
• Unchanged source-owned Django characterization — 756/756 passed, 0 skipped/failed.
• make test — 786/786 passed, 0 skipped/failed (includes 5 new adoption-contract tests).
• Copied E2E suite — 48 collected; 45 passed, 1 skipped, 2 expected xfails, 0 failures/errors in
  138.64s against a fresh local database and ephemeral localhost Datamailer preference dependency.
  All provisioning, enrollment/impersonation, homework, project, API, dashboard, leaderboard,
  cadmin, helper, and fallback-cleanup checks passed.
• PLAYWRIGHT_BROWSERS_PATH=.tmp/playwright-browsers make test-playwright-core — 3/3 passed at
  desktop/mobile viewport coverage.
• Two independent fresh SQLite databases applied the complete original migration graph; both
  succeeded. make migrations-check reports no changes.
• make lint, make format-check, make typecheck, make django-check, and
  make deployment-check all pass.
• Generated inventory smoke coverage resolves all 89 adopted routes through the unified root
  URLconf and loads all 13 management commands.

Explicit dispositions and limitations

• The two declared E2E xfails are homework/project confirmation-email send-audit checks because no
  durable worker and transactional provider were run. The one skip is the copied teardown fallback
  after admin deletion was unavailable; the generated course was parked hidden. No copied E2E
  expectation was changed.
• Without a Datamailer preference service, two account-settings checks correctly reached HTTP 200
  but timed out on browser networkidle after the dependent preference request returned 503. An
  ephemeral local preference stub supplied the missing dependency and both unchanged checks passed;
  no stub or credential is present in the repository.
• No anonymized production-like source snapshot was supplied. Upgrade/import reconciliation,
  production count/key parity, and reverse/forward migration-window verification remain open.
  Consequently no squash was attempted and no original migration was removed.
• The adopted accounts.CustomUser retains the source AbstractUser username field and database
  behavior allowing duplicate emails, while allauth remains configured for email login. This issue
  does not perform an auth or authorization redesign.
• Representative screenshot capture/inspection is intentionally left to the independent tester
  stage required by _docs/PROCESS.md.

No commit, push, pull request, source secret, local database, production data, cache/build output,
deployment state, or untracked source-worktree change was included.

[alexeygrigorev]

Independent tester QA — FAIL

The literal copied code baseline is technically sound, but this gate fails on incomplete per-file integration-patch evidence. I independently unchecked only the integration-patch criterion. The production-like snapshot criterion remains separately unchecked.

Blocking finding

_docs/adoption/course-platform/integration-patches.md does not explicitly enumerate the two new target-owned compatibility shim files website/admin_api_urls.py and website/admin_api_views.py or give a per-file rationale. They preserve the scaffold admin-health route after the source api package replaces the old target API view, and the implementation is narrow, but the issue requires each non-literal integration addition to be separately reviewable. The general website/urls.py compatibility rationale is not an explicit ledger entry for these two files.

Required follow-up: add both exact paths and their rationale to the integration-patch ledger, then repeat tester review. No code change is requested by this finding.

Copy/provenance verified

• Pinned source checkout is clean (including untracked state) at exact SHA 98a235283904b4ef9ad29e196298540756cf1bcc.
• Source has 824 tracked files: 768 match the documented copy allowlist and 56 match the documented exclusions.
• copied-files.tsv has exactly 768 unique source/destination rows. Git-derived allowlist, recorded sizes/checksums, and destination bytes agree.
• uv run python scripts/verify_course_platform_adoption.py passes: 768 copied files verified.
• Literal cp commands, source path/SHA, allowlist, exclusions, and post-copy evidence are recorded.
• The one modified copied file, course_platform_templates/base.html, differs only by the documented canonical-link integration and matches integration-patched-files.tsv.
• Copied business files are checksum-identical. I found no Course→Cohort rename or manual scoring/submission/review/leaderboard/certificate/calendar/cadmin rewrite.
• No source secret, source database, source cache/build/deploy state, notebook, or untracked source file is in the manifest/candidate. Local QA outputs are ignored; a pre-existing ignored zero-byte root test.sqlite3 and ignored caches are not source copies or commit candidates, while this QA's databases/screenshots are under .tmp/.

Structure, inventory, and migrations

• Adoption contract: 5/5 passed (checksums, inventory freshness, resolver smoke, commands, labels/migrations).
• Inventory generator --check passes.
• Inventory is exactly 89 routes: 9 accounts, 29 compatibility API, 26 cadmin, and 25 public course routes.
• Exactly 13 adopted management commands load.
• Original numbered migrations remain: 10 accounts, 40 courses, 5 data; API/cadmin retain no numbered migrations.
• Two independent fresh isolated SQLite databases applied the complete graph successfully; all 10/40/5 originals report applied.
• makemigrations --check --dry-run reports no drift. No squash/replacement was introduced and no original migration was removed.
• Production-like snapshot upgrade/import parity, count/key reconciliation, and reverse/forward window remain not run because no anonymized snapshot was supplied. That acceptance checkbox correctly remains unchecked.

Automated results

• uv lock --check, uv sync --locked, git diff --check: pass.
• Ruff lint and format check: pass (102 files checked by format target).
• Targeted mypy: pass (50 source files).
• Django system check and deployment check: pass.
• Copied source-owned Django characterization, derived only from pinned source test paths: 756 found, 756 passed, 0 skipped, 0 failed on an isolated database.
• Full unified Django suite: 786 found, 786 passed, 0 skipped, 0 failed on an isolated database.
• Copied E2E collect-only: 48 exactly.
• Dependency-complete local E2E with documented local fake credentials and an ephemeral localhost Datamailer preference responder: 45 passed, 1 skipped, 2 xfailed, 0 failed in 142.69s.
  • Skip: admin delete unavailable; teardown parked the generated course hidden and the visible-data assertion passed.
  • Xfail: no homework confirmation send audit appeared within 60s because no transactional provider/worker was run.
  • Xfail: no project confirmation send audit appeared within 60s for the same documented dependency reason.
  • The ephemeral responder enabled contact preference loading and intentionally delivered no email or external side effect.
• Target foundation Playwright: 3 passed, 0 skipped, 0 failed.

Browser/screenshots

Stored and individually inspected 27 positive desktop/mobile captures under .tmp/qa-issue-30/screenshots/ for public course discovery, registration, login, authenticated course/certificate, learner dashboard, account settings, homework, project, peer review, leaderboard, API health/OpenAPI, course-admin list/detail, and cadmin homework/project submissions.

• All positive HTML/API navigations returned 200 at their expected final routes; there were no page exceptions or unexpected console errors.
• No inspected image contains a 404, Django debug/error page, empty page, or broken desktop/mobile composition.
• The protected OpenAPI endpoint correctly returned 401 without a token and 200 with the documented local fake token; the authenticated result was captured.
• Calendar is non-HTML and was verified directly: 200, text/calendar, 1,682 bytes, containing BEGIN:VCALENDAR.
• Datamailer email-audit rendering was dependency-limited exactly as represented by the two declared E2E xfails; no external credentials or production data were used.

Verdict

QA FAIL until the two compatibility shim paths are explicitly documented per file. Apart from that evidence defect, the current 768-file copy baseline is technically acceptable and its recorded Django/E2E/browser behavior reproduces cleanly. Full issue acceptance also remains blocked by the separately unresolved production-like snapshot criterion; do not squash/remove migrations or close the issue.

[alexeygrigorev]

Grooming clarification — snapshot evidence is conditional on migration collapse

The owner permitted migration squashing/collapse; it was never a requirement to perform it in this literal-copy issue.

No production-like anonymized source snapshot was supplied, and the implementation correctly retained every original migration without attempting a squash, replacement, or removal. #30 therefore must not pretend snapshot evidence exists, but it also must not remain blocked on evidence needed only before a destructive migration-chain replacement.

The issue now makes the invariant explicit:

• Copy the existing course platform verbatim and establish its characterization baseline #30 copies, verifies, freshly applies, and retains the complete original migration graph.
• Absence of a production-like snapshot requires retaining originals; that safe fallback is satisfied.
• Any future squash/replacement/removal is forbidden until both fresh-install equivalence and production-like snapshot upgrade/import parity pass with no unexplained schema/state/data/key/behavior difference.
• Full source→unified production-like import, count/key reconciliation, reverse/forward migration-window rehearsal, and Course→Cohort migration evidence remain owned by Preserve legacy course HTML/API compatibility and rehearse full data migration #60.

The production-like snapshot checkbox has been replaced with conditional, truthful criteria rather than marked as if a rehearsal ran. The independent tester's missing per-file shim-ledger entries remain the only unchecked #30 criterion; once fixed and re-verified, the literal-copy characterization baseline can close without weakening the no-delete/no-squash gate.

[alexeygrigorev]

Software Engineer Follow-up — Tester Evidence Fix

Addressed the tester's focused failure without changing application or business behavior:

• website/admin_api_urls.py is now explicitly named and classified as a target-owned
  compatibility shim preserving the scaffold's namespaced /api/v1/admin/health route alongside
  the adopted /api/ compatibility surface.
• website/admin_api_views.py is now explicitly named and classified as a target-owned
  compatibility shim preserving the scaffold's staff-gated, versioned health response without
  modifying copied CMP API behavior.
• Added target-owned-compatibility-shims.tsv with exact path, classification, per-file rationale,
  byte size, and SHA-256 for both files.
• Extended scripts/verify_course_platform_adoption.py to require exactly those two paths and
  verify classification, non-empty rationale, size, and checksum.
• Added a sixth adoption-contract test enforcing the same record in normal Django test runs.

Verification:

• Provenance verifier: 768 copied files plus 2 target-owned compatibility shims verified.
• Adoption contract: 6/6 passed.
• Generated 89-route/13-command inventory check passed.
• Unified Django suite: 787/787 passed.
• Lint, format check, targeted mypy, and git diff --check passed.

The production-like snapshot criterion remains unchecked. No migration was squashed or removed,
and changes remain uncommitted for tester re-review.