Implement Theia Content Management Workflow

[JasonWarrenUK]

Implement Theia Content Management Workflow

Overview

This PR introduces Theia, a new workflow for uploading and managing previously generated course structures. Users can now upload JSON course files, view/edit them, and export to multiple formats (Markdown, HTML, JSON). This enables round-trip workflows: generate in Themis → export → re-upload → continue working.

Tip

No additional setup steps required after pulling. Theia is accessible from the home page.

Changes

Core Theia Implementation

New workflow for course content management with upload, validation, and export capabilities.

• New route: /theia - Main Theia page with course upload interface
• Upload component: CourseStructureUpload.svelte - File upload, drag-and-drop, validation feedback
• Validation: courseValidator.ts - Comprehensive Zod-based course structure validation
• Test data: Added partial and complete course JSON fixtures for testing

Export Enhancements

Extended Theia export service to support JSON format for round-trip workflows.

• Added JSON export format to ExportConfigModal.svelte
• Updated theiaService.ts to handle JSON exports (raw CourseData structure)
• Added informational UI explaining only JSON can be re-imported

Home Page Integration

Theia workflow card added to home page and promoted to primary workflow position.

• New Theia card with icon, description, and feature highlights
• Reordered cards: Theia → Themis → Tethys → Metis
• Expanded grid container width (900px → 1400px) for better desktop layout

Theming & Branding

Theia palette implementation with magenta/cyan colour scheme.

• Added Theia palette to palettes.css (magenta primary, cyan accent)
• Integrated palette detection in paletteLoader.ts and paletteTypes.ts
• Added Theia icon asset
• Updated palette reference documentation

Error Handling

Enhanced error types for upload validation failures.

• Added UploadValidationError to error.ts for file upload scenarios

Documentation

Comprehensive updates to all user and developer documentation:

Roadmaps & Status

• Added Theia milestone 4.1.2 (Course Structure Upload) to completed milestones
• Updated Theia roadmap status from PENDING to IN PROGRESS for upload features
• Documented Theia palette completion in Rhea roadmap
• Updated project README with course upload as Recent Win Feat/langchain flow #1

User Documentation

• README.md: Reordered workflows (Theia first), expanded features, added workflow steps
• About-Rhea.md: Added "Recently Completed" section with Theia highlights
• Getting-Started.md: No changes (still accurate)

Developer Documentation

• Technical-Overview.md: Renamed Theia section to "Content Manager & Exporter", detailed upload components
• docs/dev/status/README.md: Updated Theia status with JSON upload features
• palettes.jsonc: Already current (verified)

All documentation now emphasizes:

• Round-trip capability (export JSON → upload → continue)
• Theia's dual role as content manager + exporter
• JSON as the resumable format
• Five functional workflow modules

---

Summary

Think of this like adding a garage to your house after you've been parking your car in the driveway for months. You've been generating lovely courses with Themis and immediately exporting them to files that disappear into the void. Now Theia gives you a proper space to bring those courses back, tinker with them, check their oil, maybe export them in a different colour (format), and generally treat them like the valuable assets they are rather than one-and-done paperwork.

[claude]

Summary

This PR successfully implements Theia's course upload workflow, enabling users to re-import previously exported course structures. The implementation is well-architected with comprehensive validation, excellent error handling, and user-friendly UI components. Code quality is high with strong TypeScript typing and good separation of concerns. No major security issues identified, though some minor improvements around performance, validation edge cases, and testing would strengthen the implementation.

🎯 Code Quality & Best Practices

Excellent Patterns

✅ Strong TypeScript Usage

• Comprehensive type safety throughout (CourseStructureUpload.svelte:10-13, courseValidator.ts:3-7)
• Proper use of discriminated unions for upload states
• Type guards in validation logic (courseValidator.ts:17-19)

✅ Separation of Concerns

• Clean separation: UI component → validator utilities → error types
• Reusable validation functions that could be shared across workflows
• Event-driven communication between components using Svelte's createEventDispatcher

✅ Error Handling Architecture

• Typed error classes with structured error codes (error.ts:193-231)
• User-friendly error message mapping (courseValidator.ts:201-225)
• Clear distinction between retryable and non-retryable errors

✅ Accessibility

• Keyboard support for drop zone (CourseStructureUpload.svelte:146)
• Semantic HTML with proper roles
• Visual feedback for all interaction states

Minor Improvements

🔸 Component Size
CourseStructureUpload.svelte is 436 lines. Consider extracting:

• Success state display (lines 154-188) → UploadSuccess.svelte
• Error state display (lines 190-196) → UploadError.svelte
• Upload prompt (lines 198-202) → UploadPrompt.svelte

This would improve testability and maintainability.

🔸 Magic Numbers
Line 33: if (file.size > 10 * 1024 * 1024)
Consider extracting to a constant:

const MAX_UPLOAD_SIZE_MB = 10;
const MAX_UPLOAD_SIZE_BYTES = MAX_UPLOAD_SIZE_MB * 1024 * 1024;

🔸 Duplicate Validation Logic
courseValidator.ts:47-52 validates totalWeeks and daysPerWeek are numbers >= 1. This validation might exist elsewhere in the codebase (for Themis course creation). Consider sharing validation rules if they match business requirements across workflows.

🔸 Documentation
The theia-course-upload.md implementation plan is excellent (1129 lines!), but consider:

1. Moving the "how to use" sections to user-facing docs
2. Adding JSDoc comments to exported functions in courseValidator.ts for IDE intellisense

🐛 Potential Issues

High Priority

🔴 Missing NULL Checks in Deserialization
courseValidator.ts:176-195 - The deserializeUploadedCourse function doesn't validate that course.arcs exists before calling .map(). If validation fails but this function is still called, it could throw:

// Current:
arcs: course.arcs.map((arc: any) => ({ // Could throw if course.arcs is undefined

// Suggested:
arcs: (course.arcs || []).map((arc: any) => ({

While the validation function should catch this, defensive programming is valuable here.

🟡 Race Condition in Drag-and-Drop
CourseStructureUpload.svelte:93-105 - The dragCounter pattern for nested drag events is correct, but there's no cleanup on component unmount. If the component unmounts while dragging, the counter persists in closure. Consider:

onDestroy(() => {
  dragCounter = 0;
  isDragOver = false;
});

Medium Priority

🟡 Validation Warning vs Error Boundary
courseValidator.ts:39-41 - An empty arcs array is an error, not just a warning. However, the logic returns early on line 36 if arcs is not an array, so this check at line 39 is unreachable for the non-array case. The logic is sound, but the early return comment could be clearer:

return { valid: false, errors, warnings }; // Fatal - can't validate arcs structure

🟡 Module Data Validation
courseValidator.ts:132-136 - Modules marked complete without xmlContent only trigger a warning. Depending on business requirements, this might need to be an error to prevent broken workflows. Verify with product requirements.

🟡 Date Parsing Without Validation
courseValidator.ts:179-180 - new Date(course.createdAt) can create invalid Date objects if the string is malformed. Consider:

const parseDate = (dateString: string | undefined): Date => {
  if (!dateString) return new Date();
  const date = new Date(dateString);
  return isNaN(date.getTime()) ? new Date() : date;
};

createdAt: parseDate(course.createdAt),
updatedAt: parseDate(course.updatedAt),

Low Priority

🟢 Console.error Side Effect
CourseStructureUpload.svelte:63 - console.error during error handling is good for debugging, but consider using a proper logging service in production or at least checking dev environment.

⚡ Performance Considerations

Good Performance Characteristics

✅ Lazy Loading

• Validation only runs on upload, not on initial render
• No expensive computations in reactive statements

✅ Efficient File Handling

• File size check (line 32-37) happens before reading file contents
• Single-pass validation in validateCourseStructure

Optimization Opportunities

🔸 Large Course Handling
courseValidator.ts:76-150 - The validation uses forEach loops to validate potentially large course structures. For very large courses (e.g., 50 arcs with 20 modules each = 1000 modules), this could cause UI lag.

Consider:

1. Streaming validation: Validate in chunks and show progress
2. Web Worker: Offload validation to a worker thread
3. Early termination: Stop after N errors (e.g., 50) to avoid processing malformed data

Implementation idea:

export function validateCourseStructure(
  course: any, 
  options: { maxErrors?: number } = {}
): CourseValidationResult {
  const maxErrors = options.maxErrors ?? Infinity;
  // ... add check in loops:
  if (errors.length >= maxErrors) {
    errors.push('Too many validation errors - stopping validation');
    return { valid: false, errors, warnings };
  }
}

🔸 Validation Warnings Display
CourseStructureUpload.svelte:167-176 - If there are many warnings (e.g., 100+), the UI might become cluttered. Consider:

• Showing first 10 warnings with "Show all (N more)" link
• Grouping warnings by category
• Making the warnings section scrollable with max-height

🔸 Module Count Calculation
CourseStructureUpload.svelte:160-161 - The .reduce() is called on every render when in success state. For large courses, memoize this:

$: totalModules = uploadedCourse?.arcs.reduce((sum, arc) => sum + arc.modules.length, 0) ?? 0;

Then use: <span>{totalModules} modules</span>

File Upload Performance

✅ Good: 10MB file size limit is reasonable
🔸 Consider: For very large JSON files (even under 10MB), parsing with JSON.parse() is synchronous and blocks the main thread. For production at scale, consider:

// Stream-based parsing for large files (optional future enhancement)
const response = new Response(await file.arrayBuffer());
const reader = response.body.getReader();
// Use incremental JSON parser like 'partial-json' library

🔒 Security Concerns

Good Security Practices

✅ Input Validation

• Comprehensive validation before processing data
• File type checking (CourseStructureUpload.svelte:25)
• File size limits to prevent DoS via large uploads

✅ No Dangerous Operations

• No eval or dynamic code execution
• No direct DOM manipulation with innerHTML
• No external resource loading from user input

Recommendations

🟡 File Extension Check Insufficient
CourseStructureUpload.svelte:25 - Checking only .endsWith('.json') can be bypassed (e.g., malicious.exe.json). While this isn't critical since you're only parsing JSON, consider also checking MIME type:

if (!file.name.endsWith('.json') || file.type !== 'application/json') {
  error = 'Please upload a valid JSON file';
  // ...
}

🟡 JSON Parsing Error Exposure
CourseStructureUpload.svelte:62-66 - Catching errors and showing them to users is good UX, but be careful not to expose internal system details. The getUserFriendlyError function handles this well, but ensure err.message doesn't contain sensitive information in production.

Current approach is fine, but for extra safety:

error = err instanceof Error ? getUserFriendlyError(err) : 'An unexpected error occurred';

🟢 XSS Protection
Svelte automatically escapes content, so displaying uploadedCourse.title (line 156) is safe. Just verify that if you ever use {@html}, the content is sanitized.

🟢 CSRF Not Applicable
Since this is file upload from local system (not a POST to server), CSRF isn't a concern here.

🟢 localStorage Security
The course data is stored in localStorage via Themis stores (line 21-23). Ensure sensitive data (if any exists in course structures) is not included, or consider encryption for sensitive fields in future iterations.

✅ Test Coverage

Existing Test Data

✅ Good Test Fixtures

• valid-complete-course.json - Comprehensive valid case with mixed module statuses
• partial-course.json - Tests incomplete courses (based on PR description)

Missing Test Coverage

The implementation plan (theia-course-upload.md) includes excellent manual test scenarios, but automated tests are missing. Recommendations:

🔴 Unit Tests Needed

courseValidator.ts:

1. validateCourseStructure - Critical function, needs tests for:

   • Valid complete course → { valid: true, errors: [], warnings: [] }
   • Missing required fields → specific error messages
   • Empty arcs array → error
   • Module weeks mismatch → warning
   • Invalid date formats
   • Edge cases: null values, undefined properties, wrong types

2. deserializeUploadedCourse - Tests for:

   • Valid date strings → Date objects
   • Invalid date strings → fallback to current date
   • Missing dates → fallback to current date
   • Nested module data date conversion

3. getUserFriendlyError - Tests for:

   • Each error type keyword mapping
   • Generic errors → user-friendly fallback

Example test structure:

// courseValidator.test.ts
import { describe, it, expect } from 'vitest';
import { validateCourseStructure } from './courseValidator';

describe('validateCourseStructure', () => {
  it('validates a complete valid course', () => {
    const validCourse = { /* ... */ };
    const result = validateCourseStructure(validCourse);
    expect(result.valid).toBe(true);
    expect(result.errors).toHaveLength(0);
  });

  it('returns error for missing course.id', () => {
    const invalidCourse = { title: 'Test' };
    const result = validateCourseStructure(invalidCourse);
    expect(result.valid).toBe(false);
    expect(result.errors).toContain('Missing or invalid course.id');
  });

  // ... more tests
});

🟡 Component Tests Needed

CourseStructureUpload.svelte:

1. File selection flows
2. Drag-and-drop interactions
3. Error state rendering
4. Success state rendering with warnings
5. Event dispatching (courseUploaded, uploadError)

🟡 Integration Tests

src/routes/theia/+page.svelte:

1. Upload → store update → navigation flow
2. Error query parameter handling
3. Round-trip: export from Themis → upload to Theia → verify data preservation

🟢 E2E Test Scenarios (for future consideration)

Based on implementation plan:

1. Complete round-trip workflow (Themis → Export → Theia → Upload → Continue)
2. Partial course upload and completion
3. Large course upload (performance)
4. Invalid file upload rejection

Test File Organization

Suggested structure:

src/lib/utils/theia/
  ├── courseValidator.ts
  ├── courseValidator.test.ts
  └── __tests__/
      └── fixtures/
          ├── valid-courses/
          ├── invalid-courses/
          └── edge-cases/

Coverage Goals

Aim for:

• Validator functions: 95%+ coverage (critical path)
• Error handling: 90%+ coverage
• UI components: 70%+ coverage (focusing on logic paths, not styling)

---

Additional Observations

🌟 Strengths

1. Comprehensive implementation plan - The 1129-line markdown doc shows excellent planning
2. Consistent with codebase patterns - Follows established Metis/Themis patterns
3. Palette integration - Theia branding properly integrated (paletteLoader.ts:80-87)
4. User experience - Drag-and-drop, clear feedback, helpful error messages
5. Future-proof - Design considers round-trip workflows and extensibility

📝 Minor Nits

1. README.md - Changes are mostly whitespace cleanup (good!), but verify the closing </details> tag on line 253 is intentional
2. Unused imports - Verify ModuleSlot import in courseValidator.ts:1 is used (seems unused)
3. Consistency - normalise vs normalize - stick to British English as per CLAUDE.md ✅

🚀 Future Enhancements (out of scope for this PR)

1. Validation schema with Zod - Consider migrating to Zod for runtime validation (aligns with project's use of Zod in apiValidator.ts)
2. Progress indication - For large file uploads
3. Bulk upload - Multiple courses at once
4. Upload history - Track previously uploaded files

---

Recommendation

APPROVE ✅

This is a solid implementation that successfully delivers the Theia upload workflow. The code is well-structured, maintainable, and follows project conventions. While there are opportunities for improvement (primarily around testing and minor edge cases), none are blocking issues. The PR delivers significant value by enabling course workflow resumption.

Before Merging (Optional)

Consider addressing:

1. Add unit tests for courseValidator.ts (high value, low effort)
2. Fix null check in deserializeUploadedCourse (courseValidator.ts:181)
3. Add onDestroy cleanup for drag state

Great work! 🎉

[claude]

Summary

This PR successfully implements the Theia content management workflow, adding JSON-based course upload, validation, and round-trip export capabilities. The implementation is clean, well-structured, and follows established project patterns. A few minor improvements around error handling, accessibility, and validation edge cases would strengthen the implementation.

---

🎯 Code Quality & Best Practices

Strengths

• Excellent architectural consistency - follows established project patterns with dedicated validators, type-safe error classes, and reusable components
• Strong TypeScript usage with proper type safety (CourseUploadError discriminated union, typed event dispatchers)
• Hierarchical validation from course → arcs → modules with contextual error messages
• Clear separation between errors and warnings

Areas for Improvement

• Consider validating ID format consistency and checking for duplicate IDs across arcs/modules
• Error messages could include suggestions for common fixes
• British English consistency in documentation (per CLAUDE.md requirements)

---

🐛 Potential Issues

Medium Priority

File Upload Race Condition (src/lib/components/theia/CourseStructureUpload.svelte:78-86)

• Issue: No check for uploadState - user could drop multiple files while processing
• Fix: Add guard at start of handleDrop: if (uploadState === 'uploading') return;

JSON.parse Error Handling (src/lib/components/theia/CourseStructureUpload.svelte:45-46)

• Issue: JSON.parse errors caught but not distinguished from validation errors
• Fix: Wrap parse in try-catch to provide specific "malformed JSON" message

Date Deserialization (src/lib/utils/theia/courseValidator.ts:179-180)

• Issue: new Date() creates current timestamp when dates missing, potentially misleading
• Fix: Consider throwing a warning or using sentinel value

Low Priority

• Test data includes mock XML content - consider adding fixture with actual valid XML
• Workflow step hardcoded to 4 - consider storing "uploadedFromTheia" flag for refresh handling

---

⚡ Performance Considerations

Strengths

• 10MB file size limit prevents memory issues
• Async file reading appropriate for JSON
• Minimal re-renders with proper Svelte reactivity

Suggestions

• For 50+ module courses, consider debouncing drag events
• Stats calculation is O(n) - could memoize using Svelte {@const}
• Consider releasing uploadedCourse reference on unmount

---

🔒 Security Concerns

Strengths

• File extension check, size limit prevents DoS
• Type validation before processing
• Svelte auto-escapes displayed content (safe)

Suggestions

• Add MIME type check alongside extension: !file.type.match(/application\/json/)
• Consider max depth check in validator for deeply nested objects
• Note: Avoid {@html} for uploaded content without sanitization

---

✅ Test Coverage

Excellent Test Fixtures Provided

• valid-complete-course.json - comprehensive happy path
• partial-course.json - incomplete workflow scenarios

Recommended Additions

1. Unit tests for courseValidator.ts (edge cases, boundaries, invalid dates)
2. Component tests for CourseStructureUpload.svelte (file drop, state transitions, error recovery)
3. Integration tests for upload → Themis flow and critical round-trip workflow

Add tests in:

• src/lib/utils/theia/courseValidator.test.ts
• src/lib/components/theia/CourseStructureUpload.test.ts

---

Minor Suggestions

1. Accessibility: Add aria-label to drop zone for screen readers
2. Home Page: Max-width 1400px might cause scroll on 1366px screens - consider min(1400px, 95vw)
3. Documentation: Implementation plan is exceptionally detailed - consider archiving completed sections
4. Palette: Verify Theia palette loads when navigating directly to /theia

---

Summary Recommendation

✅ Approve with minor follow-ups

This is a well-executed feature that successfully delivers the stated round-trip workflow capability. The suggested improvements are non-blocking and can be addressed in follow-up PRs if needed.

Priority follow-ups:

1. Add JSON.parse error handling distinction (5 min fix)
2. Add upload state guard to prevent race condition (2 min fix)
3. Consider basic unit tests for validator (30-60 min)

Great work on maintaining architectural consistency and providing comprehensive documentation! 🎉