v1.1: AI-Engineering-Upgrade & Material-Pipeline#21
Merged
Conversation
…obust PDF worker The processing pipeline maintained detailed per-material state (extractionStatus/extractionError) that the UI never showed — a stuck or failed material looked identical to a ready one, and several failure modes were completely silent. This makes the whole pipeline observable and recoverable: - materials list shows pipeline badges (queued/extracting/indexing/ summarizing spinner, failed with error tooltip, skipped with reason) and polls while anything is still processing - new 'process again' action re-enqueues failed/skipped materials (processing is idempotent, so retries are safe) - skipped materials now record WHY (no transcription provider, no vision model, no extractable text) instead of silently showing nothing - a failed job enqueue marks the material failed instead of pretending the upload fully succeeded; summarization errors can no longer leave a material stuck on the 'summarizing' spinner forever - material viewer detects a missing stored file (DB row outliving the file, e.g. uploads lost before the volume fix) and explains it — previously the PDF viewer surfaced a cryptic pdf.js error - pdf.js worker is copied to public/ at build time and loaded from a stable same-origin URL — the bare-specifier new URL(...) pattern is not guaranteed to emit an asset under Turbopack, which would break ALL PDF rendering - /api/health now write-probes the storage backend and the admin page warns when storage is unwritable (the most likely cause of failing uploads) - upload errors are mapped to actionable messages (rate limit, network, server/storage) instead of raw statusText - materials page hints when no embedding model is configured (files were 'ready' but silently unsearchable) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrdkggYqhNCCZVHyJrdwBR
…obust PDF worker The processing pipeline maintained detailed per-material state (extractionStatus/extractionError) that the UI never showed — a stuck or failed material looked identical to a ready one, and several failure modes were completely silent. This makes the whole pipeline observable and recoverable: - materials list shows pipeline badges (queued/extracting/indexing/ summarizing spinner, failed with error tooltip, skipped with reason) and polls while anything is still processing - new 'process again' action re-enqueues failed/skipped materials (processing is idempotent, so retries are safe) - skipped materials now record WHY (no transcription provider, no vision model, no extractable text) instead of silently showing nothing - a failed job enqueue marks the material failed instead of pretending the upload fully succeeded; summarization errors can no longer leave a material stuck on the 'summarizing' spinner forever - material viewer detects a missing stored file (DB row outliving the file, e.g. uploads lost before the volume fix) and explains it — previously the PDF viewer surfaced a cryptic pdf.js error - pdf.js worker is copied to public/ at build time and loaded from a stable same-origin URL — the bare-specifier new URL(...) pattern is not guaranteed to emit an asset under Turbopack, which would break ALL PDF rendering - /api/health now write-probes the storage backend and the admin page warns when storage is unwritable (the most likely cause of failing uploads) - upload errors are mapped to actionable messages (rate limit, network, server/storage) instead of raw statusText - materials page hints when no embedding model is configured (files were 'ready' but silently unsearchable) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrdkggYqhNCCZVHyJrdwBR
…vider validation Core AI-engineering fixes from the v1.1 review: - re-embed backfill: changing the default embedding model used to silently orphan every existing chunk (vector search filters on the active model ref and matched nothing, with no signal and no recovery). A new reembed-materials job re-embeds stale materials (reusing stored text, resumable), removes old-model chunks only after success, rebuilds the ANN index at the end, auto-starts on model change and has an admin card with a stale-count warning and progress - generation parameters: every SDK call ran on raw provider defaults. Central params now set low temperature for content generation (cards/questions/outline/summaries/plans), temperature 0 for free-text grading (identical answers no longer flip between correct/incorrect) and OCR, an explicit chat temperature, and an output-token budget on the live generation path matching the batch path (prevents silent mid-JSON truncation -> schema failures) - grading uses an explicit rubric; card/question prompts get quality rubrics (atomic cards, plausible distractors, difficulty mix, evenly distributed correct positions) - full-text search switches from the 'simple' config to 'german' (stemming + stop words; the app's content is German-first) and from plainto_tsquery to websearch_to_tsquery (never throws, supports phrases); migration rebuilds the generated column + GIN index - retrieval noise cut: vector-only hits below a conservative cosine threshold are dropped before RRF fusion (nearest-neighbour always returns something, even when the corpus doesn't cover the query) - chat stream errors are categorized (auth / rate-limit / generic) and shown as translated messages instead of being swallowed; failed streams no longer persist empty assistant messages - provider/API-key validation: admin 'Test connection' button fires a one-token live request against the (unsaved) provider config, and BYOK keys are validated the same way before being stored - admin usage view gains a per-feature/per-model breakdown Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrdkggYqhNCCZVHyJrdwBR
…, voice input New AI features for v1.1: - clickable source citations in chat: search results carry an index and materialId, the model cites [n] inline, and deduped source chips below the answer deep-link to the material (persisted via tool parts) - stop button while streaming and one-click regenerate of the last answer; the server handles the AI SDK regenerate trigger by removing the trailing assistant rows (refused with 409 when an executed write tool is part of the answer — its side effect already exists) - AI-generated conversation titles after the first exchange (the 80-char truncation stays as instant placeholder) - tutor mode can now be switched mid-conversation (was fixed at creation) - 'chat with this document': aiConversation.materialId scopes retrieval to one material; entry button on the material viewer - weak-topics loop closed: the progress analysis returns a compact weak-topics list and offers one-click 'quiz/deck from my weak spots' - voice input in chat (MediaRecorder -> Whisper via configured OpenAI/Groq provider; hidden when unavailable) - 'Explain' on PDF highlights: extracts the highlighted region's text via the pdf.js text layer and shows an AI explanation in the annotation popover - optional LLM re-ranking of hybrid-search candidates (admin toggle, best-effort with RRF fallback) - user settings usage card now matches the enforced calendar-month window, shows utilization vs the monthly limit and a per-feature breakdown Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrdkggYqhNCCZVHyJrdwBR
veniplex
added
the
release-candidate
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v1.1 — AI-Engineering-Review, Fixes & Ausbau
Stacked PR auf #20 (
claude/app-review-functionality-usb0y6) — bitte #20 zuerst mergen; danach dieses PR gegenmainretargeten (GitHub macht das beim Merge der Base automatisch). Version ist auf 1.1.0 angehoben. Die Merge-Konflikte von #20 mit main (tus-Upload-Refactor aus #19) wurden im Zuge dieser Arbeit bereits aufgelöst — #20 ist wieder mergeable.Verifikation: Typecheck ✓, Lint ✓, 129 Tests ✓, Produktions-Build ✓.
Diagnose deiner gemeldeten Bugs (Upload / Öffnen / Verarbeitung)
Die Analyse fand keinen deterministischen Code-Defekt im Upload- und Öffnen-Pfad — der Code ist an den üblichen Fallstricken (Dateinamen-Encoding, Streaming, Proxy-Body-Limits) korrekt. Die wahrscheinlichsten Ursachen sind deployment-seitig, und genau dafür macht dieses PR sie sichtbar:
UPLOAD_DIR) ist im Container nicht beschreibbar (klassisches Docker-Bind-Mount-Ownership-Problem; der Server gibt dann 500 „Upload failed"). Prüfen:/api/healthliefert jetztstorage: up|down, und die Admin-Seite zeigt einen roten Warnbanner, wenn der Speicher nicht beschreibbar ist. Im Server-Log nach[upload] stream to disk failed(EACCES) suchen; im Containerls -laauf das gemountetedata/uploads.UPLOAD_DIR-Fix (Commita1bf02e) hochgeladen wurden, gingen bei Redeploys verloren; die DB-Zeilen zeigen ins Leere und die File-Route 404t. Prüfen: Netzwerk-Tab — 404 auf/api/materials/<id>/filebestätigt es. Der Viewer erklärt das jetzt verständlich („Datei fehlt auf dem Server — bitte neu hochladen") statt eines kryptischen pdf.js-Fehlers.public/kopiert und von einer stabilen URL geladen.Material-Verarbeitung: von unsichtbar zu beobachtbar
Der Kern deines „Verarbeitung wirkt unsauber"-Eindrucks: Die Pipeline pflegte intern präzise Status (
pending → extracting → embedding → summarizing → ready/failed/skipped) samt Fehlertexten — aber das UI hat nichts davon angezeigt. Jetzt:failed, neutral mit Begründung fürskipped), mit Auto-Refresh solange etwas läuftfailedstatt so zu tun, als wäre alles gut; hängende „summarizing"-Spinner sind unmöglich gemachtAI-Engineering-Review: kritische Befunde (gefixt)
'simple'-Config → „Vorlesungen" fand „Vorlesung" nicht. Jetzt'german'+websearch_to_tsquery(Migration baut Spalte + GIN-Index neu).Neue AI-Features
[n]-Zitate im Text, dedupierte Quellen-Chips mit Link zum Material unter der Antwort (persistiert, überleben Reload)Bewusst NICHT in v1.1 (dokumentierte Grenzen)
Migrationen
0036— FTS-Spalte auf'german'(Tabellen-Rewrite + GIN-Neuaufbau, läuft indb:migrate)0037—ai_conversation.material_id(Dokument-Chat)npm run db:migratewie üblich. Neue optionale Umgebung: keine.🤖 Generated with Claude Code
https://claude.ai/code/session_01UrdkggYqhNCCZVHyJrdwBR
Generated by Claude Code