Skip to content

Improve scene training conversation experience - #28

Merged
yydounai1234 merged 1 commit into
mainfrom
codex/improve-simulation-experience
Jul 31, 2026
Merged

Improve scene training conversation experience#28
yydounai1234 merged 1 commit into
mainfrom
codex/improve-simulation-experience

Conversation

@HansonL622

@HansonL622 HansonL622 commented Jul 31, 2026

Copy link
Copy Markdown

What changed

  • Improve custom-scene realtime conversation responsiveness by letting the teacher respond while state analysis and turn evaluation continue in the background.
  • Shorten end-of-speech detection and reduce the state extractor context window.
  • Add ordered background state processing to prevent out-of-order turn updates.
  • Fix automatic conversation completion so the closing response finishes and buffered WebRTC audio drains before disconnecting.
  • Keep the result modal on the completed simulation screen instead of falling back to the sentence-reading screen, preventing pronunciation audio from playing behind the report.
  • Add automatic transcript follow behavior for the latest dialogue line.
  • Add fixed local teacher preview audio assets and a reproducible generation script.
  • Allow local frontend origins to connect to the session WebSocket during development.

Why

The previous flow synchronously waited for state extraction before requesting each teacher response, which introduced noticeable dead time. Automatic completion also closed the peer connection as soon as response generation finished, before the browser had necessarily played the final buffered audio. Result routing could retain the sentence-reading stage, causing its pronunciation audio to play behind the report modal.

Impact

Scene practice now responds sooner, preserves the full final utterance, follows new subtitles, shows the report over the ended simulation, and serves teacher previews without a runtime TTS request.

Current status

  • Rebased cleanly onto the latest main (bcfa0ac).
  • Current PR head: 637101c.
  • No rebase conflicts or uncommitted local changes.
  • Frontend and backend validation completed successfully after the rebase.

Validation

  • npm ci
  • npm run build
  • npm run check:routes — 32 assertions passed
  • npm run check:realtime-events
  • ./mvnw --batch-mode --no-transfer-progress clean verify — 186 tests passed, 0 failures/errors
  • Backend packaging and JaCoCo report generation succeeded
  • Flyway baseline/migration verified against the existing local PostgreSQL database
  • Frontend and backend local HTTP responses verified

Docker is not installed in the current local environment, so the Testcontainers PostgreSQL/Redis integration suite was not run locally.

@HansonL622
HansonL622 marked this pull request as ready for review July 31, 2026 08:19

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

One high-confidence authorization/state-isolation issue is called out inline.

Verification: ./mvnw test -q passed; the route and realtime contract checks passed. The frontend Vite build could not run because this checkout has no usable local Vite installation.

Findings without inline locations

  • backend/unispeaking-server/src/main/java/com/unispeaking/controller/CustomSceneController.java:78: The flow is created without verifying that the authenticated user owns request.sceneId(), and SceneFlowServiceImpl stores flows in a process-wide map keyed only by sceneId. Any authenticated user who obtains another scene ID can therefore overwrite that scene's current stage (and read its learning content through /flows/{sceneId}/content), disrupting the owner's training and crossing the scene authorization boundary. Bind the flow to the user/session and enforce ownership on create, advance, content, and complete operations.

@HansonL622
HansonL622 force-pushed the codex/improve-simulation-experience branch from 8ad0d89 to 637101c Compare July 31, 2026 10:19

@yydounai1234 yydounai1234 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yydounai1234
yydounai1234 merged commit 14cb784 into main Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants