Skip to content

Staging#33

Merged
suguanYang merged 110 commits into
mainfrom
staging
May 10, 2026
Merged

Staging#33
suguanYang merged 110 commits into
mainfrom
staging

Conversation

@suguanYang
Copy link
Copy Markdown
Contributor

No description provided.

suguanYang and others added 30 commits May 6, 2026 10:53
Wires up the server-side auth contract Pi pinned down with the Dashboard
team: Notebook reads the shared Domain=.knowhere.ai session cookie, calls
users.getCurrentUser oRPC, and lifts {id, email, name} off body.json.user.
No JWT decode, no local user table, no separate auth UI. Anonymous
requests redirect to DASHBOARD_LOGIN_URL with a returnTo that points back
at NOTEBOOK_PUBLIC_URL.

Backing store is Neon Postgres via Drizzle (portable Postgres; no
Neon-only features). Only workspaces ships in this PR; sources /
chat_threads / chat_messages land in PR-B per the plan.

Implementation:
- src/lib/auth.ts: getCurrentUser forwards the Cookie header, extractUser
  tolerates shape drift, hasSessionCookie is the cheap Edge-runtime probe,
  requireUser throws the redirect.
- src/lib/workspace.ts: ensureWorkspace is idempotent on the user_id
  unique index; concurrent first-calls race safely to the same row.
- src/lib/schema.ts + drizzle/0000_perfect_stellaris.sql: workspaces
  table with user_id unique, deterministic namespace, btree index.
- src/lib/db.ts: neon-http driver, server-only guarded.
- src/proxy.ts: renamed from middleware.ts (Next.js 16 convention);
  short-circuits anonymous requests to /login without the DB roundtrip.
- src/app/page.tsx: server component that gates on requireUser and
  passes identity + workspace down to the client shell.
- .env.local.example: documents DATABASE_URL, DASHBOARD_SESSION_URL,
  DASHBOARD_LOGIN_URL, NOTEBOOK_PUBLIC_URL, SESSION_COOKIE_NAMES.

Tests (20 passing):
- extractUser against malformed / partial / extra-field envelopes
- sessionCookieNames env override + fallback
- getCurrentUser: no cookie = no roundtrip, exact Cookie forwarding,
  503 / non-JSON / network-error / body.json.user=null all map to null,
  missing DASHBOARD_SESSION_URL throws
- ensureWorkspace: warm path, cold path with generated namespace,
  concurrent first-call idempotency

Dashboard production dep tracked in Ontos-AI/knowhere-dashboard#3
(callback URL sanitization); does not block this PR.
Two contract corrections raised by @pi in review:

1. Redirect parameter: Dashboard already reads `callbackURL`, not
   `returnTo`. Notebook now sends `callbackURL` everywhere (auth.ts,
   proxy.ts, inline docs, login preview copy) so post-login lands on
   Notebook instead of Dashboard's default flow.

2. Cookie domain: for the shared-cookie design to work the browser needs
   a common parent domain. Dashboard is at knowhereto.ai, so Notebook
   aligns to notebook.knowhereto.ai (shared parent `.knowhereto.ai`).
   Local dev mirrors this with /etc/hosts entries for
   dashboard.local.knowhereto.ai and notebook.local.knowhereto.ai.

Dashboard still needs (tracked in knowhere-dashboard#3):
- add notebook.knowhereto.ai to the callback allowlist
- enable Better Auth advanced.crossSubDomainCookies with
  domain="knowhereto.ai" so the session cookie is actually shared

No code logic change; only the parameter name and the documented hosts
shift. All 20 unit tests still pass.
@pi hit 405 METHOD_NOT_SUPPORTED during Chrome MCP E2E. Dashboard's oRPC
transport expects POST with a JSON body, even for procedures that take no
input. GET was my misreading of the contract.

Change:
- src/lib/auth.ts: `getCurrentUser` now sends
    POST <DASHBOARD_SESSION_URL>
    content-type: application/json
    cookie: <incoming>
    body: `{}`
- src/lib/auth.test.ts: assertion renamed and tightened to verify
  method=POST, body="{}", content-type header, and cookie forwarding.

Response shape is unchanged (`{ json: { user } }`), so `extractUser` and
the rest of the call-site logic need no edit. 20 tests still pass.

Keeps the fix on the Notebook side per Pi's recommendation — don't add a
GET wrapper in Dashboard that would drift from the oRPC contract.
Neon's HTTP driver only speaks Neon's websocket-over-HTTP dialect, so
local dev against a Docker Postgres container fails at module load with
a generic "can only connect to remote Neon / Vercel / Supabase
instances" warning followed by a hang.

Fix: pick the Drizzle driver by env.

  DATABASE_DRIVER=neon   (default) — @neondatabase/serverless
  DATABASE_DRIVER=pg              — postgres-js

Prod on Vercel stays on the default (Neon via the Marketplace
integration). Local dev sets DATABASE_DRIVER=pg and points DATABASE_URL
at whatever Postgres is reachable (the shared e2e container at
localhost:55432 on this machine). The Drizzle schema is the same on
both paths — switching to AWS Aurora Postgres later is also just a
DATABASE_DRIVER + DATABASE_URL swap, no code change.

No test changes needed — existing unit tests mock `@/lib/db` directly
so they don't exercise the driver selection.

Verified: pnpm test (20 passing), pnpm build (clean).

Signed-off-by: suguanyang <wangbinqi77@gmail.com>
Signed-off-by: suguanyang <wangbinqi77@gmail.com>
PR-A (N-001): Dashboard-backed auth gate and workspace bootstrap
Adds the remaining three tables that back the MVP's persistence layer.
Per the persistence rule (@suguan + technical plan):

  - Postgres stores only metadata, status, Knowhere document IDs, and
    chat transcripts.
  - No file blobs. Uploads stream straight to Knowhere via a /tmp
    staging file and are deleted afterward.
  - No chunk copies. Chunks are fetched on demand from Knowhere's
    chunks API.
  - citations on assistant messages are a JSONB snapshot of the
    retrieval-result view shape — NOT a stored copy of chunk content.

Soft delete everywhere (per @pi's PR-B review criteria):
  - sources.deleted_at, chat_threads.deleted_at, nullable timestamps
  - reads filter IS NULL; hard delete is reserved for retention sweeps
  - partial index on sources (workspace_id, created_at DESC) WHERE
    deleted_at IS NULL keeps the sidebar hot path lean

New helpers in src/lib/workspace.ts (workspace-scoped mutations only,
so no accidental cross-tenant leaks):

  - findSourceInWorkspace / findChatThreadInWorkspace
  - softDeleteSource / softDeleteChatThread
  - appendMessageToThread (bumps thread.updated_at via DB now())

Tests (20 unit + 7 integration, all green):

  - Unit tests continue to mock the Drizzle client for ensureWorkspace
    contract assertions.
  - New workspace.integration.test.ts runs against a real Postgres.
    Skipped automatically when TEST_DATABASE_URL is unset, so CI stays
    stable. `pnpm test:integration` is the one-liner that runs them
    against the local Docker container.
  - Integration coverage: cross-workspace scoping rejects wrong owner,
    soft-deleted rows disappear from reads, partial index predicate
    matches the sidebar's hot-path filter, appendMessageToThread
    monotonically bumps updated_at, soft-deleted threads don't cascade
    their messages.

DB type shape:
  - src/lib/db.ts now types `db` as `NeonHttpDatabase<Schema>` (the
    production driver). postgres-js implements the same Drizzle query
    builder API, so the local/Aurora path casts at the boundary and
    call sites get consistent types regardless.

Verified: pnpm lint, pnpm build, pnpm test (20 passing + 7 skipped
without TEST_DATABASE_URL), pnpm test:integration (7 passing against
the local Docker Postgres).

Signed-off-by: suguanyang <wangbinqi77@gmail.com>
Merge PR-B into the MVP integration branch after Pi verification. Verified lint, unit tests, integration tests, db push, build, author metadata, and citation metadata-only persistence fix.
PR-C (N-002/N-003): source upload and status list
PR-D (N-004): parsed content chunks viewer
PATCH /api/sources/[sourceId] archives the document on Knowhere then
soft-deletes the local source row. The UI shows a confirm dialog before
proceeding.
Replace raw fetch with @effect/platform HttpClient (retry, timeout),
manual validation with Schema, and singleton factories with
Context.Tag + Layer for testability. Set up Effect Language Service
and strict tsconfig for the project.
- db.ts: Context.Tag DbClient service + dbLayer for testability
- temp-files.ts: Effect.acquireRelease for scoped temp file lifecycle
- source-upload.ts: scoped temp file via Effect.scoped, guaranteed cleanup
- workspace.ts: transaction for appendMessageToThread (2 writes)
- source-view.ts: Schema.Literal for status validation
- sources/[sourceId]/route.ts: Schema.Struct for body validation
- actions.ts: migrate from getKnowhereClient() to KnowhereClient service
- chat-service.ts: HandleChatTurnResult replaced with Either<Either, ChatTurnValue, ChatTurnError>
- chat/route.ts: Either.match replaces ok/error branch checks
- chat-service.test.ts: updated assertions to Either.isRight/isLeft
Remove unused Effect imports from db.ts and knowhere.ts, replace require()
with ESM import in source-reconcile.test.ts, add null guard for messages in
workspace.integration.test.ts, and cast test components to fix
React.createElement overload mismatches under React 19 types.
…rwarding

Eliminates the shared KNOWHERE_API_KEY env var requirement by
forwarding the Dashboard session cookie to Knowhere's key creation
endpoint on first login. Failed keys (401/403) are detected and can
be recreated from the UI without admin intervention.
### Chunk list scrolling
@base-ui/react ScrollArea Root didn't enforce overflow-hidden or min-h-0,
so when flex-1 was passed, the viewport grew with content and the page
scrolled instead of the panel. Adding overflow-hidden + min-h-0 to the
Root creates a proper scroll containment context.

### Optimistic chat message rendering
Previously handleChatSend waited for /api/chat before appending any
message. Now:
1. User message appended immediately with a temp id ("pending-{ts}")
2. isSending blocks the composer during the roundtrip
3. On success: remove temp user, append only assistant-role messages
   from the server (no duplicate user turn)
4. On failure: keep the optimistic user visible alongside the error

Verified: pnpm build clean, pnpm test 69 pass / 12 skip.

Signed-off-by: suguanyang <wangbinqi77@gmail.com>
The success reconcile step was removing the optimistic user message
and only appending assistant-role server messages. Since /api/chat
returns both the persisted user turn and the assistant response,
filtering out user-role meant the user's question vanished from the
visible transcript after the assistant responded.

Fix: keep the optimistic user message in place (don't remove it) and
append only the assistant messages. The optimistic "pending-{ts}" id
stays as the React key — it's functionally equivalent to the server-
assigned UUID and invisible to the user.

Signed-off-by: suguanyang <wangbinqi77@gmail.com>
suguanYang added 28 commits May 9, 2026 19:47
Support 100 MB notebook source uploads
@vercel
Copy link
Copy Markdown

vercel Bot commented May 10, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
knowhere-notebook-staging Ready Ready Preview, Comment May 10, 2026 6:07pm

Request Review

@suguanYang suguanYang merged commit 104992c into main May 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant