🔍 LearnLens — Study Copilot

Share a browser tab, window, or screen and chat about it with a local vision model. Built for studying: explanations, summaries, quizzes with hidden answers, mnemonics.

Run

npm install
npm run dev        # http://localhost:5173

Connect a local model

Any OpenAI-compatible server with a vision model works. Set the URL and model name via the ⚙️ button in the top bar (saved to localStorage).

LM Studio (default, http://localhost:1234/v1)

1. Load a vision model, e.g. gemma-3-12b-it (Gemma 3 12B has vision built in).
2. Start the server from the Developer tab.
3. ⚠️ Enable CORS in LM Studio's server settings, or the browser will block requests.

llama-server (http://localhost:8080/v1)

llama-server -m gemma-3-12b-it-Q4_K_M.gguf --mmproj mmproj-model.gguf --port 8080

CORS is allowed by default. The --mmproj projector file is required for vision.

How sharing works

• Manual (default): 📸 Snapshot captures frames you choose; they attach to your next message. With no snapshots, a fresh frame is grabbed the moment you hit Send.
• 0.5 / 1 / 5 fps: frames are captured continuously so the freshest frame is attached automatically with every question — ideal while watching videos.
• Frames are downscaled to ≤1280px JPEG before sending (see src/lib/capture.js).
• Only the newest message carries images; older turns keep a text note, so context stays small for local models.

Code map

src/
  config.js                 server presets + localStorage settings
  lib/llmClient.js          OpenAI-compatible SSE streaming
  lib/prompts.js            tutor system prompt (HTML output + quiz format)
  lib/capture.js            video frame → JPEG data-URL
  lib/sanitize.js           DOMPurify wrapper for model HTML
  hooks/useScreenShare.js   getDisplayMedia lifecycle
  hooks/useAutoCapture.js   fps-based frame loop
  hooks/useChat.js          chat state + streaming + history trimming
  components/               SharePanel, MessageList, Composer, QuickActions, SettingsBar, HtmlView

Each file is small and single-purpose — add features by adding a hook or component.

Ideas to add next

• Flashcard export (Anki format) from quiz answers
• Session notes: "save this explanation" to a notebook
• Voice input via Web Speech API
• Multi-frame burst for fast-moving videos