Accio — Multi-Agentic Autonomous Learning Tool

"Accio doesn't just watch the lecture with you; it understands exactly how to help you master it."

Accio is an autonomous, multi-agent learning engine that watches a lecture alongside its user and continuously models their comprehension in real time. By mapping lecture content into a 384-dimensional neural embedding space, Accio tracks knowledge state with mathematical precision and then acts on it, delivering personalised context and clarification without the generic advice of a traditional AI

---

How It Works — The Big Picture

A student watches a video. As they interact with it such as pausing, rewinding, acing or failing quiz questions etc, Accio captures every signal automatically. These signals feed a live Bayesian evidence engine that updates a per-concept mastery score and uncertainty score for every topic in the lecture. The weakest, most uncertain concepts bubble up and trigger the autonomous agent, which generates a structured Context Pack and delivers a targeted, grounded explanation back to the student in real time.

Why do this

Traditional LLMs have to cram the student's entire history, so things such as rewinds, quiz scores, notes etc into a text prompt. This is expensive, lossy, and hits a hard limit so when the context fills up, the model forgets.

Accio sidesteps this entirely. Rather than sending history, it sends geometry — a context extracted from a live mathematical 384 dimensional representation of the student's knowledge state. The result is a payload orders of magnitude smaller than a chat history, but denser in meaning: every field is a mathematically derived signal, not a text approximation.

The graph is the memory. The LLM is only called when it's needed, with exactly the context it needs, nothing more.

Video Lecture
      │
      ▼
Player Events (pause, rewind, quiz, …)
      │
      ▼
┌─────────────────────────────────────────────┐
│             Confusion Model                 │  ← trained on MOOCCube (~250k events)
│  outputs: confusion_probability, risk_band  │
└─────────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────────┐
│          Supabase Learning Data API         │  ← stores & retrieves telemetry
│  /store-learning-data  /get-learning-data   │
└─────────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────────┐
│         GraphState — Evidence Engine        │  ← Bayesian mastery derivation
│  mastery ∈ [-1, 1]   uncertainty ∈ [0, 1]  │
└─────────────────────────────────────────────┘
      │
      ├──── 3D WebGL Learner Graph (live visual for testing purposes)
      │
      └──── Context Pack → Gemini (personalised explanation)

---

The Codebase Structure Explained

Accio/

├──  assets/
│
├──  confusion_model/       # [INTELLIGENCE] Trained ML classifier (MOOCCube dataset) that 
│                             # predicts student confusion from binary behavioral signals.
│
├──  landing/               # [ONBOARDING] Standalone Vite-based project handling marketing, 
│                             # user waitlist flow, and the initial onboarding experience.
│                             # live at https://acciohackeurope.vercel.app/
│
├──  supabase/              # [INFRASTRUCTURE] Configuration and SQL migrations for the cloud 
│                             # database tracking long-term Bayesian mastery scores.
│
├──  webgl_visualisation/   # [TESTING] Interactive 3D diagnostic tool that renders 
│                             # student knowledge state from 384D neural embeddings.
│
│
│
├──  server.py
│
├──  app.js
│
├──  app.html
│
├──  styles.css
│
├──  requirements.txt
│
└──  README.md

---

The Semantic Graph — What It Is

Before any session begins, Accio processes the lecture's transcript into a semantic knowledge graph:

1. Each concept/topic in the video is embedded with a sentence-transformers model into a 384-dimensional vector.
2. PCA reduces these to 2D (x, y) coordinates — topics that are semantically similar end up physically close.
3. k-Means clustering groups topics into subject domains (e.g. "Linear Algebra", "Probability").
4. Cosine similarity between embeddings determines the weight of each link (cylinder) between topics.

---

The Confusion Model

confusion_model/ contains a trained ML classifier (trained on the MOOCCube dataset — ~250,000 real student video interaction events) that computes a confusion_probability ∈ [0, 1] and risk_band (low / medium / high / very_high) purely from session-level engagement metrics:

• watching_count — how many times this segment was viewed
• rewind_events_est — estimated number of rewinds
• local_watching_time_sec — total wall-clock time spent
• video_progress_time_sec — actual video time covered
• pause_seconds — cumulative pause duration

This model complements the evidence engine for cases where explicit quiz/confusion signals aren't available which gives Accio a signal even for a passive viewer.

---

Instructions to Run the App

Enter the API Keys in .env.local

# First-time setup + run
cd <path-to-project>
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python3 server.py

# Next runs
cd <path-to-project>
source .venv/bin/activate
python3 server.py