Skip to content

tenzin333/Deep-Trial

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
.claude
.claude
 
 
backend
backend
 
 
extension
extension
 
 
.gitignore
.gitignore
 
 
DeepTrail_Privacy_Policy.html
DeepTrail_Privacy_Policy.html
 
 
PRIVACY_POLICY.md
PRIVACY_POLICY.md
 
 
README.md
README.md
 
 

Repository files navigation

🧠 DeepTrail

AI-Powered Personal Research Knowledge Graph

DeepTrail transforms your browsing activity into a semantic knowledge graph. Unlike traditional bookmarks or browser history, DeepTrail understands meaning — it connects pages by topic, summarizes content automatically, and lets you query your entire research history using natural language.

"What was I reading about transformer architectures last week?"

DeepTrail finds the answer from your browsing history and shows you exactly which pages it used.

✨ Features

  • Auto-Capture — Chrome extension silently tracks pages you visit (with smart 30-min deduplication)
  • Semantic Embeddings — Every page is converted into a 768-dimension vector that captures its meaning, not just keywords
  • Knowledge Graph — Pages with similar content are automatically connected. See your research as an interactive graph
  • AI Summaries — Every captured page gets a 2-4 sentence summary + keywords, generated by GPT-4o-mini
  • RAG Search — Ask questions in plain English. DeepTrail finds relevant pages, builds context, and generates a grounded answer with source citations
  • Side Panel UI — Live knowledge graph and search right inside your browser sidebar. No tab switching needed

🏗️ Architecture

Chrome Extension (Manifest V3)
        │
        │  POST /api/nodes (URL + Title + JWT)
        ▼
FastAPI Backend
        │
        ├─→ Text Extractor (httpx + trafilatura)
        ├─→ Embedder (OpenAI text-embedding-3-small → 768-d vector)
        ├─→ Similarity Engine (pgvector cosine search, threshold ≥ 0.6)
        ├─→ Summarizer (GPT-4o-mini, async background task)
        └─→ RAG Engine (embed query → top-K retrieval → LLM answer)
        │
        ▼
PostgreSQL 16 + pgvector
        │
        ├── users (id, email, password_hash)
        ├── nodes (url, title, summary, keywords, embedding VECTOR(768))
        └── edges (source_node_id, target_node_id, similarity_score)

🛠️ Tech Stack

Layer Technology Purpose
Extension Chrome Manifest V3 Browser event capture via Service Worker
Side Panel React 18 + React Flow Interactive graph visualization in sidebar
Backend FastAPI (Python) Async API with auto-generated OpenAPI docs
ORM SQLAlchemy 2.0 Async database operations with type safety
Vector Store pgvector VECTOR(768) columns + HNSW indexing
Text Extraction trafilatura HTML to clean article text
Embeddings OpenAI text-embedding-3-small 768-d semantic vectors
LLM OpenAI GPT-4o-mini Summarization + RAG answers
Database PostgreSQL 16 Relational data + vector persistence
Auth JWT (HS256) + bcrypt Stateless authentication
Deployment Docker + Docker Compose Containerized backend + database

📁 Project Structure

deeptrail/
│
├── backend/
│   ├── app/
│   │   ├── __init__.py
│   │   ├── main.py                  # FastAPI app, CORS, lifespan
│   │   ├── config.py                # Environment variables
│   │   ├── db.py                    # Async SQLAlchemy engine + session
│   │   ├── dependencies.py          # JWT auth (get_current_user)
│   │   │
│   │   ├── models/
│   │   │   ├── __init__.py          # Barrel imports
│   │   │   ├── user.py             # User table
│   │   │   ├── node.py             # Node table + Vector(768)
│   │   │   └── edge.py             # Edge table + unique constraint
│   │   │
│   │   ├── schemas/
│   │   │   ├── __init__.py
│   │   │   └── schemas.py          # Pydantic request/response models
│   │   │
│   │   ├── routers/
│   │   │   ├── __init__.py
│   │   │   ├── auth.py             # POST /register, /login
│   │   │   ├── nodes.py            # POST/GET/DELETE /nodes
│   │   │   ├── graph.py            # GET /graph
│   │   │   └── query.py            # POST /query (RAG)
│   │   │
│   │   └── services/
│   │       ├── __init__.py
│   │       ├── extractor.py        # httpx + trafilatura
│   │       ├── embedder.py         # OpenAI embeddings + retry
│   │       ├── similarity.py       # pgvector cosine search
│   │       ├── summarizer.py       # GPT-4o-mini background task
│   │       └── rag.py              # Query → embed → top-K → answer
│   │
│   ├── Dockerfile
│   ├── docker-compose.yml
│   ├── requirements.txt
│   ├── .env
│   └── .env.example
│
├── extension/
│   ├── manifest.json                # Manifest V3 config + permissions
│   ├── background.js                # Service Worker — tab events + POST to backend
│   ├── content.js                   # (Optional) DOM text extraction fallback
│   │
│   ├── sidepanel/
│   │   ├── index.html               # Side Panel entry point
│   │   ├── src/
│   │   │   ├── App.jsx              # Root component — routing + auth context
│   │   │   ├── index.jsx            # React mount point
│   │   │   │
│   │   │   ├── components/
│   │   │   │   ├── GraphView.jsx    # React Flow graph — nodes, edges, layout
│   │   │   │   ├── SearchPanel.jsx  # Semantic query input + results
│   │   │   │   ├── NodeCard.jsx     # Hover tooltip — summary + keywords
│   │   │   │   ├── NodeDetail.jsx   # Full modal — content + RAG links
│   │   │   │   ├── TabList.jsx      # Live grouped tab list
│   │   │   │   └── AuthForm.jsx     # Login / register form
│   │   │   │
│   │   │   ├── hooks/
│   │   │   │   ├── useGraph.js      # Fetch + poll graph data
│   │   │   │   ├── useSearch.js     # Debounced RAG query
│   │   │   │   └── useAuth.js       # JWT token management
│   │   │   │
│   │   │   ├── lib/
│   │   │   │   └── api.js           # Axios instance — JWT header injection
│   │   │   │
│   │   │   └── styles/
│   │   │       └── globals.css      # Tailwind base styles
│   │   │
│   │   ├── vite.config.js           # Vite bundler config for extension
│   │   ├── tailwind.config.js
│   │   ├── postcss.config.js
│   │   └── package.json
│   │
│   └── icons/
│       ├── icon-16.png
│       ├── icon-48.png
│       └── icon-128.png
│
├── README.md
└── .gitignore

🚀 Getting Started

Prerequisites

  • Docker and Docker Compose
  • Gemini AI API key
  • GROQ AI API key
  • Python 3.12.10

1. Clone the repo

git clone https://github.com/yourusername/deeptrail.git
cd deeptrail

2. Set up environment

cp .env.example .env

Edit .env with your keys:

DATABASE_URL=postgresql+asyncpg://postgres:postgres@db:5432/knowledgegraph
OPENAI_API_KEY=sk-your-key-here
GEMINI_API_KEY=sk-your-key-here
JWT_SECRET=generate-a-random-string-here
DEBUG=True
JWT_EXPIRY_HOURS=2
SIMILARITY_THRESHOLD=0.6

3. Start the app

docker-compose up --build

4. Open the API docs

Go to http://localhost:8000/docs — you'll see all endpoints with a Swagger UI to test them.

📡 API Endpoints

Method Endpoint Description Auth
POST /api/auth/register Create a new account No
POST /api/auth/login Login and get JWT token No
POST /api/nodes Ingest a browsed page JWT
GET /api/nodes/:id Get a single node with summary JWT
DELETE /api/nodes/:id Delete a node and its edges JWT
GET /api/graph Get full knowledge graph (nodes + edges) JWT
POST /api/query Ask a question over your browsing history JWT
GET /health Health check No

Example: Ingest a page

curl -X POST http://localhost:8000/api/nodes \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://arxiv.org/abs/1706.03762",
    "title": "Attention Is All You Need",
    "timestamp": "2026-03-03T10:00:00Z"
  }'

Response:

{
  "node_id": "a1b2c3d4-...",
  "edges_created": 3,
  "status": "processing"
}

Example: Ask a question

curl -X POST http://localhost:8000/api/query \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"question": "What have I been reading about attention mechanisms?"}'

Response:

{
  "answer": "Based on your browsing history, you've been researching...",
  "sources": [
    {
      "url": "https://arxiv.org/abs/1706.03762",
      "title": "Attention Is All You Need",
      "similarity_score": 0.92
    }
  ]
}

🔄 How It Works

1. You browse the web normally
         ↓
2. Extension detects page load (with 30-min dedup)
         ↓
3. Sends URL + title to backend
         ↓
4. Backend fetches page → extracts clean text
         ↓
5. Text → OpenAI → 768-d embedding vector
         ↓
6. pgvector scans existing nodes for cosine similarity ≥ 0.6
         ↓
7. Matching nodes get connected via edges in the graph
         ↓
8. GPT-4o-mini summarizes the page in background
         ↓
9. Side Panel shows your growing knowledge graph in real-time
         ↓
10. Ask any question → RAG engine finds relevant nodes → answers with citations

⚙️ Configuration

Variable Default Description
DATABASE_URL — PostgreSQL connection string
GEMINI_API_KEY — Your GEMINI API key
OPENAI_API_KEY — Your OPENAI API key
JWT_SECRET — Secret for signing JWT tokens
JWT_EXPIRY_HOURS 24 Token expiration time
SIMILARITY_THRESHOLD 0.6 Minimum cosine similarity to create an edge
DEBUG True For debugging purpose
ALLOW_ORIGINS * Allowed origins

Note: set your backend endpoint within extension/sidepanel/config.js

📊 Database Schema

users

Column Type Notes
id UUID Primary key
email VARCHAR(255) Unique
password_hash TEXT bcrypt hashed
created_at TIMESTAMPTZ Auto-set

nodes

Column Type Notes
id UUID Primary key
user_id UUID FK → users, cascade delete
url TEXT Page URL
title TEXT Page title
raw_text TEXT Extracted article content
summary TEXT AI-generated summary
keywords TEXT[] Keyword array
key_concepts JSONB Structured concepts
embedding VECTOR(768) pgvector, HNSW indexed
summary_status VARCHAR(20) pending / complete / failed
created_at TIMESTAMPTZ Auto-set

edges

Column Type Notes
id UUID Primary key
source_node_id UUID FK → nodes, cascade delete
target_node_id UUID FK → nodes, cascade delete
similarity_score FLOAT Cosine similarity (0.6–1.0)
created_at TIMESTAMPTZ Auto-set

Potential Error case

  1. A PostgreSQL async database URL is used by async libraries like asyncpg or SQLAlchemy when working with PostgreSQL in asynchronous Python frameworks such as FastAPI. postgresql+asyncpg://USER:PASSWORD@HOST:PORT/DATABASE

🎯 Roadmap

  • Backend API (FastAPI + pgvector + RAG)
  • Chrome Extension with Side Panel
  • Interactive graph visualization (React Flow)
  • Cross-device sync
  • Usage tiers and Stripe integration
  • Team shared knowledge graphs
  • Export to Obsidian / Notion

📄 License

MIT

Built by Tenzin

About

AI-Powered Personal Research Knowledge Graph

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages