Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

27 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Sydney β€” Systematic Yielding of Disease-associated Genomic Evidence and DiscoverY

AI-Powered Biomedical Variant Intelligence Platform

Sydney is a lightweight web application that helps researchers, students, and clinicians understand genetic variants by aggregating evidence from ClinVar, PubMed, and biomedical literature. It generates structured reports with confidence scoring, AI summaries, and research gap analysis β€” without hallucinating results.

Supported genes (14): BRCA1, BRCA2, TP53, CDH1, PALB2, CHEK2, ATM, PTEN, EGFR, KRAS, ALK, BRAF, MLH1, MSH2 β€” covering breast, ovarian, gastric, lung, colorectal, and related hereditary cancer syndromes.

Why I Built This

Genetic variant interpretation is one of the most consequential tasks in modern medicine β€” a single classification of "Pathogenic" or "Uncertain significance" can change a patient's surgical decisions, screening schedule, and family planning. Yet the evidence behind these classifications is scattered across ClinVar, PubMed, and dozens of specialist databases, with no single tool that aggregates, scores, and explains it in one place.

I built Sydney to solve that aggregation problem β€” and to learn how to build a production-grade full-stack application along the way. It gave me hands-on experience with:

  • Integrating real biomedical APIs (NCBI E-utilities, gnomAD GraphQL) that have quirks, rate limits, and undocumented edge cases (non-breaking spaces in XML, deprecated rettype formats)
  • Designing a transparent, explainable scoring system where every number can be traced back to a source β€” no black boxes
  • Building AI features responsibly: hallucination prevention, evidence-grounded prompts, and graceful degradation when the API is unavailable
  • Writing tests and benchmarks for a data pipeline where "correct" depends on live external data

The name Sydney is a backronym: Systematic Yielding of Disease-associated Genomic Evidence and DiscoverY.


Table of Contents


Architecture Overview

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                        Browser                                    β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
β”‚  β”‚  Next.js 15 App Router                                      β”‚  β”‚
β”‚  β”‚  β€’ React 19 + TypeScript                                    β”‚  β”‚
β”‚  β”‚  β€’ Tailwind CSS + Dark Mode                                 β”‚  β”‚
β”‚  β”‚  β€’ React Query (caching, refetch)                           β”‚  β”‚
β”‚  β”‚  β€’ Recharts (evidence charts)                               β”‚  β”‚
β”‚  β”‚  β€’ SVG graph (knowledge relationships)                      β”‚  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                          β”‚ HTTP (localhost:3000 β†’ localhost:8000)
                          β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  FastAPI Backend                                                      β”‚
β”‚                                                                       β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”‚
β”‚  β”‚ API Routes   │───▢│ Services     │───▢│ Database           β”‚       β”‚
β”‚  β”‚ (routes.py)  β”‚    β”‚ (10 services)β”‚    β”‚ (SQLAlchemy/SQLite)β”‚       β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       β”‚
β”‚         β”‚                   β”‚                                         β”‚
β”‚         β”‚                   β”œβ”€β”€ ClinVar Service ──▢ NCBI E-utilities  β”‚
β”‚         β”‚                   β”œβ”€β”€ PubMed Service  ──▢ NCBI E-utilities  β”‚
β”‚         β”‚                   β”œβ”€β”€ AI Summary      ──▢ Groq API          β”‚
β”‚         β”‚                   β”œβ”€β”€ gnomAD Service  ──▢ Broad gnomAD v4   β”‚
β”‚         β”‚                   └── PDF Generator   ──▢ ReportLab         β”‚
β”‚         β–Ό                                                             β”‚
β”‚  OpenAPI: /docs                                                       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Component Responsibilities

Layer Technology Role
Frontend Next.js 15, TypeScript, Tailwind, React Query Search UI, evidence dashboard, knowledge graph, PDF download
Backend API FastAPI, Pydantic REST endpoints, input validation, OpenAPI docs
Business Logic Python services Variant parsing, evidence scoring, confidence calculation
Database SQLite (dev) / PostgreSQL (prod), SQLAlchemy Variants, papers, evidence, reports
External APIs NCBI E-utilities, Groq ClinVar queries, PubMed search, AI summary generation

Data Pipeline (End to End)

When a user searches for a variant (e.g., TP53 R175H), the following pipeline executes:

Step 1: Input Validation (VariantAnalysisService.parse_variant)

User Input: "TP53 R175H"
  ↓
Regex matching against 14 supported genes and 6 notation patterns:
  β€’ c. notation:   BRCA1 c.5266dupC, CDH1 c.1901C>T
  β€’ p. notation:   TP53 p.R175H
  β€’ 1-letter code: TP53 R175H
  β€’ 3-letter code: TP53 Arg175His
  β€’ Legacy:        BRCA1 185delAG
  β€’ Alias:         P53 R175H β†’ TP53
  ↓
If no match β†’ 400 error with helpful message
If match β†’ { gene: "TP53", change: "R175H" }

Step 2: Gene Resolution

Gene symbol "TP53", "BRCA1", "BRCA2", "CDH1", "PALB2", "CHEK2", "ATM", "PTEN", "EGFR", "KRAS", "ALK", "BRAF", "MLH1", "MSH2" (or alias P53 β†’ TP53)
  ↓
Query genes table β†’ if not found, create with metadata:
  β€’ symbol, full_name, chromosome, description

Step 3: Variant Lookup/Creation

gene + change
  ↓
Query variants table by HGVS c. or protein change
  ↓
If not found β†’ Create variant record:
  β€’ gene_id, hgvs_c, protein_change, variant_type

Step 4: ClinVar Retrieval (ClinVarService)

gene + variant
  ↓
1. Check local disk cache (data/cache/clinvar/)
   ↓ if cache miss:
2. ESearch: NCBI E-utilities β†’ get ClinVar ID
   β€’ Query: "TP53[gene] AND R175H[variant] OR R175H[All Fields]"
   β€’ Fallback: "TP53 R175H"
3. EFetch: rettype=vcv, retmode=xml
   β€’ VCV accession (zero-padded to 9 digits): VCV000012374
4. Parse VCV XML for:
   β€’ <GermlineClassification> β†’ <Description> β†’ "Pathogenic"
   β€’ <ReviewStatus> β†’ "reviewed by expert panel"
   β€’ <VariationArchive VariationName="...">
   β€’ <TraitSet> β†’ disease names
5. Cache result as JSON
   ↓
Update variant record:
  β€’ clinical_significance, clinvar_id, review_status, clinvar_data

Step 5: gnomAD Population Frequency (GnomadService)

gene + variant + clinvar_data (genomic coordinates)
  ↓
1. Check local disk cache (data/cache/gnomad/)
   ↓ if cache miss:
2. Extract genomic coordinates from ClinVar response
   β€’ { chr: "17", pos: 7675094, ref: "G", alt: "A" }
3. Build gnomAD v4 variant ID: "17-7675094-G-A"
4. Query gnomAD GraphQL API: https://gnomad.broadinstitute.org/api
   β€’ Returns: allele_frequency, allele_count, allele_number,
     homozygote_count, population_frequencies (by ancestry)
5. Store in variant record:
   β€’ variant.gnomad_af = allele_frequency
   β€’ variant.gnomad_data = full API response
6. Cache result as JSON
   ↓
If variant not found in gnomAD β†’ graceful None (no error)

Step 6: PubMed Retrieval (PubMedService)

gene + variant + disease (mapped per gene β€” see table below)
  ↓
1. Check local disk cache (data/cache/pubmed/)
   ↓ if cache miss:
2. ESearch example: "(TP53[Title/Abstract]) AND (R175H[Text Word]) AND (cancer[MeSH])"
   ‒ Disease term is gene-specific: BRCA1→"breast cancer", EGFR→"lung cancer", etc.
   β€’ Limit: 20 results, sorted by relevance
3. EFetch: Get XML with titles, authors, abstracts
4. Infer study type via batched Groq call (all papers in one prompt):
   β€’ Sends title + abstract for each paper, requests JSON array of study types
   β€’ On success: classifies each paper accordingly
   β€’ On failure (Groq unavailable, JSON parse error, array length mismatch):
     β†’ Falls back entire batch to keyword-based matching (see table below)
5. Cache results as JSON
   ↓
For each paper (all 14 genes share the same pipeline β€” no gene-specific logic needed beyond the symbol):
  β€’ Create Paper record (pmid, title, authors, journal, year, abstract, study_type)
  β€’ Create Evidence record (variant_id, paper_id, evidence_type="literature")

Step 7: Evidence Scoring (EvidenceScoringService)

For each evidence item:
  ↓
relevance = keyword overlap between paper and variant (0.5 - 1.0)
study_quality = STUDY_QUALITY_MAP[paper.study_type]
  β€’ Meta-Analysis: 0.95
  β€’ Clinical Trial: 0.90
  β€’ Cohort Study:   0.75
  β€’ Case Report:    0.35
recency = max(0, 1.0 - (current_year - paper_year) * 0.05)
  ↓
evidence_score = (0.50 Γ— relevance) + (0.30 Γ— study_quality) + (0.20 Γ— recency)
  ↓
Score range: 0.0 - 1.0 (displayed as 0-100)

Step 8: Confidence Calculation (ConfidenceEngine)

All evidence items for variant
  ↓
evidence_volume = count of papers
  β€’ 0 papers β†’ 0.0
  β€’ 1-2 papers β†’ 0.2
  β€’ 3-4 papers β†’ 0.4
  β€’ 5-9 papers β†’ 0.6
  β€’ 10-19 papers β†’ 0.8
  β€’ 20+ papers β†’ 1.0
  ↓
evidence_quality = average study_quality_score across all papers
  ↓
study_agreement = consistency of clinical_significance across papers
  ↓
confidence_score = (0.20 Γ— volume) + (0.40 Γ— quality) + (0.30 Γ— agreement) + (0.10 Γ— clinvar_review_strength)
  ↓
Level mapping:
  score >= 0.70 β†’ High
  score >= 0.40 β†’ Moderate
  score < 0.40 β†’ Low
  score == 0   β†’ Insufficient Evidence

Step 9: Report Generation

variant + evidence + confidence
  ↓
Create Report record with all scores and metadata
  ↓
PDF generation (on demand):
  β€’ ReportLab β†’ professional scientific PDF
  β€’ Sections: Executive Summary, Clinical Significance,
    Evidence Overview, Supporting Studies, Disease Associations,
    Confidence Assessment

Step 10: AI Summary (optional, requires Groq API key)

variant + evidence + confidence
  ↓
Build context string with all paper titles, PMIDs, scores, findings
  ↓
Groq API call: Llama 3.3 70B
  β€’ System prompt: evidence-based, no hallucinations, cite PMIDs
  β€’ Temperature: 0.3 (low creativity, high accuracy)
  β€’ Generates sections: Executive Summary, Clinical Significance,
    Disease Associations, Mechanism of Action, Evidence Overview,
    Confidence Assessment

Step 11: Research Gap Detection (ResearchGapDetector)

Analyze evidence distribution
  ↓
Rule-based checks:
  β€’ No clinical trials found?
  β€’ No functional studies?
  β€’ Fewer than 3 high-quality papers?
  β€’ Total papers < 5?
  β€’ Predominantly case reports?
  β€’ No recent studies (post-2020)?
  ↓
Generate gap list + summary
  ↓
Optional AI analysis of research directions

Database Schema

Entity Relationship

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     genes     β”‚       β”‚    variants      β”‚       β”‚    evidence      β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€       β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€       β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ id (PK)       │──1:N──│ id (PK)          │──1:N──│ id (PK)          β”‚
β”‚ symbol (UQ)   β”‚       β”‚ gene_id (FK)     β”‚       β”‚ variant_id (FK)  β”‚
β”‚ full_name     β”‚       β”‚ hgvs_c (idx)     β”‚       β”‚ paper_id (FK)    β”‚
β”‚ chromosome    β”‚       β”‚ hgvs_p (idx)     β”‚       β”‚ evidence_type    β”‚
β”‚ description   β”‚       β”‚ protein_change   β”‚       β”‚ relevance_score  β”‚
β”‚ created_at    β”‚       β”‚ variant_type     β”‚       β”‚ study_quality    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       β”‚ description      β”‚       β”‚ recency_score    β”‚
        β”‚               β”‚ clin_sig         β”‚       β”‚ evidence_score   β”‚
        β”‚               β”‚ clinvar_id       β”‚       β”‚ key_findings     β”‚
        β”‚               β”‚ clinvar_data(J)  β”‚       β”‚ source           β”‚
        β”‚               β”‚ review_status    β”‚       β”‚ created_at       β”‚
        β”‚               β”‚ created_at       β”‚       β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
        β”‚               β”‚ updated_at       β”‚                β”‚
        β”‚               β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                β”‚
        β”‚                        β”‚                          β”‚
        β”‚        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚        β”‚            reports      β”‚       β”‚     papers       β”‚
        β”‚        β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€       β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
        β”‚        β”‚ id (PK)                 β”‚       β”‚ id (PK)          β”‚
        β”‚        β”‚ variant_id (FK)         β”‚       β”‚ pmid (UQ, idx)   β”‚
        β”‚        β”‚ confidence_level        β”‚       β”‚ title            β”‚
        β”‚        β”‚ confidence_score        β”‚       β”‚ authors          β”‚
        β”‚        β”‚ evidence_volume         β”‚       β”‚ journal          β”‚
        β”‚        β”‚ evidence_quality        β”‚       β”‚ year             β”‚
        β”‚        β”‚ study_agreement         β”‚       β”‚ abstract         β”‚
        β”‚        β”‚ exec_summary            β”‚       β”‚ doi              β”‚
        β”‚        β”‚ clin_sig                β”‚       β”‚ study_type       β”‚
        β”‚        β”‚ disease_assoc(J)        β”‚       β”‚ keywords (JSON)  β”‚
        β”‚        β”‚ mechanism               β”‚       β”‚ created_at       β”‚
        β”‚        β”‚ evidence_overview       β”‚       β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
        β”‚        β”‚ confidence_assess       β”‚
        β”‚        β”‚ clinvar_review_strength β”‚
        β”‚        β”‚        research_gaps(J) β”‚
        β”‚        β”‚ ai_summary              β”‚
        β”‚        β”‚ report_data (J)         β”‚
        β”‚        β”‚ created_at              β”‚
        β”‚        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
        β”‚
        β”œβ”€β”€β”€ gene_papers (M:N join) ─── papers
        β”‚
        └─── disease_papers (M:N join) ─── papers

  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚   diseases    β”‚
  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
  β”‚ id (PK)       β”‚
  β”‚ name (idx)    β”‚
  β”‚ mondo_id      β”‚
  β”‚ description   β”‚
  β”‚ created_at    β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  (J) = JSON column
  (FK) = Foreign Key
  (PK) = Primary Key
  (UQ) = Unique
  (idx) = Indexed

Features

1. Variant Search (Feature 1)

The search interface accepts multiple variant notation formats:

Format Example Pattern
HGVS coding BRCA1 c.5266dupC gene + c. prefix + position + change
HGVS protein TP53 p.R175H gene + p. prefix + amino acid change
1-letter protein TP53 R175H gene + [A-Z]\d+[A-Z*]
3-letter protein TP53 Arg175His gene + [A-Z][a-z]{2}\d+[A-Za-z*]
Legacy BRCA1 185delAG gene + \d+del[A-Z]+
Alias P53 R175H Auto-normalized to TP53

Invalid inputs return a 400 error with a helpful message: "Could not parse variant. Use format like: BRCA1 c.5266dupC, TP53 R175H, BRCA2 c.5946delT, CDH1 c.1901C>T, PALB2 c.1592delT"

Recent searches are stored in localStorage (client-side only) and displayed as clickable badges.

2. ClinVar Integration (Feature 3)

Endpoint: NCBI E-utilities (https://eutils.ncbi.nlm.nih.gov/entrez/eutils/)

Flow:

  1. esearch.fcgi β€” Search for ClinVar records using gene[variant] query
  2. efetch.fcgi?rettype=vcv β€” Fetch VCV XML (the modern ClinVar format, not the deprecated rettype=variation)
  3. Parse XML for clinical significance, review status, disease names

VCV Accession Format:

  • Numeric IDs from esearch are zero-padded to 9 digits
  • Example: ID 12374 β†’ VCV000012374
  • This is required by NCBI's API

Caching:

  • JSON responses cached to data/cache/clinvar/
  • TTL: 24 hours (configurable via CACHE_TTL_HOURS)
  • Cache key: {gene}_{variant} with special characters sanitized

Parsed Fields:

XML Path Field Example
VariationArchive/@VariationName description NM_000546.6(TP53):c.524G>A (p.Arg175His)
VariationArchive/@Accession accession VCV000012374
GermlineClassification/Description clinical_significance Pathogenic
GermlineClassification/ReviewStatus review_status reviewed by expert panel
OncogenicityClassification/Description clinical_significance (fallback) Oncogenic
TraitSet/Trait/Name diseases (if non-empty) Li-Fraumeni syndrome

3. PubMed Retrieval (Feature 4)

Endpoint: NCBI E-utilities

Search Query Construction:

(gene[Title/Abstract]) AND (variant[Text Word]) AND (disease[MeSH])

Disease term is mapped per gene: BRCA1/BRCA2/PALB2 β†’ breast cancer, CDH1 β†’ gastric cancer, EGFR/ALK β†’ lung cancer, KRAS β†’ pancreatic cancer, BRAF β†’ melanoma, PTEN β†’ Cowden syndrome, MLH1/MSH2 β†’ colorectal cancer, others β†’ cancer.

Result Limit: 20 papers (configurable via MAX_PUBMED_RESULTS)

Parsed Fields:

XML Path Field
PMID pmid
ArticleTitle title
Author/LastName + ForeName authors (first 10)
Journal/Title journal
PubDate/Year year
AbstractText abstract (with Label attributes)
ELocationId[@EIdType="doi"] doi
Keyword keywords array

Study Type Inference: Papers are classified in a single batched Groq call (Llama 3.3 70B, temperature 0.1). The prompt sends each paper's title and abstract, requesting a JSON array of study types. If Groq is unavailable, the response is not valid JSON, or the array length doesn't match the paper count, the entire batch falls back to keyword-based matching:

Fallback Keyword Mapping:

Keywords in Abstract Study Type Quality Score
"clinical trial", "randomized", "phase I/II/III" Clinical Trial 0.90
"meta-analysis", "systematic review" Meta-Analysis 0.95
"case report", "case study" Case Report 0.35
"cohort", "case-control", "longitudinal" Cohort Study 0.75
"review", "overview" Review 0.50
"in vitro", "cell line", "functional study" Functional Study 0.70
"genome-wide", "gwas", "association study" Genome-Wide Study 0.80
None of the above Research Article 0.50

Caching: Same scheme as ClinVar, stored in data/cache/pubmed/ with 24-hour TTL.

4. gnomAD v4 Population Frequency (Feature 5)

Endpoint: Broad Institute gnomAD GraphQL API (https://gnomad.broadinstitute.org/api)

Flow:

  1. After ClinVar retrieves variant data, genomic coordinates are extracted from the ClinVar response
  2. Coordinates formatted as {chr}-{pos}-{ref}-{alt} gnomAD variant ID
  3. GraphQL query requests genome & exome AF (computed from ac/an), AC, AN, homozygote_count, and per-population breakdowns
  4. Genome AF preferred; exome AF used as fallback

GraphQL Query:

query VariantFrequency($datasetId: DatasetId!, $variantId: String!) {
  variant(variantId: $variantId, dataset: $datasetId) {
    variant_id
    genome {
      af ac an homozygote_count
      populations { id af ac an }
    }
    exome {
      af ac an homozygote_count
      populations { id af ac an }
    }
  }
}

Dataset: gnomad_r4

Database Fields:

Column Type Description
gnomad_af FLOAT Global allele frequency (genome or exome)
gnomad_data JSON Full API response including per-population breakdown

Color Coding (Frontend):

AF Range Color Badge Label
< 0.0001 Red Rare
0.0001–0.001 Amber Low
> 0.001 Green Common
Not found Grey outline Absent from gnomAD

Caching: Same scheme as ClinVar, stored in data/cache/gnomad/ with 24-hour TTL. Cache key: {gene}_{variant}.

API Endpoint: GET /api/v1/variants/{id}/gnomad β€” returns full population breakdown by ancestry group.

Configuration: No additional environment variables required. Uses the existing ClinVar data pipeline for coordinate extraction.

5. Evidence Scoring Engine (Feature 6)

Formula:

EvidenceScore = 0.50 Γ— relevance + 0.30 Γ— study_quality + 0.20 Γ— recency

Components:

Component Weight Calculation
Relevance 50% Keyword overlap between paper keywords and variant key findings. Baseline: 0.5, bonus: up to +0.5 for keyword matches. Range: 0.5–1.0
Study Quality 30% Mapped from study type (0.35 for Case Report up to 0.95 for Meta-Analysis)
Recency 20% max(0, 1.0 - (current_year - paper_year) Γ— 0.05). A paper from 2026 scores 1.0, from 2020 scores 0.7, from 2010 scores 0.2

Score Display: Scores are multiplied by 100 for the UI (0–100 scale).

6. Confidence Engine (Feature 10)

Formula:

ConfidenceScore = (0.20 Γ— volume_score) + (0.40 Γ— quality_score) + (0.30 Γ— agreement_score) + (0.10 Γ— clinvar_review_strength)

Components:

Component Weight Calculation
Evidence Volume 20% Logarithmic scale based on paper count: 0 papers = 0.0, 1-2 = 0.2, 3-4 = 0.4, 5-9 = 0.6, 10-19 = 0.8, 20+ = 1.0
Evidence Quality 40% Average study_quality_score across all papers (0.0–1.0)
Study Agreement 30% Proportion of papers with the same clinical significance classification (0.0–1.0)
ClinVar Review Strength 10% Maps review_status to a score: expert panel = 1.0, multi-submitter = 0.9, single submitter = 0.7, conflicting = 0.5, no assertion criteria = 0.3, no assertion = 0.0

Levels:

Score Range Level Meaning
0.70–1.00 High Well-characterized variant with strong, consistent evidence
0.40–0.69 Moderate Some evidence available, but gaps remain
0.01–0.39 Low Limited evidence, further studies needed
0.00 Insufficient Evidence No supporting papers found; no hallucination

7. AI Research Summary (Feature 9)

Provider: Groq API with Llama 3.3 70B (requires GROQ_API_KEY)

Generation: Triggered via /api/v1/variants/{id}/summary

Prompt Engineering:

  • System prompt instructs the model to be evidence-based and cite specific PMIDs
  • Context includes all paper titles, PMIDs, years, study types, evidence scores, and key findings
  • Temperature set to 0.3 (minimal creativity, prioritizes accuracy)

Output Sections:

  1. Executive Summary
  2. Clinical Significance
  3. Disease Associations
  4. Mechanism of Action
  5. Evidence Overview
  6. Confidence Assessment

Hallucination Prevention:

  • Model explicitly instructed: "Only use the provided evidence. Do not hallucinate."
  • No retrieved evidence β†’ model returns "No evidence available for this variant"
  • All claims should reference supporting PMIDs

8. Knowledge Relationships View (Feature 8)

Implementation: Custom SVG-based graph visualization (no external graph library dependency)

Entity Types (color-coded):

Type Color Description
Gene Blue (#3b82f6) BRCA1, BRCA2, TP53, CDH1, PALB2, CHEK2, ATM, PTEN, EGFR, KRAS, ALK, BRAF, MLH1, MSH2
Variant Purple (#8b5cf6) The specific mutation
Paper Green (#059669) PubMed articles
Disease Amber (#d97706) Associated conditions

Relationships: has variant, evidence, associated with

Layout: Force-directed layout with gene at top, variant in center, papers and diseases arranged radially.

Data Source: /api/v1/graph/{variant_id} endpoint constructs nodes and edges from the database.

9. Research Gap Detection (Feature 11)

Rule-Based Analysis (ResearchGapDetector.analyze_gaps):

Check Condition Gap Message
Clinical trials Count == 0 "No clinical trials found for this variant"
Functional studies Count == 0 "Functional characterization studies are limited"
High-quality studies Count < 3 "Only N high-quality studies available (need 3+)"
Total evidence Count < 5 "Limited evidence volume (N papers)"
Case report dominance Case reports > 50% of total "Evidence is predominantly case reports; larger cohort studies needed"
Recent publications Post-2020 papers < 2 "Recent studies (post-2020) are lacking"
All checks pass None triggered "Relatively well-studied; further meta-analyses could strengthen evidence"

Output: Gap list + well-studied boolean + summary text + study type distribution.

10. PDF Report Export (Feature 12)

Library: ReportLab

Sections:

  1. Variant Header β€” Gene, HGVS notation, clinical significance, confidence level
  2. Executive Summary β€” AI-generated or evidence overview
  3. Clinical Significance β€” ClinVar data with review status
  4. gnomAD Frequency β€” Population allele frequency with 1-in-N formatting (or "Absent from gnomAD")
  5. Evidence Overview β€” Volume, quality, agreement scores
  6. Supporting Studies β€” Top 10 papers with titles, PMIDs, years, scores
  7. Disease Associations β€” Disease names from ClinVar
  8. Confidence Assessment β€” Level + score + detailed breakdown

Generation: Synchronous, returns PDF as download (~4KB for average reports).


API Reference

Base URL

http://localhost:8000/api/v1

Endpoints

Method Path Description Auth
GET /health Health check None
GET /dashboard Platform statistics None
POST /variants/search Search and analyze a variant None
GET /variants List all variants None
GET /variants/{id} Variant detail with evidence None
GET /variants/{id}/evidence Evidence list with scores None
GET /variants/{id}/report Confidence report None
POST /variants/{id}/summary Generate AI summary (Groq) None
GET /variants/{id}/gaps Research gap analysis None
GET /variants/{id}/evidence-provenance Per-paper score contribution breakdown None
GET /variants/{id}/acmg ACMG-inferred variant classification None
GET /variants/{id}/classification-timeline Historical ClinVar classification changes None
GET /variants/{id}/gnomad gnomAD v4 allele frequency by ancestry None
GET /variants/{id}/report/pdf Download PDF report None
GET /graph/{id} Knowledge graph data None
POST /compare Compare two variants side by side None
DELETE /variants/{id} Delete variant and its cache files None

Request/Response Examples

POST /variants/search

// Request
{ "query": "TP53 R175H" }

// Response (200)
{
  "id": 1,
  "hgvs_c": null,
  "hgvs_p": null,
  "protein_change": "R175H",
  "gene": "TP53",
  "gene_full_name": "Tumor Protein P53",
  "variant_type": "snv",
  "clinical_significance": "Pathogenic",
  "clinvar_id": "12374",
  "review_status": "reviewed by expert panel",
  "diseases": ["Li-Fraumeni syndrome"]
}

// Response (400 - invalid)
{
  "detail": "Could not parse variant. Use format like: BRCA1 c.5266dupC, TP53 R175H, BRCA2 c.5946delT, CDH1 c.1901C>T, PALB2 c.1592delT"
}

// Response (400 - unsupported gene)
{
  "detail": "Could not parse variant. Use format like: BRCA1 c.5266dupC, TP53 R175H, BRCA2 c.5946delT, CDH1 c.1901C>T, PALB2 c.1592delT"
}

GET /variants/{id}/report

{
  "id": 1,
  "variant_id": 1,
  "confidence_level": "High",
  "confidence_score": 0.76,
  "evidence_volume": 18,
  "evidence_quality": 0.55,
  "study_agreement": 0.89,
  "executive_summary": "TP53 R175H is a well-characterized pathogenic variant...",
  "clinical_significance": "Pathogenic",
  "disease_associations": [{"name": "Li-Fraumeni syndrome"}],
  "evidence_overview": "Found 18 supporting papers. Average evidence score: 0.55...",
  "confidence_assessment": "Based on 18 supporting papers; evidence volume score: 0.80...",
  "research_gaps": [],
  "ai_summary": "## Executive Summary\nThe TP53 R175H mutation...",
  "created_at": "2026-06-16T12:00:00"
}

Full OpenAPI documentation at http://localhost:8000/docs (Swagger UI) or http://localhost:8000/redoc (ReDoc).


Services Deep Dive

VariantAnalysisService

The orchestrator β€” coordinates all other services for a single variant analysis.

Key Methods:

  • parse_variant(query) β†’ Parses user input using regex patterns
  • get_or_create_gene(symbol) β†’ Returns existing gene or creates with metadata
  • analyze_variant(query) β†’ Full pipeline: parse β†’ gene β†’ variant β†’ ClinVar β†’ PubMed β†’ evidence
  • get_variant_detail(variant_id) β†’ Returns variant + gene + evidence with scores

Regex Patterns (in priority order):

r"^(GENE)\s+(c\.\d+[A-Za-z_>delinsup*]{1,30})$"     # HGVS c.
r"^(GENE)\s+(p\.[A-Za-z]{1,3}\d+[A-Za-z*]{1,5})$"   # HGVS p.
r"^(GENE)\s+([A-Z][a-z]{2}\d+[A-Za-z*]{1,5})$"      # 3-letter protein
r"^(GENE)\s+([A-Z]\d+[A-Z*]{1,3})$"                 # 1-letter protein
r"^(GENE)\s+(\d+del[A-Z]+)$"                         # Legacy del notation
r"^(GENE)\s+(\d+ins[A-Z]+)$"                         # Legacy ins notation

ClinVarService

Critical Implementation Details:

  • Uses rettype=vcv (not the deprecated rettype=variation)
  • VCV IDs must be zero-padded to 9 digits: str.zfill(9)
  • XML namespaces are NOT required (VCV XML doesn't use them)
  • Falls back from gene[variant] query to gene variant plain text query
  • Caches aggressively to avoid hitting NCBI rate limits
  • Parses classification_history from all VariationArchive + GermlineClassification entries, collecting (classification, review_status, date) triplets β€” enables the Classification Timeline feature

PubMedService

Critical Implementation Details:

  • Constructs complex query with [Title/Abstract], [Text Word], and [MeSH Terms] fields
  • Study type inference uses a batched Groq call (all papers classified in one prompt); falls back to keyword pattern matching if Groq unavailable
  • _batch_infer_study_types sends title + abstract per paper, parses JSON array response, validates length matches paper count
  • On failure (Groq error, unparseable JSON, length mismatch): falls back entire batch to _infer_study_type keyword matching β€” never silently drops papers
  • _infer_study_type checks keywords in priority order (Clinical Trial β†’ Meta-Analysis β†’ Case Report β†’ ... β†’ Research Article)
  • Caches results with gene+variant+disease as composite key (disease-aware caching)

EvidenceScoringService

Critical Implementation Details:

  • score_evidence_for_variant(variant_id) β€” Updates scores for ALL evidence linked to a variant
  • Scores are stored in the database (not computed on-the-fly)
  • Recency calculation: linear decay from current year (2026) backward

ConfidenceEngine

Critical Implementation Details:

  • calculate_confidence(variant_id) β€” Pure function, no side effects
  • generate_report(variant_id) β€” Creates Report record if one doesn't exist
  • Study agreement calculated by finding the most common clinical significance across all evidence

AISummaryService

Critical Implementation Details:

  • Requires GROQ_API_KEY in environment; returns fallback message if not set
  • Context builder constructs structured text with all paper metadata
  • System prompt explicitly prohibits hallucination
  • Temperature locked at 0.3

ACMGService

Critical Implementation Details:

  • classify(variant_id) β€” Evaluates ACMG/AMP 2015 criteria against available variant data (limited to evidence we have: ClinVar significance, publication volume, variant type, and review status). Does NOT replace full ACMG interpretation which requires population frequency, segregation, functional studies, and family history.
  • Detects null variants (PVS1) via variant_type + HGVS c. + protein_change pattern matching
  • Uses ClinVar significance and review status for PS1, BS1, BP4
  • PM2 triggered when gnomad_af < 0.0001 (or variant absent from gnomAD); evidence text shows real AF value or "absent from gnomAD"
  • Evidence volume thresholds trigger PP4 (β‰₯5 papers) and PS4 (β‰₯10 papers)
  • Missense pathogenic variants trigger PP3
  • Scoring: Very Strong=4, Strong=3, Moderate=2, Supporting=1
  • Returns classification level: Pathogenic / Likely pathogenic / Uncertain significance / Likely benign / Benign

ResearchGapDetector

Critical Implementation Details:

  • analyze_gaps(variant_id) β€” Pure rule-based analysis
  • compare_variants(gene_symbol) β€” Cross-variant comparison for a gene
  • Gap rules designed to be conservative (avoid false positives)

PDFReportGenerator

Critical Implementation Details:

  • Uses ReportLab's platypus (Platform Independent Page Layout)
  • Custom paragraph styles with Sydney color scheme (#1a365d, #2b6cb0)
  • Returns raw bytes for HTTP response with Content-Disposition: attachment

Quick Start

Prerequisites

  • Python 3.12+
  • Node.js 20+
  • 8GB RAM (system requirement, app uses ~350MB)
  • Dual-core CPU
  • Internet connection (for NCBI API calls)

Backend Setup

cd backend

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Linux/macOS
# venv\Scripts\activate   # Windows

# Install dependencies
pip install -r requirements.txt

# Configure (minimal β€” works out of box for basic features)
echo "GROQ_API_KEY=gsk_your_key_here" > .env  # Optional, for AI summaries

# Run
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Frontend Setup

cd frontend
npm install
npm run dev

Open http://localhost:3000

Docker Setup

docker compose up --build

Single-Command Run

./run.sh

Starts both backend and frontend, kills existing processes on ports 8000/3000 automatically. Press Ctrl+C to stop both.


Variant Format Reference

Accepted Inputs

# BRCA1
BRCA1 c.5266dupC       β†’ HGVS coding (duplication)
BRCA1 185delAG         β†’ Legacy notation (deletion)
BRCA1 5382insC         β†’ Legacy notation (insertion)

# BRCA2
BRCA2 c.5946delT       β†’ HGVS coding (deletion)
BRCA2 6174delT         β†’ Legacy notation

# TP53
TP53 R175H             β†’ Protein change (1-letter code)
TP53 p.R175H           β†’ Protein change (HGVS p. format)
TP53 Arg175His         β†’ Protein change (3-letter code)
TP53 R273H             β†’ Another common TP53 mutation
TP53 R248Q             β†’ Another common TP53 mutation
P53 R175H              β†’ Alias (auto-normalized to TP53)
TP53 R999X             β†’ Nonsense mutation (rare/not in ClinVar)

# CDH1
CDH1 c.1901C>T         β†’ HGVS coding (missense)

# PALB2
PALB2 c.1592delT       β†’ HGVS coding (frameshift deletion)

# CHEK2
CHEK2 c.1100delC       β†’ HGVS coding (frameshift deletion)

# ATM
ATM c.7271T>G          β†’ HGVS coding (missense)

# PTEN
PTEN c.697C>T          β†’ HGVS coding (nonsense)

Rejected Inputs (400 Error)

EGFR T790M              β†’ Unsupported gene (consider contributing)
KRAS G12D               β†’ Unsupported gene
ALK F1174L              β†’ Unsupported gene
hello world             β†’ Not a variant
12345                   β†’ Not a variant
TP53 mutation           β†’ Too vague
BRCA1 cancer            β†’ Not a variant
BRCA1'; DROP TABLE...   β†’ SQL injection (rejected)
<script>alert(1)</script> β†’ XSS (rejected)

Testing Guide

Backend Tests

cd backend
pytest ../tests/backend -v

# With coverage
pytest ../tests/backend -v --cov=app

# Specific test file
pytest ../tests/backend/test_services.py -v
pytest ../tests/backend/test_api.py -v

Test Coverage

48 tests covering:

Test Suite Tests What It Tests
test_api.py 19 Health, dashboard, search, 404 handling, evidence, report, graph, gaps, OpenAPI, compare, trends, why-matters, full pipeline
test_services.py 29 Variant parsing (7), gene lookup (2), evidence scoring (3), confidence engine (2), research gaps (1), gnomAD service (8), batch study type (2), new genes (8), confidence weights (2)

Manual QA Test Suite

Run the comprehensive test script:

# Test all variant formats
for q in "TP53 R175H" "TP53 p.R175H" "P53 R175H" "TP53 Arg175His" \
         "BRCA1 c.5266dupC" "BRCA1 185delAG" \
         "BRCA2 c.5946delT" \
         "CDH1 c.1901C>T" "PALB2 c.1592delT" "CHEK2 c.1100delC" \
         "ATM c.7271T>G" "PTEN c.697C>T" \
         "TP53 R999X" "EGFR T790M" "invalid"; do
  echo ">>> $q"
  curl -s -m 15 -X POST http://localhost:8000/api/v1/variants/search \
    -H 'Content-Type: application/json' \
    -d "{\"query\":\"$q\"}" | python3 -m json.tool 2>/dev/null
done

Benchmark Suite

Regression test the full retrieval pipeline against known variants.

benchmark.json

17 test cases across 14 genes with expected results (benchmark passes may require calibration):

Variant Min Papers Expected Confidence Expected Significance
TP53 R175H β‰₯15 Moderate, High Pathogenic
BRCA1 c.5266dupC β‰₯15 High Uncertain significance
BRCA2 c.5946delT β‰₯10 Moderate, High no classifications from unflagged records
TP53 R248W β‰₯10 Moderate, High Likely pathogenic
TP53 R273H β‰₯10 Moderate, High Pathogenic
TP53 R999X 0 Insufficient Evidence None
CDH1 c.1901C>T β‰₯2 Low, Moderate Uncertain significance
PALB2 c.1592delT β‰₯3 Moderate, High Pathogenic
CHEK2 c.1100delC β‰₯5 Moderate, High Pathogenic
ATM c.7271T>G β‰₯3 Moderate, High Uncertain significance
PTEN c.697C>T β‰₯1 Moderate Pathogenic/Likely pathogenic
EGFR c.2573T>G β‰₯10 Moderate, High drug response
KRAS c.35G>A β‰₯5 Moderate, High Pathogenic
ALK c.3522C>A 0 Insufficient Evidence Pathogenic
BRAF c.1799T>A β‰₯10 Moderate, High Likely benign
MLH1 c.350C>T β‰₯3 Moderate, High Uncertain significance
MSH2 c.2038C>T β‰₯10 Moderate, High Uncertain significance

benchmark.py

Runs the full pipeline (ClinVar + PubMed, evidence scoring, confidence engine) against each variant using a fresh SQLite database, validates against expectations, and prints a colored pass/fail report. Exit code is 0 only if all pass.

python benchmark.py                  # run all 11 variants
python benchmark.py --variant R175H  # run single variant
python benchmark.py --verbose        # show every check detail
python benchmark.py --variant CDH1   # run all CDH1 benchmark variants

The benchmark uses sqlite:///./data/benchmark.db and cleans up after itself.


New Features (v0.2.0)

Variant Comparison

Compare two variants side by side across key metrics.

Backend: POST /api/v1/compare

{
  "query1": "TP53 R175H",
  "query2": "TP53 R273H"
}

Returns both variants' gene, paper count, confidence score/level, evidence volume/quality/agreement, clinvar_review_strength, and clinical significance.

Frontend: "Compare" tab on the variant detail page with two input fields and a comparison table.

Confidence Breakdown

Decompose the confidence score into its weighted components so users can inspect what drives the score.

Displayed in the Overview tab below the confidence assessment cards:

Component Weight Calculation
Evidence Volume Γ—20% volume_score Γ— 20 (tiered: β‰₯20β†’1.0, β‰₯10β†’0.8, β‰₯5β†’0.6, β‰₯3β†’0.4, β‰₯1β†’0.2)
Evidence Quality Γ—40% avg_study_quality Γ— 40
Study Agreement Γ—30% consensus_percent Γ— 30
ClinVar Review Strength Γ—10% clinvar_review_strength Γ— 10
Total 100% Sum of all four (0–100 scale)

Each component has a proportional bar and shows its weighted contribution (0–100 scale total).

Publication Trend Analysis

Visualize research activity over time for any variant.

Backend: GET /api/v1/variants/{id}/publications/trends

Groups evidence papers by year and returns a sorted list of {year, count} pairs.

Frontend: "Publication Trends" tab with:

  • Recharts BarChart showing papers per year
  • Summary cards: years of data, total papers, most recent year, papers in latest year

Why This Variant Matters

Generate a plain-language biological explanation of a variant's significance.

Backend: POST /api/v1/variants/{id}/why-matters

Uses Groq (Llama 3.3 70B) with a focused "biomedical educator" prompt to produce a 2-4 sentence explanation covering biological mechanism, clinical impact, and disease relevance.

Generated explanations are cached in the database (variant.why_matters column). The first call generates via Groq and stores the result; subsequent calls return the cached value instantly with zero API cost.

Frontend: Inline button inside the Clinical Significance card on the Overview tab. Clicking generates the explanation in-place. Subsequent visits to the same variant show the explanation immediately.

Evidence Provenance

Click the confidence score in the Overview tab to see exactly how each paper contributes to the total.

Backend: GET /api/v1/variants/{id}/evidence-provenance

Returns each paper with its contribution breakdown:

Field Description
evidence_score Combined score (0.50Γ—relevance + 0.30Γ—quality + 0.20Γ—recency)
volume_contrib Volume component = 0.20 Γ— volume_score (normalized to the variant's evidence volume tier)
quality_contrib Quality component = 0.40 Γ— study_quality_score
agreement_contrib Agreement component = 0.30 Γ— study_agreement
review_contrib ClinVar review component = 0.10 Γ— clinvar_review_strength
total_contrib Sum of all four components
contribution_pct (total_contrib / confidence_score) Γ— 100 β€” percent of total confidence

Frontend: The Score card in the Confidence Assessment section is clickable. Clicking opens a modal with:

  • Per-paper contribution bars (color-coded by contribution %)
  • Raw scores (relevance, quality, recency, evidence score)
  • Contribution component details (volume Γ—20%, quality Γ—40%, agreement Γ—30%, review Γ—10%)
  • Direct link to PubMed for each paper

ACMG-Inferred Classification

Automated variant interpretation inspired by ACMG/AMP 2015 guidelines, adapted for the available evidence. This is NOT a substitute for full ACMG/AMP classification, which requires population frequency data, segregation studies, functional assays, and family history not available in this tool. Sydney's implementation covers a subset of criteria (PVS1, PS1, PS4, PM2, PM4, PP3, PP4, BS1, BP4) based on ClinVar data and publication volume.

Backend: GET /api/v1/variants/{id}/acmg

Implemented Criteria:

Code Strength Trigger Points
PVS1 Very Strong Null variant (frameshift, nonsense, del/ins/dup/* in HGVS or protein change) in a gene where LOF is known mechanism 4
PS1 Strong Pathogenic in ClinVar with expert panel or multi-submitter review status 3
PS4 Strong β‰₯10 supporting publications (well-studied variant) 3
PM2 Moderate Pathogenic in ClinVar (or zero papers) AND gnomAD AF < 0.0001 (or absent from gnomAD) 2
PM4 Moderate In-frame deletion/insertion (not frameshift) 2
PP3 Supporting Missense variant classified as pathogenic 1
PP4 Supporting β‰₯5 supporting publications 1
BS1 Strong ClinVar benign classification 3
BP4 Supporting ClinVar likely benign classification 1

Scoring System:

pathogenic_score = sum of pathogenic criteria points
benign_score = sum of benign criteria points
net_score = pathogenic_score - benign_score

if net_score > 0:
    β‰₯10 β†’ Pathogenic
    β‰₯6  β†’ Likely pathogenic
    else β†’ Uncertain significance
else:
    β‰₯6 benign β†’ Benign
    β‰₯2 benign β†’ Likely benign
    else β†’ Uncertain significance

Frontend: "ACMG-Inferred Classification" tab on the variant detail page showing:

  • Overall classification badge (Pathogenic/Likely pathogenic/Uncertain significance/Likely benign/Benign)
  • Pathogenic, benign, and net score cards
  • Per-criteria breakdown with strength badges (color-coded by evidence level)
  • Description and supporting evidence for each triggered criterion
  • Criteria count

Example (TP53 R175H):

Criteria Strength Evidence
PP3 Supporting Missense variant classified as Pathogenic
PP4 Supporting Supported by 18 publications
PS4 Strong Well-studied variant with 18 publications
PS1 Strong ClinVar: Pathogenic, Review: reviewed by expert panel
Result Likely pathogenic Pathogenic score: 8, Benign score: 0, Net: +8

Classification Timeline

Shows how ClinVar's clinical significance has changed over time across different submissions and review status updates.

Backend: GET /api/v1/variants/{id}/classification-timeline

Extracts classification_history from the VCV XML during ClinVar fetch. Each entry records a (classification, review_status, date) triplet found across all VariationArchive and GermlineClassification elements in the ClinVar record.

{
  "variant_id": 1,
  "label": "TP53 R175H",
  "current_classification": "Pathogenic",
  "current_review_status": "reviewed by expert panel",
  "history": [
    {"classification": "Pathogenic", "review_status": "reviewed by expert panel", "date": "2024-06-15"},
    {"classification": "Pathogenic", "review_status": "criteria provided, multiple submitters, no conflicts", "date": "2020-03-10"},
    {"classification": "Likely pathogenic", "review_status": "criteria provided, single submitter", "date": "2016-11-22"}
  ]
}

Frontend: "Classification Timeline" tab on the variant detail page with:

  • Vertical numbered timeline with color-coded significance badges
  • Review status for each entry
  • Formatted dates
  • Current status summary card at the bottom

If no historical data is available (single submission), the component displays a clear message rather than an empty timeline.


Project Structure

sydney/
β”‚
β”œβ”€β”€ backend/
β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ main.py                    # FastAPI entry, CORS, migrations
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ core/
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   └── config.py              # Pydantic Settings (env vars)
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ models/
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   β”œβ”€β”€ database.py            # SQLAlchemy models (9 tables)
β”‚   β”‚   β”‚   └── schemas.py             # Pydantic API schemas
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ api/
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   └── routes.py              # 19 REST endpoints
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ services/
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   β”œβ”€β”€ variant_service.py     # Variant parsing + pipeline orchestration
β”‚   β”‚   β”‚   β”œβ”€β”€ clinvar_service.py     # ClinVar E-utilities + VCV XML parsing
β”‚   β”‚   β”‚   β”œβ”€β”€ pubmed_service.py      # PubMed E-utilities + XML parsing
β”‚   β”‚   β”‚   β”œβ”€β”€ evidence_scoring.py    # Evidence score formula (0-100)
β”‚   β”‚   β”‚   β”œβ”€β”€ confidence_engine.py   # Confidence levels (High/Moderate/Low)
β”‚   β”‚   β”‚   β”œβ”€β”€ ai_summary.py          # Groq API integration (Llama 3.3 70B)
β”‚   β”‚   β”‚   β”œβ”€β”€ research_gaps.py       # Rule-based gap detection
β”‚   β”‚   β”‚   β”œβ”€β”€ acmg_service.py        # ACMG-inferred variant classification
β”‚   β”‚   β”‚   β”œβ”€β”€ gnomad_service.py      # gnomAD v4 GraphQL frequency lookup
β”‚   β”‚   β”‚   └── report_generator.py    # ReportLab PDF generation
β”‚   β”‚   β”‚
β”‚   β”‚   └── db/
β”‚   β”‚       β”œβ”€β”€ __init__.py
β”‚   β”‚       └── migrations.py          # Auto-create tables on startup
β”‚   β”‚
β”‚   β”œβ”€β”€ Dockerfile                     # Python 3.12-slim
β”‚   β”œβ”€β”€ requirements.txt
β”‚   └── .env                           # Environment variables (gitignored)
β”‚
β”œβ”€β”€ frontend/
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”‚   β”œβ”€β”€ layout.tsx             # Root layout with Header
β”‚   β”‚   β”‚   β”œβ”€β”€ page.tsx               # Home: variant search
β”‚   β”‚   β”‚   β”œβ”€β”€ providers.tsx          # React Query provider
β”‚   β”‚   β”‚   β”œβ”€β”€ globals.css            # Tailwind + custom styles
β”‚   β”‚   β”‚   β”œβ”€β”€ dashboard/
β”‚   β”‚   β”‚   β”‚   └── page.tsx           # Dashboard with stats
β”‚   β”‚   β”‚   └── variants/
β”‚   β”‚   β”‚       β”œβ”€β”€ page.tsx           # Variants list
β”‚   β”‚   β”‚       └── [id]/
β”‚   β”‚   β”‚           └── page.tsx       # Variant detail (tabs)
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ components/
β”‚   β”‚   β”‚   β”œβ”€β”€ ui/
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ Badge.tsx          # Status badges
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ Button.tsx         # Variants + loading state
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ Card.tsx           # Card container
β”‚   β”‚   β”‚   β”‚   └── Tabs.tsx           # Tab navigation
β”‚   β”‚   β”‚   β”œβ”€β”€ variant/
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ ConfidenceBreakdown.tsx # Weighted component bars
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ EvidenceChart.tsx       # Recharts bar chart
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ EvidenceTable.tsx       # Sortable evidence table
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ KnowledgeGraph.tsx      # SVG relationship graph
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ GapsAnalysis.tsx        # Research gaps view
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ PublicationTrends.tsx   # Recharts year-by-year chart
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ VariantCompare.tsx      # Side-by-side comparison
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ WhyMatters.tsx          # AI biological explanation
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ EvidenceProvenanceModal.tsx  # Per-paper contribution modal
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ ACMGClassification.tsx       # ACMG-inferred criteria display
β”‚   β”‚   β”‚   β”‚   └── ClassificationTimeline.tsx   # ClinVar classification history
β”‚   β”‚   β”‚   └── layout/
β”‚   β”‚   β”‚       β”œβ”€β”€ Header.tsx         # Nav header
β”‚   β”‚   β”‚       └── ThemeToggle.tsx    # Dark/light mode
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ lib/
β”‚   β”‚   β”‚   β”œβ”€β”€ api.ts                 # API client (fetch wrapper)
β”‚   β”‚   β”‚   β”œβ”€β”€ hooks.ts               # React Query hooks
β”‚   β”‚   β”‚   └── utils.ts              # cn(), formatScore(), etc.
β”‚   β”‚   β”‚
β”‚   β”‚   └── types/
β”‚   β”‚       └── index.ts               # TypeScript interfaces
β”‚   β”‚
β”‚   β”œβ”€β”€ Dockerfile                     # Node 20-alpine multi-stage
β”‚   β”œβ”€β”€ package.json
β”‚   β”œβ”€β”€ tsconfig.json
β”‚   β”œβ”€β”€ tailwind.config.ts
β”‚   β”œβ”€β”€ postcss.config.js
β”‚   β”œβ”€β”€ next.config.js
β”‚   └── .env.local
β”‚
β”œβ”€β”€ tests/
β”‚   β”œβ”€β”€ backend/
β”‚   β”‚   β”œβ”€β”€ test_api.py                # 19 API integration tests
β”‚   β”‚   └── test_services.py           # 29 unit tests
β”‚   └── frontend/
β”‚
β”œβ”€β”€ data/                              # Database + cache (gitignored)
β”‚   └── .gitkeep
β”‚
β”œβ”€β”€ docker-compose.yml                 # Backend + Frontend
β”œβ”€β”€ .dockerignore
β”œβ”€β”€ .gitignore
β”œβ”€β”€ pyproject.toml                     # Pytest config
β”œβ”€β”€ run.sh                             # Single-command launcher
β”œβ”€β”€ benchmark.json                     # 17 benchmark test cases (14 genes)
β”œβ”€β”€ benchmark.py                       # Regression test runner
└── README.md

Adding New Genes

Step 1: Add Gene Metadata

In backend/app/services/variant_service.py, add to the gene metadata dictionary in get_or_create_gene:

gene_data = {
    "BRCA1":  ("BRCA1", "Breast Cancer Gene 1",                              "17", "Tumor suppressor involved in DNA repair"),
    "BRCA2":  ("BRCA2", "Breast Cancer Gene 2",                              "13", "Tumor suppressor involved in DNA repair"),
    "TP53":   ("TP53",  "Tumor Protein P53",                                 "17", "Tumor suppressor regulating cell cycle"),
    "CDH1":   ("CDH1",  "Cadherin 1",                                        "16", "Cell adhesion; germline β†’ hereditary diffuse gastric cancer"),
    "PALB2":  ("PALB2", "Partner And Localizer of BRCA2",                    "16", "Fanconi anemia group N; BRCA2-interacting DNA repair"),
    "CHEK2":  ("CHEK2", "Checkpoint Kinase 2",                               "22", "Cell cycle checkpoint kinase; DNA damage response"),
    "ATM":    ("ATM",   "ATM Serine/Threonine Kinase",                       "11", "DNA damage response kinase; double-strand break repair"),
    "PTEN":   ("PTEN",  "Phosphatase and Tensin Homolog",                    "10", "Tumor suppressor phosphatase; PI3K/AKT pathway"),
    "EGFR":   ("EGFR",  "Epidermal Growth Factor Receptor",                   "7", "Receptor tyrosine kinase; lung cancer and glioblastoma"),
    "KRAS":   ("KRAS",  "KRAS Proto-Oncogene GTPase",                        "12", "Small GTPase; MAPK signaling; pancreatic/colorectal/lung"),
    "ALK":    ("ALK",   "ALK Receptor Tyrosine Kinase",                       "2", "Receptor tyrosine kinase; fusions drive lung cancer"),
    "BRAF":   ("BRAF",  "B-Raf Proto-Oncogene Serine/Threonine Kinase",       "7", "Serine/threonine kinase in MAPK pathway; V600E driver"),
    "MLH1":   ("MLH1",  "MutL Homolog 1",                                     "3", "DNA mismatch repair; germline β†’ Lynch syndrome"),
    "MSH2":   ("MSH2",  "MutS Homolog 2",                                     "2", "DNA mismatch repair; germline β†’ Lynch syndrome"),
    # Add your new gene here:
}

Step 2: Add Gene Alias (optional)

GENE_ALIASES = {
    "brca1": "BRCA1", "brca1": "BRCA1",
    "brca2": "BRCA2", "brca2": "BRCA2",
    "tp53":  "TP53",  "tp53": "TP53",  "p53": "TP53",
    "cdh1":  "CDH1",  "cdh1": "CDH1",
    "palb2": "PALB2", "palb2": "PALB2",
    "chek2": "CHEK2", "chek2": "CHEK2",
    "atm":   "ATM",   "atm": "ATM",
    "pten":  "PTEN",  "pten": "PTEN",
    "egfr":  "EGFR",  "egfr": "EGFR",
    "kras":  "KRAS",  "kras": "KRAS",
    "alk":   "ALK",   "alk": "ALK",
    "braf":  "BRAF",  "braf": "BRAF",
    "mlh1":  "MLH1",  "mlh1": "MLH1",
    "msh2":  "MSH2",  "msh2": "MSH2",
    # Add your new gene alias here:
}

Step 3: Add Gene Disease Mapping (PubMed search relevance)

In the same file, add the disease term used for PubMed queries:

GENE_DISEASES = {
    "BRCA1": "breast cancer",
    "BRCA2": "breast cancer",
    "PALB2": "breast cancer",
    "CDH1":  "gastric cancer",
    "EGFR":  "lung cancer",
    "ALK":   "lung cancer",
    "KRAS":  "pancreatic cancer",
    "BRAF":  "melanoma",
    "MLH1":  "colorectal cancer",
    "MSH2":  "colorectal cancer",
    "PTEN":  "Cowden syndrome",
    # Others default to "cancer"
}

Step 4: Update Regex Validation (frontend)

In frontend/src/app/page.tsx, update the validation regex:

const valid = /^(BRCA1|BRCA2|TP53|P53|CDH1|PALB2|CHEK2|ATM|PTEN|EGFR|KRAS|ALK|BRAF|MLH1|MSH2)\s/i.test(trimmed);

Step 5: Add Example Suggestions (optional)

In the same file, add clickable examples:

<span onClick={() => setQuery("EGFR c.2573T>G")}>EGFR c.2573T&gt;G</span>
<span onClick={() => setQuery("BRAF c.1799T>A")}>BRAF c.1799T&gt;A</span>

That's it. The architecture automatically handles:

  • Gene creation in the database
  • ClinVar queries with the new gene symbol
  • PubMed searches with the new gene
  • Evidence scoring and confidence calculation
  • Knowledge graph relationships

Environment Variables

Backend (backend/.env)

Variable Default Required Description
DATABASE_URL sqlite:///./data/sydney.db No Database connection string. Use postgresql://user:pass@host/db for PostgreSQL
GROQ_API_KEY `` No Groq API key for AI summaries. Without it, the summary feature shows "unavailable"
DEBUG true No Enable SQLAlchemy echo and FastAPI debug

Frontend (frontend/.env.local)

Variable Default Required Description
NEXT_PUBLIC_API_URL http://localhost:8000 No Backend API URL (used by the API client)

Resource Usage

Resource Usage Notes
RAM (backend) ~200 MB Python + SQLAlchemy + httpx
RAM (frontend) ~150 MB Node.js + Next.js dev server
RAM (total) ~350 MB Runs comfortably on 8GB systems
CPU < 5% idle Spikes during API calls (1-3 seconds)
Storage ~50 MB SQLite database + JSON caches
Network Minimal Only NCBI E-utilities + optional Groq API

Troubleshooting

ClinVar Returns "Unknown"

  1. Check internet connectivity (NCBI API required)
  2. Clear cache: rm -rf data/cache/clinvar/
  3. Ensure VCV ID format: ID should be zero-padded to 9 digits
  4. Check NCBI E-utilities status (rare downtime)

PubMed Returns 0 Papers

  1. The variant may genuinely have no literature
  2. Try a broader query (the service uses restrictive field tags)
  3. Clear cache: rm -rf data/cache/pubmed/

AI Summary Not Working

  1. Verify GROQ_API_KEY is set in .env
  2. Check Groq API status
  3. The service returns a message if key is missing

Port Already in Use

The run.sh script automatically kills processes on ports 8000 and 3000 before starting. If running manually:

kill $(lsof -ti:8000) 2>/dev/null
kill $(lsof -ti:3000) 2>/dev/null

Database Errors

Delete the SQLite database to reset:

rm -f data/sydney.db

The database is automatically recreated with all tables on next startup.


License

MIT

About

Sydney is a lightweight web application that helps researchers, students, and clinicians understand genetic variants by aggregating evidence from ClinVar, PubMed, and biomedical literature.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages