Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🧬 BioGraph Intelligence

AI-powered biomedical evidence intelligence platform for literature synthesis, knowledge graph generation, and evidence-aware scientific reasoning.

Python Streamlit Ollama PubMed License


Overview

BioGraph Intelligence transforms biomedical literature into structured scientific knowledge.

Instead of only retrieving PubMed papers, the platform automatically retrieves relevant biomedical literature, extracts structured scientific claims using a local LLM, clusters related evidence into themes, scores evidence strength, detects conflicting findings, identifies research gaps, generates an AI research brief, builds an interactive biomedical knowledge graph, and enables evidence-grounded question answering through an AI Copilot.

The goal is to reduce literature review from hours of manual reading to minutes of AI-assisted evidence synthesis.


Why This Project

Biomedical literature is growing faster than researchers can manually review. Existing search tools retrieve papers, but they often do not synthesize evidence across studies, identify contradictions, or highlight research gaps.

BioGraph Intelligence was built to explore how AI systems can convert unstructured biomedical publications into structured, explainable, decision-ready evidence.


Key Capabilities

Literature Retrieval

  • PubMed search integration
  • PMID, title, journal, author, DOI, and abstract extraction
  • Query-driven biomedical evidence collection

AI Claim Extraction

Each paper is converted into a structured biomedical claim.

Example:

Subject: SGLT2 inhibitors
Relationship: REDUCES_RISK_OF
Object: kidney failure progression
Evidence Type: cohort study
Confidence: medium
Stance: support
PMID: 12345678

Evidence Theme Generation

Related claims are grouped into higher-level biomedical evidence themes.

Evidence Scoring

Evidence themes are ranked based on current support signals such as paper count, confidence, and evidence type.

Conflict Detection

The system identifies whether claims are supportive, contradictory, or inconclusive.

Research Gap Detection

The system identifies limitations such as:

  • limited evidence coverage
  • lack of longitudinal validation
  • lack of clinical trial evidence
  • unclear population-level generalizability

AI Research Brief

The platform generates a structured executive research brief covering:

  • overall evidence
  • key findings
  • current consensus
  • important limitations
  • future research directions

Knowledge Graph

Extracted claims are visualized as a biomedical knowledge graph.

Subject → Relationship → Object

AI Evidence Copilot

Users can ask follow-up questions grounded in the extracted evidence from the current analysis run.


System Architecture

Biomedical Research Question
        ↓
PubMed Retrieval
        ↓
Paper Objects
        ↓
Local LLM Claim Extraction
        ↓
Structured Claims
        ↓
Evidence Theme Aggregation
        ↓
Evidence Scoring
        ↓
Conflict Detection
        ↓
Research Gap Detection
        ↓
AI Research Brief
        ↓
Knowledge Graph
        ↓
Evidence Copilot

Project Structure

KnowledgeGraph/
├── app/
│   ├── llm/
│   │   ├── base.py
│   │   ├── factory.py
│   │   └── ollama.py
│   │
│   ├── models/
│   │   ├── paper.py
│   │   ├── claim.py
│   │   ├── evidence_theme.py
│   │   ├── research_brief.py
│   │   ├── research_gap.py
│   │   └── conflict.py
│   │
│   ├── services/
│   │   ├── pubmed.py
│   │   ├── extractor.py
│   │   ├── aggregator.py
│   │   ├── scorer.py
│   │   ├── research_brief.py
│   │   ├── gap_detector.py
│   │   ├── conflict_detector.py
│   │   ├── copilot.py
│   │   └── graph_builder.py
│   │
│   └── main.py
│
├── data/
├── docs/
├── requirements.txt
└── README.md

Tech Stack

Category Technology
Language Python
UI Streamlit
Local LLM Ollama
Literature Source PubMed
Graph Visualization PyVis
Data Modeling Pydantic
Data Processing Pandas

Engineering Highlights

Local LLM Architecture

The project was migrated from a paid API-based LLM workflow to a local Ollama-based LLM workflow.

Benefits:

  • no API credit dependency
  • local development
  • privacy-preserving inference
  • reproducible testing
  • easier iteration during development

Structured Evidence Representation

Instead of generating free-text summaries only, the system converts each paper into a structured claim object containing:

  • subject
  • relationship
  • object
  • population
  • evidence type
  • confidence
  • stance
  • PMID
  • publication metadata

This enables downstream aggregation, scoring, conflict detection, graph construction, and copilot reasoning.

Paper-Level Extraction Pipeline

Initial extraction used one large prompt for multiple papers. This worked for small runs but became unreliable with larger retrieval sizes because the local LLM ignored the required JSON format.

The pipeline was redesigned to extract claims one paper at a time.

Before:
20 papers → one large prompt → unreliable JSON

After:
Paper 1 → claim
Paper 2 → claim
Paper 3 → claim
...
Merge claims → aggregate evidence

This makes extraction slower but more reliable and scalable.


Technical Challenges and Fixes

1. API Credit Dependency

Problem

The initial system used an external LLM API for extraction and semantic grouping. During development, repeated calls quickly exhausted available credits.

Solution

The LLM layer was refactored to use Ollama locally.

Result

The project can now be developed and tested without API costs.


2. Streamlit Rerun Behavior

Problem

Streamlit reruns the entire script whenever a button is clicked. This caused the full literature retrieval and claim extraction pipeline to restart when using the Copilot.

Solution

Implemented st.session_state to persist papers, claims, evidence themes, research gaps, conflict summaries, and research briefs.

Result

Follow-up interactions reuse the previous analysis instead of rerunning the expensive pipeline.


3. Local LLM JSON Reliability

Problem

For larger prompts, the local LLM sometimes returned narrative summaries instead of valid JSON.

Solution

The extraction architecture was changed from batch extraction to paper-level extraction. The parser also includes JSON extraction, validation, and retry logic.

Result

The system became more reliable for larger literature searches.


Example Use Cases

GLP-1 and Kidney Outcomes

Example query:

How do GLP-1 receptor agonists compare with SGLT2 inhibitors for renal outcomes?

The system can retrieve relevant studies, extract comparative claims, identify evidence themes, surface research gaps, and generate a knowledge graph of drug-outcome relationships.

Alzheimer's Biomarkers

Example query:

What is the evidence supporting pTau217 as a blood-based biomarker for Alzheimer's disease?

The system can synthesize biomarker claims, identify evidence strength, and generate an AI research brief.


Current Limitations

  • Claim extraction currently relies primarily on abstracts rather than full-text papers.
  • Evidence scoring is an early prototype and does not yet fully weight sample size, study quality, or journal impact.
  • Conflict detection depends on the quality of extracted stance labels.
  • Graph visualization is optimized for moderate-sized evidence sets and will require filtering for larger corpora.
  • Ontology normalization with MeSH, UMLS, DrugBank, or MONDO is not yet implemented.

Roadmap

  • Add evidence scoring v2 with study design and sample size weighting
  • Add PDF evidence report export
  • Improve graph filtering and node type classification
  • Add MeSH/UMLS ontology normalization
  • Add temporal evidence tracking
  • Add persistent caching by PMID
  • Add Neo4j graph database backend
  • Add full-text paper ingestion
  • Add multi-query evidence comparison
  • Add deployment-ready FastAPI backend

What I Learned

This project reinforced that building useful AI systems requires more than connecting an LLM to a user interface. The most important engineering challenges were:

  • designing reliable data flow
  • converting unstructured text into structured objects
  • validating LLM outputs
  • managing application state
  • separating UI, services, models, and LLM logic
  • making evidence explainable rather than opaque

BioGraph Intelligence is an end-to-end applied AI system that combines biomedical NLP, local LLM orchestration, structured data modeling, evidence synthesis, and knowledge graph visualization.

About

AI-powered biomedical evidence intelligence platform that transforms PubMed literature into structured claims, knowledge graphs, research briefs, and evidence-grounded AI insights.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages