VeriTrace

VeriTrace is a scientific claim verification system that treats an uploaded paper as the claim source, discovers related research papers from scholarly APIs, and retrieves cross-paper evidence to classify support, contradiction, or uncertainty.

Overview

This project is structured around cross-paper verification rather than same-document retrieval. Instead of acting like a generic PDF chatbot, VeriTrace focuses on:

• claim extraction from uploaded papers
• external scholarly paper discovery
• evidence retrieval across other papers
• support vs contradiction analysis
• source-grounded verification summaries
• graph-style evidence relationships

SciFact-backed dataset strategy

The intended public dataset setup is:

Corpus

• SciFact abstracts / cited scientific evidence documents

Claim set

• SciFact expert-written scientific claims

This gives the system a real benchmark-aligned structure instead of fabricated claims from random text.

System pipeline

flowchart LR
    A["Upload research paper"] --> B["Extract claim-like sentences"]
    B --> C["Query OpenAlex for related papers"]
    C --> D["Build comparison corpus from external abstracts"]
    D --> E["Chunk abstracts into passages"]
    B --> F["Claim input layer"]
    E --> G["TF-IDF evidence retrieval"]
    F --> G
    G --> H["Support / contradiction labeling"]
    H --> I["Claim-level verdicts"]
    I --> J["Evidence graph and verification dashboard"]

Loading

Current features

• .txt upload and optional .pdf upload support
• automatic claim extraction from uploaded papers
• OpenAlex, Semantic Scholar, or hybrid related paper discovery
• automatic comparison corpus construction from external paper abstracts
• linked evidence cards that point back to the referenced paper
• SciFact-style local fallback support
• TF-IDF evidence retrieval
• stance labeling as Supports, Contradicts, or Mixed / unclear
• claim-level verdicts such as Supported, Contradicted, Disputed, and Unresolved
• source-to-source evidence graph
• Streamlit verification dashboard
• quantified verification metrics

Quantified outputs

The current metric layer reports:

• number of comparison documents
• number of extracted or loaded claims
• number of passages
• number of evidence links
• average evidence links per claim
• cross-source evidence coverage
• disputed-claim rate

Tech stack

• Python
• Streamlit
• pandas
• scikit-learn
• Plotly
• PyPDF2
• requests

Repository structure

.
├── app.py
├── requirements.txt
├── data/
│   ├── external/
│   └── sample_corpus/
└── src/
    ├── dataset_adapters.py
    ├── evaluation.py
    ├── extraction.py
    ├── ingestion.py
    ├── retrieval.py
    ├── stance.py
    └── verification.py

How to run

git clone <your-repo-url>
cd claimscope
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
streamlit run app.py

External dataset setup

For the uploaded-paper workflow, VeriTrace queries OpenAlex, Semantic Scholar, or both and builds a comparison corpus from discovered research abstracts.

For a local fallback corpus, place SciFact exports in data/external/ using names like:

data/external/scifact_corpus.jsonl
data/external/scifact_claims.jsonl

The app supports .json, .jsonl, and .csv variants for both, and also accepts the raw nested SciFact download layout under data/external/data/.

If SciFact files are not present, the app falls back to the bundled sample corpus.

Resume-ready description

Built a scientific claim verification system that retrieves evidence abstracts across documents, classifies support vs contradiction, and visualizes source-grounded verification relationships in an interactive Streamlit dashboard.