ONTEXA V2.0 - AI-Powered Biomedical Data Discovery Platform

Status: ✅ Ready for Testing

All core components have been created and dependencies installed. Services are ready to start.

What's Been Completed

Phase 1: Environment Setup ✅

• Node.js v24.5.0 installed
• Python 3.11.14 installed
• Go 1.23.11 verified
• Rust 1.94.0 verified
• PostgreSQL configured and running
• Ollama running with local models

Phase 2: Backend Services ✅

• Go Dataset Search Service (Port 8081)

  • Location: ingest-go/
  • Dependencies installed
  • Compiles successfully

• Rust Ranking Service (Port 8082)

  • Location: ranker-rust/
  • Built in release mode
  • Binary: ranker-rust/target/release/ranker-rust

• Python AI Orchestrator (Port 8084)

  • Location: services/ai_orchestrator/
  • Custom multi-agent framework (NO LangChain/CrewAI)
  • Dependencies installed
  • Configured to use granite3.2-vision:2b model

Phase 3: Frontend ✅

• React + Vite setup
• All dependencies installed
• UI components created
  • Search interface
  • Results display
  • Beautiful gradient design

Phase 4: Configuration ✅

• Environment variables configured
• Docker Compose file created
• Startup scripts created
• Verification script created

Current Configuration

Available Ollama Models

Your system has these local models available:

• granite3.2-vision:2b (Primary LLM - 2.4GB)
• qwen2.5vl:7b (Judge LLM - 6.0GB)
• nomic-embed-text:latest (Embeddings)
• And 8 more models...

Active Services

✅ PostgreSQL (localhost:5432) ✅ Ollama (localhost:11434)

Graph Configuration

• GRAPH_SEARCH_ENABLED=true (graph-first search)
• GRAPH_RANKER_ENABLED=true (use graph ranking before Rust)
• GRAPH_CACHE_TTL=60 (seconds to cache graph queries)
• LLM_TIMEOUT=60 (seconds for local LLM calls)
• CURRENT_YEAR=2025 (used for date filters)
• ONTOLOGY_DICTS= (comma-separated JSONL dictionaries)
• ONTOLOGY_MATCHER=aho (use Aho-Corasick matcher if available)
• ONTOLOGY_MIN_TERM_LEN=3 (ignore very short terms)
• METRICS_ENABLED=true (collect latency metrics)
• METRICS_WINDOW=500 (rolling window size)
• EMBEDDINGS_ENABLED=false (store embeddings during ingest)
• VECTOR_SEARCH_ENABLED=false (enable vector search in Neo4j)
• EMBEDDING_MODEL=nomic-embed-text (Ollama embedding model)
• EMBEDDING_MAX_CHARS=4000 (truncate text for embeddings)
• PUBMED_EFETCH_ENABLED=true (fetch abstract + conclusion from PubMed XML)
• PMC_EFETCH_ENABLED=false (fetch open-access full text from PMC)
• PMC_EFETCH_LIMIT=20 (max PMC full texts per run)
• ONTOLOGY_MATCH_MAX_CHARS=12000 (truncate long text for ontology matching)
• PAPER_FT_INDEX=paper_fulltext_v3 (full-text index for papers)
• DATASET_CITATIONS_ENABLED=false (attach citing paper metadata to datasets)
• DATASET_CITATIONS_LIMIT=5 (max citing papers per dataset)
• PAPER_SECTION_BOOST=true (boost papers with query matches in abstract/conclusion/results/methods)

Services Ready to Start

⏳ Redis (localhost:6379) - Start with Docker ⏳ Neo4j (localhost:7687) - Start with Docker ⏳ Go Service (localhost:8081) ⏳ Rust Service (localhost:8082) ⏳ AI Orchestrator (localhost:8084) ⏳ React Frontend (localhost:5173) ⏳ Graph Ingest (PubMed/GEO/SRA → Neo4j)

Quick Start Guide

Option 1: Start Individual Services (Recommended for testing)

# Terminal 1: Start Go Service
cd ~/ontexa/ingest-go
export $(grep -v '^#' ../\.env | xargs)
go run main.go

# Terminal 2: Start Rust Service
cd ~/ontexa/ranker-rust
./target/release/ranker-rust

# Terminal 3: Start AI Orchestrator
cd ~/ontexa/services/ai_orchestrator
export $(grep -v '^#' ../../\.env | xargs)
python main.py

# Terminal 4: Start Frontend
cd ~/ontexa
npm run dev

Option 2: Start All Services (Requires Docker)

Note: Docker daemon needs to be running first.

cd ~/ontexa
./start-all.sh

Verify Services

cd ~/ontexa
bash verify.sh

Testing the Platform

1. Test Go Service

curl http://localhost:8081/health
curl "http://localhost:8081/search?q=Parkinson"

2. Test Rust Service

curl http://localhost:8082/health
curl -X POST http://localhost:8082/rank \
  -H "Content-Type: application/json" \
  -d '{"query": "test", "datasets": ["DS1", "DS2", "DS3"]}'

3. Test AI Orchestrator

curl http://localhost:8084/health
curl -X POST http://localhost:8084/workflow/execute \
  -H "Content-Type: application/json" \
  -d '{"question": "Find datasets about Parkinson disease"}'

3c. Graph Search (Neo4j)

curl -X POST http://localhost:8084/graph/search \
  -H "Content-Type: application/json" \
  -d '{"query": "lung adenocarcinoma", "limit": 50}'

curl -X POST http://localhost:8084/literature/meta \
  -H "Content-Type: application/json" \
  -d '{"query": "TGF-beta signaling EMT lung adenocarcinoma", "limit": 1000}'

Note: Apply infra/neo4j/schema.cypher after updates to keep full-text indexes current. Note: Enable EMBEDDINGS_ENABLED=true during ingest and VECTOR_SEARCH_ENABLED=true to activate semantic vector search.

Graph Monitoring (Neo4j)

cat /home/vsam/ontexa/infra/neo4j/monitoring.cypher | docker exec -i ontexa-neo4j-1 cypher-shell -u neo4j -p ontexa_secure_password

3d. Discovery (Meta-analysis + Datasets)

curl -X POST http://localhost:8084/discovery/execute \
  -H "Content-Type: application/json" \
  -d '{"hypothesis": "TGF-beta signaling promotes EMT in lung adenocarcinoma", "dataset_limit": 50, "paper_limit": 1000}'

3e. Latency Metrics

curl http://localhost:8084/metrics/latency

3f. Latency Alert (Cron)

chmod +x /home/vsam/ontexa/scripts/schedule-metrics.sh
THRESHOLD_MS=1000 PATH_TO_CHECK=/discovery/execute /home/vsam/ontexa/scripts/schedule-metrics.sh

3g. Ingest Cron (Daily)

chmod +x /home/vsam/ontexa/scripts/schedule-ingest.sh
/home/vsam/ontexa/scripts/schedule-ingest.sh

3b. Track 1: Compound Prediction (Hackathon)

Input format: protein name (expandable later).

curl -X POST http://localhost:8084/compound/predict \
  -H "Content-Type: application/json" \
  -d '{"target": "EGFR", "max_results": 10}'

Notes:

• Uses ChEMBL public API for target → activity → compound lookup.
• Lipinski drug-likeness is applied if rdkit is installed.
• If you wire external LLMs, set API keys via env vars (never hardcode).

4. Test Frontend

Open browser to: http://localhost:5173

Project Structure

~/ontexa/
├── .env                          # Environment configuration
├── docker-compose.yml            # Docker services
├── package.json                  # Node dependencies
├── vite.config.js               # Vite configuration
├── index.html                   # HTML entry point
├── src/                         # React frontend
│   ├── main.jsx
│   ├── App.jsx
│   ├── App.css
│   └── index.css
├── ingest-go/                   # Go dataset search service
│   ├── go.mod
│   └── main.go
├── ranker-rust/                 # Rust ranking service
│   ├── Cargo.toml
│   ├── src/main.rs
│   └── target/release/ranker-rust
├── services/
│   └── ai_orchestrator/         # Python AI orchestrator
│       ├── requirements.txt
│       └── main.py
│   └── graph_ingest/            # PubMed/GEO/SRA → Neo4j ingest
│       ├── README.md
│       ├── requirements.txt
│       └── main.py
├── logs/                        # Service logs
├── start-all.sh                 # Start all services
├── stop-all.sh                  # Stop all services
└── verify.sh                    # Verify services

Architecture

AI Orchestrator (Custom Implementation)

The platform uses a custom-built multi-agent orchestration system:

• LLM Interface Layer: Abstraction supporting Ollama and other backends
• Tool Registry: Dynamic tool registration and execution
• Agent Executor: Custom agent reasoning and action loop
• Orchestrator: Multi-agent workflow management

No external frameworks like LangChain or CrewAI - 100% custom code.

Data Flow

1. User Query → React Frontend
2. Frontend → AI Orchestrator (POST /workflow/execute)
3. AI Orchestrator → Agents:
   • SearchAgent → Go Service (dataset search)
   • RankingAgent → Rust Service (relevance ranking)
   • SynthesisAgent → Final result synthesis
4. Results → Frontend → User

Next Steps

Immediate (Start Services)

1. Start Docker daemon (if using Docker Compose)
2. Start individual services in separate terminals
3. Test each service endpoint
4. Access frontend at http://localhost:5173

Short Term (Integration)

1. Download Mistral-7B model: ollama pull mistral
2. Connect Go service to PostgreSQL database
3. Implement real dataset search logic
4. Connect AI Orchestrator to actual services
5. Add graph visualization with Cytoscape.js

Medium Term (Features)

1. Integrate Neo4j graph database
2. Add Qdrant vector database (optional)
3. Add Redis caching layer
4. Implement workflow timeline UI
5. Add authentication/authorization
6. Create Tauri desktop app wrapper

Long Term (Production)

1. Deploy to cloud infrastructure
2. Add monitoring and logging
3. Performance optimization
4. Load testing
5. Documentation
6. User onboarding

Troubleshooting

PostgreSQL Connection Issues

# Test connection
PGPASSWORD='ontexa_secure_password' psql -h localhost -U ontexa_user -d ontexa_db -c "SELECT 1;"

Docker Not Running

# Check Docker status
docker --version
ps aux | grep docker

# Start Docker Desktop or systemctl
sudo systemctl start docker

Port Already in Use

# Find process using port
lsof -i :8081
lsof -i :8082
lsof -i :8084

# Kill process
kill -9 <PID>

Ollama Model Not Found

# List available models
ollama list

# Pull a model
ollama pull granite3.2-vision:2b

Technology Stack

• Backend: Go, Rust, Python
• Frontend: React, Vite
• Databases: PostgreSQL, Neo4j, (Qdrant optional)
• Caching: Redis
• AI/ML: Ollama (local models), Custom orchestrator
• Desktop: Tauri (future)

License

Proprietary - ONTEXA Project

Support

For issues or questions, check logs in ~/ontexa/logs/ directory.

---

Status: Ready for Development Testing 🚀 Last Updated: 2026-01-10