DrHouse: LLM-Empowered Diagnostic Reasoning with Sensor Data and Expert Knowledge

Bufang Yang†   Siyang Jiang†   Lilin Xu  Kaiwei Liu  Hai Li  Guoliang Xing 
Hongkai Chen  Xiaofan Jiang  Zhenyu Yan‡ 
The Chinese University of Hong Kong Columbia University

🏠 About

We introduce DrHouse, a novel LLM-based multi-turn consultation virtual doctor system, which incorporates three significant contributions: 1) It utilizes sensor data from smart devices in the diagnosis process, enhancing accuracy and reliability. 2) DrHouse leverages continuously updating medical knowledge bases to ensure its model remains at diagnostic standard’s forefront. 3) DrHouse introduces a novel diagnostic algorithm that concurrently evaluates potential diseases and their likelihood, facilitating more nuanced and informed medical assessments. Through multi-turn interactions, DrHouse determines the next steps, such as accessing daily data from smart devices or requesting in-lab tests, and progressively refines its diagnoses.

📂 Project Structure

DrHouse/
├─ checkpoints/
├─ data/
├─ exp_real/
├─ exp_simulation/
├─ prompt/
├─ src/
│  ├─ adaptive_retrieval/
│  ├─ baselines/
│  ├─ baseLLMs/
│  ├─ guideline_tree_retrieval/
│  ├─ simulation/
│  ├─ create_medical_database.py
│  ├─ eval_dialogues.py
│  ├─ main.py
│  ├─ semantic_filter.py
│  ├─ utils.py
├─ .gitignore
├─ environment.yml
├─ README.md
├─ requirements.txt

⚙️ Installation

1. Clone the repository:

git clone https://github.com/bf-yang/DrHouse.git
cd DrHouse

2. Install packages:

conda env create -f environment.yml
conda activate drhouse

▶️ Usage

Setup

Create the medical database:

python src/create_medical_database.py

Train semantic filter:

python src/semantic_filter.py

Consultation

Real-world data consultation:

python src/main.py --exp real --user_name cj_t2

Simulation data consultation:

• Two-agent consultation: python src/simulation/simulation_consultation.py
• Real person interactions: python src/main.py --exp simulation --user_name sim_data_ab_1

Evaluation

python src/eval_dialogues.py

User Interfaces

Web UI:

python src/mainWebUI.py

Android UI:

1. Start the server: python src/mainAndroidUIServer.py
2. Run the Android app: Interaction-Speak-Sockets-Android

🔗 Citation

If you find our work helpful, please consider citing:

@article{yang2024drhouse,
  title={DrHouse: An LLM-empowered diagnostic reasoning system through harnessing outcomes from sensor data and expert knowledge},
  author={Yang, Bufang and Jiang, Siyang and Xu, Lilin and Liu, Kaiwei and Li, Hai and Xing, Guoliang and Chen, Hongkai and Jiang, Xiaofan and Yan, Zhenyu},
  journal={Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  volume={8},
  number={4},
  pages={1--29},
  year={2024},
  publisher={ACM New York, NY, USA}
}