LLM

PneumoLLM

• Input: Chest Radiography
• Method: finetuning adaptors
• Use a pre trained CLIP model for both text and image encoder (CLAP in audio case)
• Add adaptor, trainable prompt, classifier, three part trainable
• mask attention for prompt, unidirectional attention

A foundation model for generalizable disease detection from retinal images nature 2023

• Input: retinal images
• Method: fine-tuning entire
• curate open datasets to train two modalities 2 models
• 8 datasets, three groups of evaluation tasks, fine-tuning, including external evaluation (fine tune on A test on B)
• MAE and 4 contrastive methods
• interpretability through a tool using salient map
• some analysis on distribution shift

Large Language Models for Cuffless Blood Pressure Measurement From Wearable Biosignals

• Data: handcrafted features from ECG and PPG
• Way: prompt designing & finetuning
• context-enhanced prompts, domain knowledge + user information
• compared 10 LLMs, compared size of training data needed