AI-Based Cough Acoustic Analysis for Early Respiratory Disease Screening
-orange){kind=icon}
Team Auscultate — Aryan Verma (B.Tech AI) & Arfa Alam (B.Tech Civil Engineering)
🔗 Live Demo · Screening Tool · Analytics Dashboard
Note: hosted on Render's free tier — the app may take 30–60s to wake up on first load after inactivity.
- Overview
- Live Demo
- The Problem
- Our Solution
- Key Value Points
- How It Works
- Results
- Explainability
- Data Quality Experiment
- Tech Stack
- Project Structure
- Team Contributions
- Setup & Installation
- Running the App
- Testing & Validation
- Limitations
- Future Work
- Disclaimer
CoughSense is a machine-learning and deep-learning system that screens for likely respiratory conditions (such as COVID-19) from a short cough recording. It needs nothing more than a smartphone microphone — no lab, no clinic visit, no cost. It is a screening aid, not a diagnosis: it answers the narrow, high-value question "should this person get tested?"
The project deliberately implements both classical ML and deep learning side by side and compares them honestly — a strong demonstration of how the two approaches behave on small, real clinical data.
The full app — custom frontend + FastAPI backend — is deployed together on Render:
| Page | What it does |
|---|---|
| Home | Project overview: problem, approach, results, team |
| Screening Tool | Upload or record a cough → live prediction from all 3 models |
| Dashboard | Dataset composition, per-fold CV results, confusion matrices, feature importance |
The deployed version is built from the self-contained deploy/ folder (its own copy of
serve.py, frontend/, models/, and a minimal src/), so it can be redeployed anywhere that runs a
Dockerfile — Render, Fly.io, HuggingFace Docker Spaces, etc.
Render's free tier spins the service down after inactivity — the first request after a while may take 30–60 seconds to wake up. Subsequent requests are fast.
Access to respiratory screening is limited in low-resource settings: clinics are far, tests cost money, and results take time. Yet the cough itself carries information — clinicians have long used cough character (wet vs. dry, productive vs. barking) as a diagnostic cue. If a machine can learn those same acoustic patterns, screening becomes as accessible as a phone call.
A pipeline that takes a cough clip and returns a prediction from three models, with confidence scores and a clear medical disclaimer, exposed through both a REST API and a browser interface:
- Feature extraction — MFCCs, spectral features, zero-crossing rate, RMS energy (for classical ML); log-mel spectrograms (for the CNN).
- Three models — Random Forest, XGBoost, and a compact CNN — trained and cross-validated.
- Explainability — SHAP analysis showing why the model predicts what it does.
- Next-steps guidance — a result-specific panel with general public-health guidance (get a confirmatory test, isolate, monitor symptoms, when to seek emergency care) — always deferring to a real test and a healthcare professional, never prescribing treatment.
- Web app — a live screening tool + an analytics dashboard, deployed together (frontend + backend in one container).
- Real-world impact — cheap, instant respiratory pre-screening for places with no easy lab access.
- Underexplored modality — audio biomarkers get far less attention than image or text ML, despite solid clinical grounding in cough acoustics.
- Honest science — every number is 5-fold cross-validated and reported with its standard deviation, never a single lucky split.
- Explainable, not a black box — SHAP shows which acoustic features drive each decision, and they match clinical intuition (cough timbre).
- ML vs DL comparison — demonstrates a real, well-documented tradeoff: on small clinical data, gradient-boosted trees beat deep learning.
- A genuine data-quality experiment — we tested scaling the data and learned why quality beats quantity in medical ML (details below).
- Actionable, responsible guidance — a result triggers general public-health next steps (get tested, isolate, monitor, when to seek emergency care), not a diagnosis or treatment prescription.
- Deployable — runs fully locally on free tooling; FastAPI backend + static frontend.
cough.mp3 ──► feature extraction ──► ┌── Random Forest ─┐
(MFCC / spectral) │ │
──► mel-spectrogram ──► ├── XGBoost ──────┤──► prediction + confidence
│ │ (+ SHAP explanation)
└── CNN ───────────┘
- Audio is resampled to 22.05 kHz and standardized to a fixed length.
- Two representations are computed: a 37-value statistical feature vector (for the tree models) and a 128×87 log-mel spectrogram (for the CNN).
- Each model predicts COVID vs. healthy; the API returns all three with confidence scores.
- Training data is expanded 4× with augmentation (noise, pitch-shift, time-shift) — applied only to the training split, after the train/test partition, to avoid leakage.
5-fold stratified cross-validation on the Virufy clinical dataset (121 clips: 48 COVID, 73 healthy):
| Model | Accuracy | Std Dev | Type |
|---|---|---|---|
| XGBoost | 85.1% | ±6.2% | Classical ML (best) |
| Random Forest | 83.5% | ±6.9% | Classical ML |
| CNN (augmented) | 78.5% | ±9.3% | Deep Learning |
Always cite the number with its spread — e.g. "85.1% ± 6.2% (5-fold CV)". On a 121-clip dataset a single train/test split is too noisy to trust; cross-validation is the honest metric.
Why the CNN trails the tree models: with ~100 training clips, a CNN can't learn robust spectrogram patterns the way it would with thousands of samples. Hand-crafted MFCC features inject decades of audio-engineering knowledge, letting tree models generalize from far less data — a well-documented tradeoff in applied ML, not a bug.
We apply SHAP (SHapley Additive exPlanations) to the XGBoost model to quantify each feature's contribution. The analysis confirms that MFCC-based timbral features carry the strongest signal — which aligns with clinical intuition: the "wet vs. dry" quality a doctor listens for is exactly what these coefficients encode. The model is learning something real, not a spurious artifact.
Figures (in reports/figures/): SHAP summary & bar plots, ROC curves (XGBoost AUC ≈ 0.93),
confusion matrices.
We tested whether more data would help by scaling from 121 clinical clips to 321 using COUGHVID (a 30,000-clip crowdsourced corpus). Counter-intuitively, accuracy dropped from 85% to ~65%.
Investigation showed why: COUGHVID labels are self-reported and inherently noisy, while Virufy labels
are laboratory-confirmed. Even after quality filters (cough_detected ≥ 0.8, SNR sorting) the recovery
was small. We also tried a Random-Forest + XGBoost ensemble and an enriched 102-feature set — neither beat
the simple 37-feature XGBoost.
Lesson: in medical ML, data quality beats quantity. The final system uses the clean clinical data.
This experiment is itself a strength — it shows real experimentation, investigation, and a reasoned engineering decision rather than blindly stacking data.
| Layer | Tools |
|---|---|
| Language | Python 3.11 |
| Classical ML | scikit-learn (Random Forest), XGBoost |
| Deep Learning | PyTorch (CNN) |
| Audio | librosa, soundfile, ffmpeg |
| Explainability | SHAP |
| Backend | FastAPI + Uvicorn |
| Frontend | HTML / CSS / JavaScript (Web Audio API) |
| Visualization | matplotlib |
cough-detect/
├── assets/ # logo
├── data/
│ ├── raw/ # cough clips, one folder per class (covid/, healthy/)
│ └── processed/ # extracted features + spectrograms
├── models/ # trained models (RF, XGBoost, CNN)
├── frontend/
│ ├── home.html # project overview / landing page
│ ├── index.html # live screening tool
│ ├── dashboard.html # analytics dashboard
│ └── favicon.svg
├── deploy/ # self-contained copy for Docker deployment (Render, HF Spaces, etc.)
│ ├── Dockerfile
│ ├── serve.py # backend + serves the frontend together
│ ├── requirements.txt
│ ├── frontend/ # copy of the 3 pages above
│ ├── models/ # copy of the trained models
│ └── src/ # copy of features.py + dl_model.py
├── reports/
│ ├── CoughSense_Technical_Report.docx
│ ├── figures/ # SHAP, ROC, confusion matrices
│ └── test_log.csv # validation log
├── src/
│ ├── features.py # feature extraction
│ ├── build_dataset.py # build feature datasets
│ ├── ml_baseline.py # train RF + XGBoost
│ ├── dl_model.py # CNN architecture + training
│ ├── augment.py # audio augmentation
│ ├── build_augmented_dataset.py
│ ├── dl_model_augmented.py # train CNN on augmented data
│ ├── cross_validate.py # 5-fold CV for all models
│ ├── ensemble.py # RF + XGBoost ensemble experiment
│ ├── enhanced_features.py # richer-feature experiment
│ ├── explain.py # SHAP + ROC + confusion plots
│ ├── test_log.py # testing & validation log
│ ├── serve.py # FastAPI inference API
│ ├── download_more_data.py # optional COUGHVID downloader
│ ├── reset_coughvid.py # revert to clinical-only data
│ └── run_all.py # runs the whole pipeline in order
├── requirements.txt
└── README.md
Aryan Verma — Machine Learning & Deep Learning lead. Audio feature extraction (MFCCs, spectral features), Random Forest / XGBoost / CNN model design and training, 5-fold cross-validation, data augmentation, ensemble and enhanced-feature experiments, SHAP explainability, and the FastAPI inference backend.
Arfa Alam — Frontend, documentation & validation. Web interface (screening tool + analytics dashboard UI/UX), project documentation (technical report and README), testing & validation (running cough samples through the system and logging predictions), and dataset organization (sorting and cleaning cough clips into class folders).
# 1. Virtual environment (3.11 — some libs don't build cleanly on 3.13/3.14 yet)
py -3.11 -m venv venv
.\venv\Scripts\Activate.ps1
# If activation is blocked: Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
# 2. Dependencies
pip install -r requirements.txt
# 3. ffmpeg (needed to decode .mp3/.webm). If you don't have it and winget is unavailable:
pip install imageio-ffmpeg
python -c "import imageio_ffmpeg,shutil,os; exe=imageio_ffmpeg.get_ffmpeg_exe(); d=os.path.dirname(exe); shutil.copy(exe, os.path.join(d,'ffmpeg.exe')); print('ffmpeg ready at', d)"
# then add that printed folder to PATH for the session:
# $env:PATH = "<that folder>;" + $env:PATHpip install -r requirements.txt # use a venv
cd src
python run_all.py # runs the whole pipeline in orderrun_all.py chains every stage (features → train RF/XGBoost → augment → train CNN → cross-validate →
SHAP/plots). Run stages individually any time — see src/.
The web interface needs two terminals.
Terminal 1 — backend (API):
cd src
uvicorn serve:app --reload --port 8000
# wait for "Application startup complete"Terminal 2 — frontend:
cd frontend
python -m http.server 5500Open http://localhost:5500 in your browser. Look for the API ONLINE badge (top-right). Upload or record a cough to get a live prediction; click Dashboard for the analytics view.
Windows note: activate the venv in each new terminal and re-apply the ffmpeg PATH line if audio decoding fails.
Run a batch of clips through all three models and log the predictions:
cd src
python test_log.py --n 20Prints a per-clip table (actual vs. each model's prediction vs. consensus) and saves
reports/test_log.csv. On a 20-clip sample, RF and XGBoost each scored 90% and the 3-model consensus
90% — consistent with the cross-validated results.
- Small, single-source dataset — not yet validated across populations, devices, or other respiratory conditions. Results are a proof of concept for the approach, not a clinical-grade claim.
- Binary scope — currently COVID vs. healthy; more conditions need more labeled audio.
- Screening, not diagnosis — the tool detects acoustic patterns, not disease. The medical disclaimer must stay in any demo.
- Larger, multi-source clinical datasets to close the CNN gap.
- Multimodal fusion — combine cough audio with a short self-reported symptom form.
- Grad-CAM visualizations of the spectrogram regions the CNN attends to.
- Extend to additional respiratory conditions (asthma, bronchitis, TB).
- A conversational assistant layer on top of the current next-steps guidance, for more personalized (but still non-prescriptive) follow-up questions.
CoughSense is a research prototype and screening aid, not a medical device and not a diagnosis. It must not be used to make health decisions. Always consult a qualified healthcare professional.
Team Auscultate — CoughSense — Not for clinical use