Nutrisense-AI

Personal disease risk intelligence, calibrated for East Africa.

Most nutrition apps were built for Western diets and have never heard of Isombe, Ibishyimbo, or Matoke. Nutrisense-AI is built differently: it lets Rwandan users photograph their everyday meals, tracks nutritional intake in real time, and predicts personalised risk scores for anaemia, type 2 diabetes, and overweight using machine learning models trained on Rwanda DHS 2019–20 + STEPS Rwanda 2012 microdata — so the predictions reflect East African realities, not Western averages.

Live app → nutrisense-seven-omega.vercel.app ML API → huggingface.co/spaces/JeanJabo/nutrisense-api Demo video → Demo

The Problem

Rwanda and East Africa face a dual burden of malnutrition. Existing tools (ZOE, DayTwo) cost USD 300–2,000 and are trained on Western food databases. Apps like MyFitnessPal lack disease prediction entirely and do not cover Rwandan staples like Isombe, Ibihaza, Ugali, or Matoke.

Disease	Rwanda prevalence (DHS 2019–20)
Anaemia (women 15–49)	~38%
Overweight	~22%
Diabetes	~3.4%

Nutrisense-AI is a free, accessible web platform that predicts personalised risk for all three conditions from daily food logs, calibrated to this population.

User Roles

The platform supports three distinct roles, each with their own portal:

Role	Portal	Access
`patient`	`/dashboard`	Log meals, view risk scores, share Connect Code
`nutritionist`	`/nutritionist`	Monitor assigned patients' logs and risk scores
`admin`	`/admin`	Create nutritionist accounts, approve/revoke access

A patient links to a nutritionist by sharing their Connect Code (their user UUID) from the Profile sheet. The nutritionist enters it in the "Add Patient" dialog — no data is shared until the patient initiates this.

Architecture

Browser (React 19 + TanStack Router)
        │
        ├── Supabase (PostgreSQL + RLS + Auth + Edge Functions)
        │       ├── profiles
        │       ├── food_logs
        │       ├── nutritionist_applications
        │       └── patient_assignments
        │
        └── HuggingFace Spaces (FastAPI)
                ├── POST /api/predict/food   ← ViT-B/16 image classifier (114 classes)
                ├── POST /api/predict/risk   ← GradientBoosting × 3 + SHAP
                └── GET  /health
                          │
                          └── HuggingFace Model Repo (weights downloaded at startup)

Layer	Technology
Frontend	React 19, TanStack Router, TailwindCSS 4, Vite
Auth & DB	Supabase (PostgreSQL + Row-Level Security)
Edge Functions	Supabase Edge Functions (Deno / TypeScript)
Food classifier	ViT-B/16 (`vit_base_patch16_224.orig_in21k`) fine-tuned on Food-101 + KenyanFood13 (114 classes)
Risk models	GradientBoostingClassifier × 3 (scikit-learn) + SHAP TreeExplainer
Nutrition lookup	`nutrition_db.py` — 114 entries, 12 nutrient fields, sourced from USDA SR Legacy + FAO/INFOODS East Africa
Training data	DHS Rwanda 2019–20 + STEPS Rwanda 2012 microdata (risk) · Food-101 + KenyanFood13 (food classifier)
Model storage	HuggingFace Model Repo `JeanJabo/nutrisense-food-model`
API	FastAPI + Uvicorn on HuggingFace Spaces
Deployment	Vercel (frontend) · HuggingFace Spaces (API)

Pages & Routes

Route	Description
`/`	Landing page — hero, features, call to action
`/login`	Email/password login + Google OAuth
`/signup`	Patient registration
`/dashboard`	Patient portal — Overview, Risk Engine, Food Lab, Trends tabs
`/nutritionist`	Nutritionist portal — patient list, risk scores, food logs
`/admin`	Admin portal — create/revoke nutritionist accounts
`/apply-nutritionist`	Self-application form for clinicians
`/find-nutritionist`	Browse approved nutritionists
`/reset-password`	Password reset (linked from email)
`/legal`	Privacy Policy & Terms of Use (12 sections)

Installation — Frontend

Prerequisites

Node.js ≥ 18
A Supabase project (free tier works)

# 1. Clone
git clone https://github.com/JaboJean/nutrisense.git
cd nutrisense

# 2. Install dependencies
npm install

# 3. Configure environment
cp .env.local.example .env.local

Edit .env.local:

VITE_SUPABASE_URL=https://your-project.supabase.co
VITE_SUPABASE_ANON_KEY=your-anon-key-here
VITE_ML_API_URL=https://jeanjabox-nutrisense-api.hf.space

# 4. Run
npm run dev
# → http://localhost:3000

Installation — ML API (local)

Prerequisites

Python ≥ 3.10

cd api

# 1. Create virtual environment
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

# 2. Install dependencies
pip install -r requirements.txt

# 3. Start server
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
# → http://localhost:8000/docs  (Swagger UI)

On first start the API downloads food_finetuned_model.pth, class_names.txt, and nutrisense_model.joblib from the HuggingFace Model Repo automatically.

Set VITE_ML_API_URL=http://localhost:8000 in .env.local to point the frontend at your local API.

Database Setup (Supabase)

Run this SQL in your Supabase SQL editor (SQL Editor → New query):

-- Profiles (one row per user)
create table profiles (
  id         uuid primary key references auth.users(id) on delete cascade,
  name       text,
  age        integer,
  sex        text check (sex in ('male','female')),
  weight_kg  numeric,
  height_cm  numeric,
  role       text default 'patient'
               check (role in ('patient','nutritionist','admin','pending_nutritionist')),
  created_at timestamptz default now()
);

-- Food logs
create table food_logs (
  id         uuid primary key default gen_random_uuid(),
  user_id    uuid references auth.users(id) on delete cascade,
  name       text not null,
  meta       text,
  meal       text,
  tag        text,
  tone       text,
  glyph      text,
  img        text,
  logged_at  timestamptz default now()
);

-- Nutritionist credential records
create table nutritionist_applications (
  id            uuid primary key default gen_random_uuid(),
  user_id       uuid references auth.users(id) on delete cascade,
  full_name     text,
  email         text,
  credential_no text,
  institution   text,
  note          text,
  status        text default 'pending' check (status in ('pending','approved','rejected')),
  reviewed_at   timestamptz,
  created_at    timestamptz default now()
);

-- Patient–nutritionist assignments
create table patient_assignments (
  nutritionist_id uuid references auth.users(id) on delete cascade,
  patient_id      uuid references auth.users(id) on delete cascade,
  created_at      timestamptz default now(),
  primary key (nutritionist_id, patient_id)
);

-- Row-Level Security
alter table profiles                enable row level security;
alter table food_logs               enable row level security;
alter table nutritionist_applications enable row level security;
alter table patient_assignments     enable row level security;

-- Patients manage their own data
create policy "Users manage own profile"
  on profiles for all using (auth.uid() = id);

create policy "Users manage own logs"
  on food_logs for all using (auth.uid() = user_id);

-- Nutritionists can read their assigned patients' logs
create policy "Nutritionists read assigned patient logs"
  on food_logs for select
  using (
    exists (
      select 1 from patient_assignments
      where nutritionist_id = auth.uid()
        and patient_id = food_logs.user_id
    )
  );

-- Nutritionists manage their own assignments
create policy "Nutritionists manage assignments"
  on patient_assignments for all using (auth.uid() = nutritionist_id);

Then go to Authentication → Email and disable "Confirm email" so the onboarding flow redirects directly to the dashboard after signup.

Edge Functions

Deploy the create-nutritionist Edge Function:

npx supabase functions deploy create-nutritionist --project-ref your-project-ref

This function handles admin-initiated nutritionist account creation using the service role key server-side.

API Reference

Method	Endpoint	Description
`GET`	`/health`	Liveness check
`POST`	`/api/predict/food`	Classify a food photo (`multipart/form-data`, field: `image`)
`POST`	`/api/predict/risk`	Predict risk scores from food logs + profile

Food prediction

POST /api/predict/food
Content-Type: multipart/form-data
Field: image (file)

Response:

{
  "name": "ugali",
  "confidence": 0.823,
  "kcal": 357,
  "protein": 7.2,
  "iron": 1.8,
  "fiber": 3.2,
  "vitC": 0.0,
  "glyph": "🫓",
  "tone": "amber",
  "tag": "Staple"
}

Confidence thresholds:

Confidence	Behaviour
< 15%	422 `not_food` — "Dish not recognised"
15–65%	Returns result with low-confidence warning
≥ 65%	Returns confirmed result

Risk prediction

POST /api/predict/risk
Content-Type: application/json

Request:

{
  "logs": [
    { "name": "ugali", "meal": "Lunch" },
    { "name": "isombe", "meal": "Lunch" }
  ],
  "profile": { "age": 24, "sex": "female", "weightKg": 58, "heightCm": 165 }
}

Response:

{
  "scores": { "anemia": 62, "diabetes": 8, "overweight": 25, "overall": 32 },
  "shap": {
    "anemia": [
      { "f": "Iron intake", "v": -0.21 },
      { "f": "Vitamin C",   "v": -0.08 }
    ],
    "diabetes":   [...],
    "overweight": [{ "f": "Body Mass Index", "v": 0.04 }, ...]
  }
}

SHAP values: positive = increases disease risk, negative = reduces risk. Top 5 features returned per disease.

ML Pipeline

Food Classifier (ViT-B/16)

Architecture: vit_base_patch16_224.orig_in21k via timm — pretrained on ImageNet-21k, fine-tuned on food dataset
Classes: 114 total — 101 from Food-101 + 13 from KenyanFood13 (ugali, pilau, chapati, mandazi, githeri, sukuma wiki, nyama choma, matoke, kukuchoma, bhaji, kachumbari, masala chips, mukimo)
Input: 224×224 RGB, normalised with ImageNet mean/std
Output: Top-1 class name + softmax confidence
Accuracy: 80.3% top-1 · 96.8% top-5 on held-out test set

Risk Models (GradientBoosting × 3)

Input features (14): age, sex, energy_kcal, protein_g, fat_g, carb_g, iron_mg, vitC_mg, vitA_mcg, fiber_g, sugar_g, calcium_mg, zinc_mg, sodium_mg
Scaling: If fewer than 3 meals are logged, nutrients are scaled to full-day estimate (scale = 3.0 / n_meals)
Training data: DHS Rwanda 2019–20 (primary) + STEPS Rwanda 2012 (supplementary)
Explainability: SHAP TreeExplainer — top-5 feature contributions per prediction

Model	AUROC	Notes
Anaemia	0.973	Haemoglobin-derived label from DHS
Overweight	0.980	BMI-derived label; score overridden with direct BMI lookup at inference
Diabetes	0.598	Proxy label (no blood glucose in training data); score overridden with glycaemic load heuristic

Post-processing overrides:

Overweight: GBM output replaced by a BMI lookup table (BMI ≥ 25 → score 48, BMI ≥ 30 → score 80, etc.) since BMI from user profile is more reliable than dietary proxies alone
Diabetes: GBM output replaced by a glycaemic load heuristic (daily sugar + net carbs) due to AUROC 0.598
All scores: Capped at 90 to prevent display of false certainty from uncalibrated GBM probabilities

Nutrition Database

nutrition_db.py maps all 114 food class names to 12 nutrient fields per typical serving:

Food-101 values: USDA SR Legacy database
East African values: FAO/INFOODS East African Food Composition Table (2012) + Kenya NFCT (KEBS 2018)

Key Design Decisions

Decision	Rationale
DHS Rwanda + STEPS Rwanda microdata	Ground-truth population data from Rwanda rather than generic Western datasets
ViT-B/16 over CNN	Self-attention captures global image context; outperforms ResNet on Food-101 benchmark (80.3% vs ~68%)
KenyanFood13 added to Food-101	Food-101 has no East African dishes; 13 additional classes cover regional staples
GradientBoosting over neural networks	Tabular data; smaller dataset; directly compatible with SHAP TreeExplainer for exact Shapley values
SHAP TreeExplainer	Per-prediction explainability — users see which specific nutrients drive each risk score
Meal-count scaling	Models trained on daily totals; `scale = max(1.0, 3.0 / n_meals)` extrapolates partial logs
BMI override for overweight	Clinical gold standard is more reliable than dietary proxy alone
Glycaemic load heuristic for diabetes	Replaces weak GBM (AUROC 0.598) with a physiologically sound score based on daily sugar + net carbs
Scores capped at 90	Raw GBM probabilities are uncalibrated; cap prevents display of false certainty
Supabase RLS	Data access enforced at the database layer, not application layer — a bug in React code cannot leak another user's data
Edge Function for account creation	Service role key must never reach the browser; privileged operations run server-side
URL-based tab state (TanStack Router)	Tab persists across page refresh and browser back button without extra state management
Optimistic updates in food log	Meal appears instantly in UI; Supabase insert runs in background with rollback on failure

Testing

Five levels of testing were applied:

Unit tests (api/test_api.py — 9 tests):

Health endpoint returns 200
Food endpoint returns correct schema
422 returned for non-food input
413 returned for images > 10 MB
Risk endpoint returns correct schema with SHAP values
CORS headers present on all responses

Validation: ML models evaluated on held-out test sets (AUROC, top-1/top-5 accuracy)

Integration: Frontend ↔ API ↔ Supabase end-to-end flows

Functional/System: Complete user journeys — signup → onboarding → log meal → view risk; admin → create nutritionist → nutritionist login

Acceptance testing (8 participants):

Metric	Result
Tasks completed unaided	82.5%
Preferred photo logging over manual entry	89%
Photo logging time vs manual entry	15s vs 60s
Mean usability score	4.1 / 5
SHAP panel interpreted correctly unaided	67%

Formal criteria:

Criterion	Target	Achieved
Response time	≤ 3s	1.2s mean · 2.6s p95
AUROC per model	≥ 0.75	2 of 3 met (diabetes: 0.598)
Cross-user data access	Denied	DB refuses query (RLS)
Explainability	100%	100% (SHAP on every prediction)
Recurring cost	USD 0	USD 0

Project Structure

nutrisense/
├── api/                              # FastAPI ML backend
│   ├── main.py                       # Routes + lifespan handler
│   ├── food_predictor.py             # ViT-B/16 inference (114 classes)
│   ├── risk_predictor.py             # GradientBoosting inference + SHAP
│   ├── nutrition_db.py               # Nutrient lookup (114 entries, 12 fields)
│   ├── test_api.py                   # 9 unit tests (UT-01 to UT-09)
│   └── requirements.txt
├── src/
│   ├── routes/                       # File-based routing (TanStack Router)
│   │   ├── index.tsx                 # Landing page
│   │   ├── dashboard.tsx             # Patient portal (4 tabs)
│   │   ├── nutritionist.tsx          # Nutritionist portal
│   │   ├── admin.tsx                 # Admin portal
│   │   ├── login.tsx
│   │   ├── signup.tsx
│   │   ├── reset-password.tsx
│   │   ├── apply-nutritionist.tsx
│   │   ├── find-nutritionist.tsx
│   │   ├── legal.tsx                 # Privacy Policy & Terms of Use
│   │   └── __root.tsx
│   ├── components/
│   │   ├── dashboard/                # Feature components
│   │   │   ├── PhotoCapture.tsx      # Photo → food log pipeline
│   │   │   ├── RiskGauges.tsx        # Animated risk score gauges
│   │   │   ├── AIInsightPanel.tsx    # SHAP-driven narrative panel
│   │   │   ├── HeroSection.tsx       # Daily summary header
│   │   │   ├── FoodLog.tsx           # Today's meal list
│   │   │   ├── LogMealSheet.tsx      # Manual meal search & log
│   │   │   ├── ProfileSheet.tsx      # Profile editor + Connect Code
│   │   │   ├── MealDetailSheet.tsx   # Per-meal nutrient detail
│   │   │   ├── TrendChart.tsx        # 7-day calorie trend chart
│   │   │   ├── Recommendations.tsx   # Diet recommendations
│   │   │   ├── OnboardingFlow.tsx    # New user setup (name/age/sex/weight/height)
│   │   │   ├── PredictionPipeline.tsx # ML pipeline diagram
│   │   │   └── sections/
│   │   │       ├── RiskEngineSection.tsx
│   │   │       ├── FoodLabSection.tsx
│   │   │       └── TrendsSection.tsx
│   │   └── ui/                       # shadcn/ui primitives (40+ components)
│   ├── hooks/
│   │   ├── useAuth.ts                # Auth state + register/login/logout/updateProfile
│   │   ├── useFoodLogs.ts            # Food log state + optimistic add/remove
│   │   ├── useAdmin.ts               # Admin actions + nutritionist management
│   │   ├── usePatients.ts            # Nutritionist patient list + assignments
│   │   ├── useProfile.ts             # UserProfile type definition
│   │   ├── useNutritionistDirectory.ts
│   │   └── use-count-up.ts           # Number animation hook
│   ├── lib/
│   │   ├── mlApi.ts                  # predictRisk() — API call + 6s timeout + mock fallback
│   │   └── supabase.ts               # Supabase client initialisation
│   └── data/
│       └── mock.ts                   # LogItem/FoodEntry types + FOOD_DATABASE + RISKS config
├── supabase/
│   └── functions/
│       └── create-nutritionist/
│           └── index.ts              # Edge Function — admin-only nutritionist account creation
└── ML/
    ├── nutrisense_training.ipynb     # Full training pipeline
    └── outputs/                      # ROC curves, SHAP plots, confusion matrices

ML Training

The notebook is in ML/nutrisense_training.ipynb. Run on Kaggle (GPU T4 or P100 recommended):

Add the dansbecker/food-101 dataset as input (Food-101)
Add KenyanFood13 dataset for East African classes
Add your HF_TOKEN as a Kaggle secret
Run end-to-end — trains ViT-B/16 on 114 classes and three GradientBoosting classifiers, then pushes weights to JeanJabo/nutrisense-food-model

Important: Risk model training requires DHS Rwanda 2019–20 + STEPS Rwanda 2012 microdata. These are restricted-licence WHO/DHS datasets and must not be committed to any public repository or redistributed in any form.

Deploying API Updates

Option 1 — Upload individual files (fast):

python push_to_hf.py --token hf_YOUR_TOKEN_HERE

Uploads risk_predictor.py, nutrition_db.py, food_predictor.py, and requirements.txt to the Space.

Option 2 — Full git subtree push:

git push space "$(git subtree split --prefix api main)":main --force

Pushes the entire api/ directory as the Space's main branch and triggers a rebuild.

Supabase Edge Functions:

npx supabase functions deploy create-nutritionist --project-ref your-project-ref

Limitations & Future Work

Current limitations:

Food classifier covers 114 classes. Rwanda-specific dishes outside Food-101 and KenyanFood13 (e.g. Ibihaza, Umutsima, Isombe) return a low-confidence best guess
Diabetes model AUROC 0.598 — proxy label used in training (no blood glucose data in DHS/STEPS); output is exploratory only
STEPS Rwanda data is from 2012; Rwanda's nutritional profile has changed since
Risk scores are uncalibrated probabilities (capped at 90, not Platt-scaled)
HuggingFace Spaces free tier sleeps after inactivity — first request has 30–60s cold-start delay

Planned future work:

Build a Rwanda-specific food image dataset covering dishes outside the current 114 classes
Audit for data leakage and report 5-fold cross-validation results
Calibrate probabilities using Platt scaling or isotonic regression
Add automatic portion-size estimation from photos
Validate prospectively against clinical biomarkers (haemoglobin, blood glucose)

Ethics & Privacy

Platform complies with Rwanda Law No. 058/2021 on Personal Data Protection
Training microdata (DHS + STEPS) held under restricted licence — never redistributed
All connections use HTTPS; passwords are never stored in plain text
Row-Level Security enforced at the PostgreSQL layer — cross-user data access is structurally impossible
The platform is not a medical device and does not provide clinical diagnoses
Full Privacy Policy & Terms of Use available at /legal

Author

JABO Jean Jacques — BSc Software Engineering, African Leadership University Supervisor: Murairi Dirac Contact: j.jabo@alustudent.com

Name		Name	Last commit message	Last commit date
Latest commit History 122 Commits
ML		ML
api		api
data		data
public		public
src		src
supabase		supabase
.env.local.example		.env.local.example
.gitattributes		.gitattributes
.gitignore		.gitignore
.prettierignore		.prettierignore
.prettierrc		.prettierrc
.vercelignore		.vercelignore
JABO Proposal_mission Capstone.pdf		JABO Proposal_mission Capstone.pdf
README.md		README.md
bun.lock		bun.lock
bunfig.toml		bunfig.toml
components.json		components.json
eslint.config.js		eslint.config.js
index.html		index.html
package-lock.json		package-lock.json
package.json		package.json
push_to_hf.py		push_to_hf.py
train_risk_local.py		train_risk_local.py
tsconfig.json		tsconfig.json
upload_to_hf.py		upload_to_hf.py
vercel.json		vercel.json
vite.config.ts		vite.config.ts

Navigation Menu

Repository files navigation

Nutrisense-AI

The Problem

User Roles

Architecture

Pages & Routes

Installation — Frontend

Prerequisites

Installation — ML API (local)

Prerequisites

Database Setup (Supabase)

Edge Functions

API Reference

Food prediction

Risk prediction

ML Pipeline

Food Classifier (ViT-B/16)

Risk Models (GradientBoosting × 3)

Nutrition Database

Key Design Decisions

Testing

Project Structure

ML Training

Deploying API Updates

Limitations & Future Work

Ethics & Privacy

Author

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

Nutrisense-AI

The Problem

User Roles

Architecture

Pages & Routes

Installation — Frontend

Prerequisites

Installation — ML API (local)

Prerequisites

Database Setup (Supabase)

Edge Functions

API Reference

Food prediction

Risk prediction

ML Pipeline

Food Classifier (ViT-B/16)

Risk Models (GradientBoosting × 3)

Nutrition Database

Key Design Decisions

Testing

Project Structure

ML Training

Deploying API Updates

Limitations & Future Work

Ethics & Privacy

Author

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages