Skip to content
BranchesTags

Repository files navigation

BovineID — Intelligent Cattle Biometric Platform

AI-powered cattle identification and registry system — combining deep computer vision, multi-modal biometric fusion, and a cross-platform mobile app to give every cow a unique digital identity.

Built as a high-level full-stack + AI research project with extensive computer vision, deep learning, and advanced algorithmic biometrics. This system enables farmers to register, search, and verify cattle using non-invasive muzzle-print and face biometrics — analogous to fingerprint identification for bovines.

Table of Contents

System Overview

BovineID is a vertically integrated biometric system comprising four tightly coupled layers:

Layer Description
Mobile Client (client/) React + Capacitor cross-platform app (Android/iOS/Web) for farmers
Admin Dashboard (admin-app/) Vite + React admin panel with analytics and dispute management
Backend API (server/) Express + TypeScript REST API with MongoDB, Cloudinary, Redis
AI/ML Engine (dl-api/) FastAPI + Python deep learning service with GPU acceleration

The AI pipeline processes multi-view cattle photographs through a seven-stage computer vision cascade — including YOLO detection, spoofing prevention, multi-model embedding, keypoint matching, textural feature extraction, and probabilistic score fusion — before making a final biometric identity decision.

Architecture Diagrams

1. High-Level System Architecture

graph TB
    subgraph Clients
        APP["Farmer App React + Capacitor"]
        ADMIN["Admin Dashboard React + Vite"]
    end

    subgraph "Backend Express API Node.js TypeScript"
        SERVER["Express Server Port 2424"]
        AUTH["JWT Auth Middleware"]
        RATE["Rate Limiter"]
        HELMET["Helmet CORS NoSQL Sanitize"]
        FARMER_ROUTES["Farmer Routes\n/api/auth\n/api/cattle\n/api/location\n/api/user"]
        ADMIN_ROUTES["Admin Routes\n/api/admin/auth\n/api/admin/cattle\n/api/admin/disputes\n/api/admin/analytics"]
        WEBHOOK["Webhook Receiver\n/api/cattle/webhook/dl-api-complete"]
        CATTLE_SVC["CattleService"]
        TELEMETRY_SVC["TelemetryService"]
        CLEANUP_JOB["Cleanup Job node-cron Stale PENDING records"]
        PING_JOB["Ping Job Service health keep-alive"]
    end

    subgraph "AI Engine FastAPI Python"
        FASTAPI["FastAPI Server Port 8000"]
        DL_PIPELINE["DL Pipeline dl_pipeline.py"]
        REG_SVC["Registration Service"]
        SEARCH_SVC["Search Service"]
        TOURNAMENT["Biometric Tournament tournament_service.py"]
        FUSION["Dempster-Shafer Fusion fusion_service.py"]
        VECTOR_STORE["Vector Store vector_store.py"]
    end

    subgraph "Databases and Storage"
        MONGO[("MongoDB Atlas\nCattle Users Disputes AILogs")]
        QDRANT[("Qdrant Vector DB\ncattle_vectors_spatial\nHNSW + INT8 Quantization")]
        CLOUDINARY["Cloudinary Image CDN"]
        REDIS["Redis Queue Cache"]
    end

    APP -->|"HTTPS REST"| SERVER
    ADMIN -->|"HTTPS REST"| SERVER
    SERVER --> AUTH --> RATE --> HELMET
    HELMET --> FARMER_ROUTES & ADMIN_ROUTES
    FARMER_ROUTES --> CATTLE_SVC
    CATTLE_SVC -->|"Async Fire-and-Forget POST"| FASTAPI
    FASTAPI --> DL_PIPELINE
    DL_PIPELINE --> REG_SVC & SEARCH_SVC
    REG_SVC --> TOURNAMENT
    SEARCH_SVC --> TOURNAMENT
    TOURNAMENT --> FUSION
    FASTAPI --> VECTOR_STORE
    VECTOR_STORE <-->|"gRPC HTTP"| QDRANT
    FASTAPI -->|"Webhook on Completion"| WEBHOOK
    WEBHOOK --> TELEMETRY_SVC
    SERVER <--> MONGO
    SERVER <--> CLOUDINARY
    SERVER <--> REDIS
    CLEANUP_JOB -->|"TOCTOU guard"| MONGO
Loading

2. AI Biometric Pipeline — Registration Flow

flowchart TD
    START(["Farmer uploads Face + Muzzle images"]) --> UPLOAD["Express: Upload to Cloudinary\nCompute SHA-256 image hash\nCreate PENDING record in MongoDB"]
    UPLOAD -->|"Async Fire and Forget POST"| PRECHECK

    subgraph "FastAPI DL-API Registration Pipeline"
        PRECHECK["Pre-checks\nPortrait orientation enforced\nTOCTOU lock block duplicate cow_id in-flight"]
        PRECHECK --> PARALLEL_FETCH["Parallel Image Fetch asyncio.gather"]
        PARALLEL_FETCH --> GPU_PIPELINE

        subgraph "GPU Pipeline max 3 concurrent"
            GPU_PIPELINE["GPU Pipeline Thread"]
            GPU_PIPELINE --> COW_CLASS["ViT Classifier\ngoogle/vit-base-patch16-224\nReject non-bovine images"]
            COW_CLASS --> CLIP_QA

            subgraph "CLIP Unified Analyzer"
                CLIP_QA["OpenAI CLIP ViT-B/16 Single forward pass"]
                CLIP_QA --> ORIENTATION_GATE["Orientation Gate Zero-GPU pure Python check"]
                ORIENTATION_GATE --> CONTAM_GATE["Contamination Gate\nFoam/Dirt/Food detection Threshold 0.42 cosine"]
                CONTAM_GATE --> SEMANTIC_TAG["Semantic Tagger\nColor Pattern Horns as DB keywords"]
            end

            SEMANTIC_TAG --> YOLO_DETECT

            subgraph "YOLO Detection YOLOv8"
                YOLO_DETECT["YOLO Dual Detection"]
                YOLO_DETECT --> YOLO_FACE["best_face.pt Face Region Detection"]
                YOLO_DETECT --> YOLO_MUZZLE["best.pt Muzzle Region Detection"]
            end

            YOLO_FACE --> SPOOF_CHECK
            YOLO_MUZZLE --> SPOOF_CHECK

            subgraph "Anti-Spoofing"
                SPOOF_CHECK["MuzzleSpoofDetector ResNet-based\nFP16 inference torch.compile\nReject printed/screen images"]
            end

            SPOOF_CHECK --> CLAHE["CLAHE Enhancement LAB colorspace\n+ Nostril Auto-Leveler contour-based rotation"]
            CLAHE --> EMBED_PARALLEL

            subgraph "Parallel Embedding Extraction"
                EMBED_PARALLEL["Concurrent Embedding"]
                EMBED_PARALLEL --> MEGA["MegaDescriptor siamese_resnet18\n384x384 FP16 1536-d vector"]
                EMBED_PARALLEL --> SPATIAL_MUZZLE["TF Spatial Muzzle MuzzleCMPD568.keras\nHeadless CNN 1280-d vector"]
                EMBED_PARALLEL --> SPATIAL_FACE["TF Spatial Face FaceBasedIdentification.keras\nHeadless CNN 1280-d vector"]
                EMBED_PARALLEL --> SUPERPOINT["SuperPoint max 2048 keypoints\nfor LightGlue matching cache"]
            end
        end

        MEGA & SPATIAL_MUZZLE & SPATIAL_FACE & SUPERPOINT --> DUP_CHECK
        DUP_CHECK["Duplicate Detection\nRRF multi-vector Qdrant search Top-30 candidates"]
        DUP_CHECK --> TOURNAMENT_PHASE

        subgraph "Biometric Tournament"
            TOURNAMENT_PHASE["run_biometric_tournament()"]
            TOURNAMENT_PHASE --> CANDIDATE_LOOP["For each candidate async"]
            CANDIDATE_LOOP --> CPU_FEATS["CPU: LBP + HOG Texture feature distances"]
            CANDIDATE_LOOP --> LG_MATCH["GPU: LightGlue Physical ridge keypoint matching"]
            CANDIDATE_LOOP --> XGB_SCORE["XGBoost Ensembler xgb_biometric_model.json\n30 features to match probability"]
        end

        XGB_SCORE --> DS_FUSION

        subgraph "Dempster-Shafer Fusion"
            DS_FUSION["DS Combination Rule"]
            DS_FUSION --> B_MUZZLE["Expert 1 Spatial Muzzle Sim b_match if > 0.70"]
            DS_FUSION --> B_FACE["Expert 2 Spatial Face Sim b_match if face_conf >= 0.4"]
            DS_FUSION --> B_LG["Expert 3 LightGlue Ridges b_match if matches > 100"]
            B_MUZZLE & B_FACE & B_LG --> VERDICT["Final Belief Scores\nMATCH if belief_match >= 0.90"]
        end

        VERDICT -->|"MATCH: likely duplicate"| REJECT_DUP["Reject Already Registered Notify via Webhook"]
        VERDICT -->|"NO MATCH: unique animal"| SAVE_VECTORS["Upsert vectors to Qdrant\nmegadescriptor spatial_muzzle spatial_face\n+ SuperPoint cache zlib-compressed base64"]
    end

    SAVE_VECTORS --> WEBHOOK_NOTIFY["Webhook to Express /api/cattle/webhook/dl-api-complete"]
    WEBHOOK_NOTIFY --> UPDATE_MONGO["MongoDB: aiMetadata.status = SUCCESS\nconfidenceScore stored Telemetry logged to AILogs"]
Loading

3. AI Biometric Pipeline — Search and Identification Flow

flowchart TD
    START(["Search Request user_id + muzzle/face image"]) --> FETCH["Parallel Image Fetch asyncio.gather()"]
    FETCH --> PORTRAIT["Portrait Mode Check reject landscape w > h"]
    PORTRAIT --> GPU_SEARCH

    subgraph "GPU Search Pipeline"
        GPU_SEARCH["GPU Inference"]
        GPU_SEARCH --> COW_VERIFY["ViT: Is this a cow?"]
        COW_VERIFY --> CLIP_PASS["CLIP: Orientation + Contamination QA"]
        CLIP_PASS --> YOLO_CROPS["YOLO: Detect face + muzzle crops"]
        YOLO_CROPS --> SPOOF["Anti-Spoof: Real muzzle?"]
        SPOOF --> EMBEDS["Extract all embeddings\nMegaDescriptor 1536-d\nSpatial Muzzle 1280-d\nSpatial Face 1280-d\nSuperPoint keypoints"]
    end

    EMBEDS --> QDRANT_SEARCH["Qdrant Multi-Vector Search\nRRF Fusion across 3 vector spaces\nTop-30 candidate retrieval"]

    QDRANT_SEARCH --> CHECK_DISCONNECT{"Client still connected?"}
    CHECK_DISCONNECT -->|"No"| ABORT["HTTP 499 Client Disconnected"]
    CHECK_DISCONNECT -->|"Yes"| TOURNAMENT

    subgraph "Biometric Tournament Search"
        TOURNAMENT["run_biometric_tournament()"]
        TOURNAMENT --> PER_CANDIDATE["For each of 30 candidates"]
        PER_CANDIDATE --> COSINE["Cosine Similarity\nMegaDescriptor Spatial Muzzle Spatial Face"]
        PER_CANDIDATE --> LBP_HOG["LBP Distance HOG Distance"]
        PER_CANDIDATE --> LG["LightGlue Ridge Matches physical muzzle bead count"]
        PER_CANDIDATE --> MORPH["Morphology bead_count avg_area avg_eccentricity"]
        COSINE & LBP_HOG & LG & MORPH --> XGB["XGBoost 30-feature ensemble score"]
    end

    XGB --> DS["Dempster-Shafer Fusion 3 experts Muzzle Face LightGlue"]
    DS --> DECISION{"belief_match >= 0.90?"}
    DECISION -->|"YES"| MATCH["MATCH\nReturn cow_id confidence\nbest wrong answer URL telemetry"]
    DECISION -->|"NO"| NO_MATCH["NO MATCH\nReturn diagnostic reason DS scores telemetry"]
    MATCH & NO_MATCH --> TELEMETRY["Full Telemetry Payload\nStored in MongoDB AILogs\n30+ fields per inference event"]
Loading

4. Multi-Model Ensemble and Fusion

graph LR
    subgraph "Input Signals"
        IMG_M["Muzzle Image"]
        IMG_F["Face Image"]
    end

    subgraph "Deep Learning Models"
        MEGA["MegaDescriptor Siamese ResNet-18\n1536-d embedding\n384x384 input FP16"]
        SPAT_M["TF Spatial Muzzle CNN\nMuzzleCMPD568.keras\n1280-d spatial embedding"]
        SPAT_F["TF Spatial Face CNN\nFaceBasedIdentification.keras\n1280-d spatial embedding"]
        SUPERPOINT["SuperPoint Extractor\nmax 2048 keypoints"]
    end

    subgraph "Classical Feature Extraction"
        LBP["LBP Histogram P=24 R=3 uniform\nNormalized frequency"]
        HOG["HOG Descriptor 9 orientations\n8x8 px/cell 2x2 blk 256x256 resize"]
        MORPH["Muzzle Morphology\nBead count Avg area Avg eccentricity"]
    end

    subgraph "Geometric Matcher"
        LG["LightGlue Matcher\ndepth_conf=0.9 width_conf=0.9\nInlier ridge-match count + Alignment SSIM"]
    end

    IMG_M --> MEGA & SPAT_M & SUPERPOINT & LBP & HOG & MORPH
    IMG_F --> MEGA & SPAT_F & SUPERPOINT

    subgraph "XGBoost Ensembler 30 features"
        XGB["xgb_biometric_model.json\nCosine sim scores x4\nLBP/HOG distances\nLightGlue match count\nMorphology features\nYOLO confidence scores\nSpoof probabilities"]
    end

    MEGA --> XGB
    SPAT_M --> XGB
    SPAT_F --> XGB
    LBP --> XGB
    HOG --> XGB
    MORPH --> XGB
    LG --> XGB

    subgraph "Dempster-Shafer Fusion 3 Experts"
        DS_M["Expert 1 Spatial Muzzle belief x0.90"]
        DS_F["Expert 2 Spatial Face belief x0.85\nguarded by face_conf"]
        DS_LG["Expert 3 LightGlue Ridges belief x0.90\nthresh 100-130 matches"]
        DS_COMBINE["DS Combination Rule sequential\nK = 1 - conflict_mass"]
    end

    SPAT_M --> DS_M
    SPAT_F --> DS_F
    LG --> DS_LG
    XGB --> DS_M
    DS_M & DS_F & DS_LG --> DS_COMBINE
    DS_COMBINE --> VERDICT["Final Decision\nMATCH if belief_match >= 0.90"]
Loading

5. Vector Database Search Strategy

flowchart LR
    subgraph "Qdrant Collection cattle_vectors_spatial"
        direction TB
        CONFIG["HNSW Config\nm=32 ef_construct=200\nHNSW EF search=256\nINT8 Scalar Quantization\nquantile=0.99 always_ram=true"]

        subgraph "Named Vectors per point"
            V1["megadescriptor 1536-d COSINE"]
            V2["spatial_muzzle 1280-d COSINE"]
            V3["spatial_face 1280-d COSINE"]
        end

        subgraph "Payload Indexes"
            P1["keyword: cow_id"]
            P2["keyword: farmer_id"]
            P3["keyword: part muzzle/face/face_muzzle"]
            P4["keyword: semantic_color"]
            P5["keyword: semantic_pattern"]
            P6["keyword: semantic_horns"]
        end

        subgraph "Stored Payload"
            S1["cow_name image_url crop_url"]
            S2["muzzle_crop_b64 zlib+base64"]
            S3["superpoint_cache serialized tensors"]
            S4["semantic_color semantic_pattern semantic_horns"]
        end
    end

    subgraph "RRF Multi-Vector Search COHORT=200 TOP_K=30"
        direction TB
        Q1["Query: megadescriptor vector"] --> SEARCH_M["Qdrant search megadescriptor space"]
        Q2["Query: spatial_muzzle vector"] --> SEARCH_SM["Qdrant search spatial_muzzle space"]
        Q3["Query: spatial_face vector"] --> SEARCH_SF["Qdrant search spatial_face space"]

        SEARCH_M & SEARCH_SM & SEARCH_SF --> RRF["Reciprocal Rank Fusion\nRRF_K=30 rank-based score aggregation"]
        RRF --> SEM_BOOST["Optional Semantic Boost\n+0.04 per matching tag color/pattern/horns"]
        SEM_BOOST --> TOP30["Top-30 Candidates to Tournament"]
    end

    COHORT_FILTER["Concurrent parallel searches\nacross 200-point cohorts ThreadPoolExecutor"] --> SEARCH_M & SEARCH_SM & SEARCH_SF
Loading

6. Data Flow and Async Job Architecture

sequenceDiagram
    actor Farmer
    participant App as Mobile App
    participant Express as Express Server
    participant Cloudinary as Cloudinary
    participant MongoDB as MongoDB
    participant FastAPI as FastAPI DL-API
    participant Qdrant as Qdrant

    Farmer->>App: Opens camera, captures face + muzzle
    App->>App: Offline sync check (LocalForage)
    App->>Express: POST /api/cattle/register multipart/form-data
    Express->>Express: JWT verify + Rate limit + NoSQL sanitize
    Express->>Express: SHA-256 image hash for idempotency
    Express->>MongoDB: Atomicity check for existing PENDING (TOCTOU guard)
    Express->>Cloudinary: Upload face + muzzle images (async)
    Express->>MongoDB: Insert cattle record (status: PENDING)
    Express-->>App: 202 Accepted (background processing begins)

    Express->>FastAPI: POST /process-registration (fire and forget)

    Note over FastAPI: GPU Semaphore (max 3 concurrent)
    FastAPI->>FastAPI: TOCTOU: check in_flight_registrations set
    FastAPI->>FastAPI: Download images concurrently asyncio.gather
    FastAPI->>FastAPI: ViT cow classifier
    FastAPI->>FastAPI: CLIP QA + semantic tagging single forward pass
    FastAPI->>FastAPI: YOLO detect face + muzzle crops
    FastAPI->>FastAPI: Anti-spoof check ResNet MuzzleSpoofDetector
    FastAPI->>FastAPI: CLAHE enhancement + nostril leveler
    FastAPI->>FastAPI: MegaDescriptor embedding FP16
    FastAPI->>FastAPI: TF Spatial embeddings muzzle + face
    FastAPI->>FastAPI: SuperPoint keypoint extraction
    FastAPI->>Qdrant: RRF search for duplicate detection top-30
    FastAPI->>FastAPI: Biometric Tournament XGBoost + LightGlue + DS Fusion

    alt Duplicate Detected
        FastAPI->>Express: Webhook status=DUPLICATE
        Express->>MongoDB: Update status=DUPLICATE
        Express-->>App: Push notification
    else Unique Animal
        FastAPI->>Qdrant: Upsert 3 named vectors + SuperPoint cache
        FastAPI->>Express: Webhook status=SUCCESS + telemetry payload
        Express->>MongoDB: Update status=SUCCESS confidenceScore
        Express->>MongoDB: Insert AILog 30+ telemetry fields
        Express-->>App: Push notification
    end

    Note over Express: Background Jobs node-cron
    Express->>MongoDB: cleanupJob expire PENDING > 5min to FAILED
    Express->>FastAPI: pingJob health keepalive
Loading

7. MongoDB Schema Relationships

erDiagram
    User {
        ObjectId _id PK
        string name
        string role "farmer or collector or admin"
        object contact "phone and email"
        object auth "password_hash and otpSession"
        object location "state district block village pincode"
        string aadharHash
        string profilePicture
        ObjectId_array cows FK
        Date lastLogoutAt
        Date createdAt
    }

    Cattle {
        ObjectId _id PK
        ObjectId farmerId FK
        string tagNumber "sparse unique"
        string name
        string species "Cow or Buffalo"
        string breed
        string sex "Male or Female or Freemartin"
        number ageYears
        number ageMonths
        string sireTag
        string damTag
        string source "Home Born or Purchase"
        object purchaseDetails "date and price"
        object location "lat and lng"
        object photos "faceProfile muzzle leftProfile rightProfile backView tailView selfie imageHash"
        object aiMetadata "isRegistered status confidenceScore lastScannedAt"
        string currentStatus "Milking Dry Pregnant Heifer Calf"
        boolean isSick
        boolean isDispute
        object healthStats "birthWeight motherWeightAtCalving calvingCounter"
        Date createdAt
    }

    Dispute {
        ObjectId _id PK
        ObjectId cattleId FK
        ObjectId raisedBy FK
        string reason
        string status "OPEN or RESOLVED"
        Date createdAt
    }

    AILog {
        ObjectId _id PK
        Date timestamp
        string endpoint
        boolean success
        string matchStatus
        string cowId
        string farmerId
        string matchedCowId
        number inferenceTimeMs
        number muzzleConfM
        number spoofProbM
        number faceSimilarityScore
        number muzzleSimilarityScore
        number spatialMuzzleSim
        number spatialFaceSim
        number lgMatches
        number dsBeliefMatch
        number dsBeliefMismatch
        number dsUncertainty
        number xgbScore
        object tradMorphology "beadCount avgArea avgEccentricity"
        number tradLbpDist
        number tradHogDist
        object semanticTags "color pattern horns"
        object clipScores
        boolean isAiOutcomeCorrect
        Date createdAt
    }

    User ||--o{ Cattle : "owns"
    User ||--o{ Dispute : "raises"
    Cattle ||--o| Dispute : "subject of"
    Cattle ||--o{ AILog : "generates"
Loading

8. Client Application Page Flow

stateDiagram-v2
    [*] --> Onboarding : First launch
    Onboarding --> Login : Have account
    Onboarding --> Register : New farmer

    Register --> Login : Account created
    Login --> Home : JWT stored

    state Home {
        [*] --> Dashboard
        Dashboard --> MyCows : View herd
        Dashboard --> Scan : Identify cow
        Dashboard --> Disputes : View disputes
        Dashboard --> UserProfile : Profile settings
    }

    MyCows --> CowProfile : Select cow
    CowProfile --> Register : Edit or Re-register
    CowProfile --> Disputes : Raise dispute

    state CowProfile {
        [*] --> ViewPhotos
        ViewPhotos --> AIStatus : View registration status
        AIStatus --> AIMetadata : Confidence and Match scores
    }

    Home --> OfflineSync : Network lost
    OfflineSync --> Home : Reconnected sync pending ops

    state OfflineSync {
        [*] --> LocalForage
        LocalForage --> QueuedRequests : Pending registrations
        QueuedRequests --> AutoSync : On reconnect
    }
Loading

Tech Stack

AI / ML Engine (dl-api/)

Category Technology
Framework FastAPI (Python), Uvicorn ASGI
Deep Learning PyTorch 2.x, TensorFlow 2.12, HuggingFace Transformers
Object Detection YOLOv8 (Ultralytics) — dual models for face and muzzle
Biometric Embedding Siamese ResNet-18 (MegaDescriptor, 1536-d)
Spatial CNN Custom Keras CNNs — muzzle (1280-d) + face (1280-d)
Image QA OpenAI CLIP ViT-B/16 — contamination gate + semantic tagger
Animal Classifier Google ViT-B/16 (HuggingFace pipeline)
Keypoint Matching SuperPoint + LightGlue (cvg/LightGlue)
Ensemble XGBoost (~30 features)
Score Fusion Dempster-Shafer Theory of Evidence (3 experts)
Classical Features LBP (scikit-image), HOG (scikit-image), OpenCV morphology
Vector DB Qdrant (HNSW m=32, INT8 quantization, RRF multi-vector)
GPU Optimization FP16 inference, torch.compile(), cuDNN benchmark, TF mixed_float16, CUDA warmup
Rate Limiting SlowAPI
Image Processing OpenCV, Pillow, scikit-image, CLAHE enhancement

Backend API (server/)

Category Technology
Runtime Node.js, TypeScript
Framework Express 5.x
Database MongoDB + Mongoose
Auth JWT (jsonwebtoken), bcrypt
Storage Cloudinary (image CDN)
Security Helmet, CORS, express-rate-limit, NoSQL sanitizer
Logging Pino + pino-http
Jobs node-cron (cleanup + ping)
Validation Zod
Testing Jest, Supertest, mongodb-memory-server

Mobile Client (client/)

Category Technology
Framework React 19 + TypeScript
Build Tool Vite 7
Mobile Capacitor 8 (Android + iOS)
UI Components Material UI (MUI) v7
Animations Framer Motion
State / Data TanStack Query (React Query v5)
Offline LocalForage (IndexedDB-backed offline sync)
Camera @capacitor/camera + @capacitor-community/camera-preview
On-device ML TensorFlow.js (WebGL backend)
Testing Vitest, @testing-library/react, MSW

Admin Dashboard (admin-app/)

Category Technology
Framework React + TypeScript
Build Vite
Mobile Capacitor (Android)
Testing Vitest

Repository Structure

BovineID/
├── client/                     # Farmer mobile app (React + Capacitor)
│   └── src/
│       ├── pages/              # CowProfile, Home, MyCows, Register, Search, Login
│       ├── components/         # Reusable UI components
│       ├── apis/               # Axios API layer
│       └── theme/              # MUI theme tokens
│
├── admin-app/                  # Admin dashboard
│   └── src/
│       └── pages/              # Analytics, Dispute management, User management
│
├── server/                     # Express REST API (TypeScript)
│   └── src/
│       ├── models/             # Mongoose schemas: Cattle, User, Dispute, AILog
│       ├── controllers/        # farmer/ and admin/ controller groups
│       ├── routes/             # farmer/ and admin/ route groups
│       ├── services/           # cattleService, cloudinaryService, telemetryService
│       ├── jobs/               # cleanupJob (stale PENDING), pingJob
│       ├── middleware/         # errorHandler, sanitize, auth
│       └── utils/              # logger (pino), dlApiClient, qdrantClient
│
├── dl-api/                     # FastAPI AI/ML engine (Python)
│   ├── api/
│   │   └── router.py           # FastAPI route definitions
│   ├── engine/
│   │   ├── dl_pipeline.py      # DLPipeline class — all model loading + inference
│   │   ├── clip_analyzer.py    # UnifiedCLIPAnalyzer — QA + semantic tagging
│   │   ├── vector_store.py     # CattleVectorStore — Qdrant CRUD + RRF search
│   │   ├── megadescriptor_model.py  # Siamese ResNet-18 wrapper
│   │   ├── spoof_model.py      # MuzzleSpoofDetector (ResNet-based)
│   │   └── traditional_features.py # LBP, HOG, morphology extraction
│   ├── services/
│   │   ├── registration_service.py  # Full registration pipeline orchestration
│   │   ├── search_service.py        # Search / identification pipeline
│   │   ├── tournament_service.py    # Biometric tournament (XGBoost + LightGlue)
│   │   ├── fusion_service.py        # Dempster-Shafer fusion + cosine similarity
│   │   ├── image_service.py         # Image download, crop, Cloudinary upload
│   │   ├── telemetry_builder.py     # Build 30+ field telemetry payload
│   │   └── webhook_service.py       # Send result webhook to Express
│   ├── core/
│   │   ├── config.py           # Environment configuration
│   │   ├── globals.py          # Shared GPU semaphore, db, dl pipeline singletons
│   │   ├── security.py         # SlowAPI rate limiter
│   │   └── logging_config.py   # Structured logging setup
│   ├── models/                 # Trained ML model files (*.pt, *.keras, *.json)
│   │   ├── best.pt             # YOLOv8 muzzle detector (22.5 MB)
│   │   ├── best_face.pt        # YOLOv8 face detector (6.2 MB)
│   │   ├── best_model.pth      # MuzzleSpoofDetector weights (4.5 MB)
│   │   ├── siamese_resnet18_newdataset.pt  # MegaDescriptor (45 MB)
│   │   ├── MuzzleCMPD568.keras # Spatial muzzle CNN (31 MB)
│   │   ├── FaceBasedIdentification.keras   # Spatial face CNN (28.5 MB)
│   │   └── xgb_biometric_model.json        # XGBoost ensembler
│   ├── main.py                 # FastAPI app entry — model loading lifespan
│   ├── schemas.py              # Pydantic request/response schemas
│   └── requirements.txt        # Python dependencies
│
├── packages/
│   └── shared/                 # Shared TypeScript types/utilities
│
└── docker-compose.yml          # Local dev orchestration (server + dl-api + redis)

AI Models and Computer Vision

Model Zoo

Model File Architecture Purpose Size
YOLOv8 Muzzle best.pt YOLOv8n Muzzle region detection 22.5 MB
YOLOv8 Face best_face.pt YOLOv8n Face/head detection 6.2 MB
MegaDescriptor siamese_resnet18_newdataset.pt Siamese ResNet-18 Primary biometric embedding (1536-d) 45 MB
Spatial Muzzle MuzzleCMPD568.keras Custom CNN Spatial attention muzzle features (1280-d) 31 MB
Spatial Face FaceBasedIdentification.keras Custom CNN Spatial attention face features (1280-d) 28.5 MB
Spoof Detector best_model.pth ResNet-based Anti-spoofing for muzzle images 4.5 MB
XGBoost xgb_biometric_model.json Gradient Boosted Trees Feature-level ensemble fusion 88 KB
CLIP ViT-B/16 HuggingFace CDN Vision Transformer QA gateway + semantic tagging ~600 MB
ViT-B/16 HuggingFace CDN Vision Transformer Cattle vs non-cattle classification ~330 MB
SuperPoint LightGlue lib CNN detector Keypoint extraction for ridge matching ~5 MB
LightGlue LightGlue lib Transformer matcher Geometric verification of ridges ~25 MB

GPU Optimization Stack

FP16 Inference       → All PyTorch models run in float16 on CUDA
torch.compile()      → JIT compilation (PyTorch 2.0+), 20-50% throughput gain
cuDNN Benchmark      → Auto-selects fastest conv algorithm for current hardware
TF32 Matmul          → Enabled for PyTorch matmul + cuDNN convolutions
TF mixed_float16     → TensorFlow global policy for spatial CNN models
CUDA Warmup          → Pre-allocated memory + JIT-compiled kernels before first request
autocast('cuda')     → Automatic mixed precision in CLIP and LightGlue inference
GPU Semaphore        → Max 3 concurrent GPU tasks to prevent OOM crashes

Feature Engineering Pipeline

The biometric identification pipeline extracts ~30 distinct features from each cattle image pair:

Deep Embedding Features

  • MegaDescriptor cosine similarity — 1536-d Siamese embedding distance (primary biometric)
  • Spatial muzzle cosine similarity — 1280-d spatial attention feature from Keras CNN
  • Spatial face cosine similarity — 1280-d spatial attention feature from face Keras CNN
  • MegaDescriptor face cosine similarity — same embedding applied to face region

Geometric / Keypoint Features

  • LightGlue match count — number of physically verified ridge keypoint matches
  • SuperPoint inlier ratio — geometric consistency of matched keypoints
  • SSIM after alignment — structural similarity post-geometric alignment

Texture / Classical Features

  • LBP histogram distance — Local Binary Pattern (P=24, R=3, uniform) — skin texture
  • HOG descriptor distance — Histogram of Oriented Gradients (9 orientations, 8x8 px/cell)
  • Morphological bead count — number of muzzle ridge beads (adaptive threshold)
  • Average bead area — mean contour area of muzzle beads
  • Average eccentricity — ellipse eccentricity of bead contours

Confidence / Quality Features

  • YOLO muzzle detection confidence — detection quality score
  • YOLO face detection confidence — face detection quality score
  • Spoof probability muzzle — probability of being a printed/screen image
  • Spoof probability face — face spoof probability

Semantic Features (optional boost)

  • semantic_color match — CLIP-predicted coat color tag (+0.04 RRF boost)
  • semantic_pattern match — CLIP-predicted pattern tag (+0.04 RRF boost)
  • semantic_horns match — CLIP-predicted horn type tag (+0.04 RRF boost)

Security Architecture

Layer 1:  Network       CORS strict allowlist · Helmet HTTP headers
Layer 2:  Auth          JWT · bcrypt password hashing · Aadhar hash storage
Layer 3:  Input         Zod schema validation · NoSQL injection sanitization
Layer 4:  Rate Limits   Express rate limiter (per-IP) · SlowAPI (FastAPI per-route)
Layer 5:  TOCTOU        MongoDB partial unique index on PENDING status (atomic)
Layer 6:  TOCTOU        in_flight_registrations Set (async guard in FastAPI)
Layer 7:  Idempotency   SHA-256 image hash dedupe before processing
Layer 8:  Cleanup       node-cron job expires stale PENDING > 5 min to FAILED
Layer 9:  Anti-Spoof    ResNet spoof detector blocks printed/screen muzzle images
Layer 10: CLIP QA       Contamination gate rejects dirty muzzle photos

Getting Started

Prerequisites

  • Node.js >= 18
  • Python >= 3.10
  • CUDA-capable GPU (optional, CPU fallback available)
  • Docker + Docker Compose (for local dev orchestration)
  • MongoDB Atlas / local MongoDB
  • Qdrant Cloud / local Qdrant instance

Quick Start with Docker Compose

# Clone the repository
git clone <repo-url>
cd BovineID

# Copy environment files
cp server/.env.example server/.env
cp dl-api/.env.example dl-api/.env

# Start all services
docker-compose up --build

Services will be available at:

  • Express API: http://localhost:5000
  • FastAPI DL Engine: http://localhost:8000
  • Redis: localhost:6379

Manual Setup

Backend (Express)

cd server
npm install
npm run dev

AI Engine (FastAPI)

cd dl-api
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
pip install git+https://github.com/cvg/LightGlue.git
uvicorn main:app --reload --port 8000

Mobile Client

cd client
npm install
npm run dev
# For Android:
npx cap sync android
npx cap open android

Environment Variables

Server (server/.env)

PORT=2424
NODE_ENV=development
MONGO_URI=mongodb+srv://...
JWT_SECRET=your_jwt_secret_here
CLOUDINARY_CLOUD_NAME=...
CLOUDINARY_API_KEY=...
CLOUDINARY_API_SECRET=...
DL_API_URL=http://localhost:8000
EXPRESS_WEBHOOK_SECRET=...
QDRANT_URL=https://...
QDRANT_API_KEY=...
CLIENT_LINK=http://localhost:5173
ADMIN_CLIENT_LINK=http://localhost:5174

AI Engine (dl-api/.env)

QDRANT_URL=https://your-cluster.qdrant.io
QDRANT_API_KEY=...
EMBEDDING_MODEL_PATH=models/siamese_resnet18_newdataset.pt
EMBEDDING_VECTOR_SIZE=1536
EXPRESS_WEBHOOK_URL=http://localhost:2424/api/cattle/webhook/dl-api-complete
CLOUDINARY_CLOUD_NAME=...
CLOUDINARY_API_KEY=...
CLOUDINARY_API_SECRET=...

API Reference

Farmer Endpoints (/api/)

Method Path Description
POST /auth/register Register new farmer account
POST /auth/login Login with phone + password
GET /user/profile Get farmer profile
POST /cattle/register Register new cattle (multipart: face + muzzle images)
POST /cattle/search Identify unknown cattle via biometrics
GET /cattle/my-herd Get all cattle for farmer
GET /cattle/:id Get cattle details
GET /location/states Get location data (state/district/block)

Admin Endpoints (/api/admin/)

Method Path Description
POST /auth/login Admin login
GET /cattle List all cattle (paginated)
GET /cattle/:id Get cattle by ID
DELETE /cattle/:id Delete cattle + Qdrant vectors
GET /disputes List all disputes
PUT /disputes/:id Resolve dispute
GET /analytics/overview Registration stats + AI telemetry
GET /user List all farmers
GET /user/:id Get farmer details

AI Engine Endpoints (FastAPI :8000)

Method Path Description
POST /register Synchronous registration (returns when complete)
POST /search Identify cattle biometrically
POST /process-registration Async registration (webhook on complete)
GET /health Health check

License

This project is proprietary. All rights reserved.

BovineID — Every cow, uniquely identified.

Built with passion using Computer Vision, Deep Learning, and Full-Stack Engineering.

About

AI-powered cattle identification using muzzle-print & computer vision. Tested on 100+ Indian cattle in real field conditions. YOLOv8 · MegaDescriptor · LightGlue · Dempster-Shafer fusion · React/Capacitor · FastAPI · Qdrant.

Topics

biometricscapacitorcomputer-visiondeep-learningfastapilivestockmobile-apppytorchqdrant-cattlereactresearch-projectyolov8

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Contributors

Languages