💡 Full-stack on-device AI inference via ExecuTorch for offline-capable inspections

[github-actions[bot]]

Summary

Integrate React Native ExecuTorch to run Whisper STT, photo classification (varroa/brood patterns), and a fine-tuned small LLM (SmolLM 2 or Qwen 3) for recommendations entirely on-device. This enables fully offline AI-powered inspections with zero cloud dependency, complete data privacy, and sub-second inference — a capability no competitor currently offers end-to-end.

Market Signal

• HiveSense already ships on-device Whisper (142MB offline model) for offline voice transcription — validating the on-device approach
• Beentry, APiLOG, HiveMasterPro, Hive Pal, BeeKeeperVoice all offer voice AI, but cloud-dependent
• React Native ExecuTorch reached production maturity in 2026: sub-20ms inference on modern devices, supports Whisper/CLIP/Llama 3.2/Qwen 3, with Expo config plugin
• HiveLog AI leads in photo-based varroa detection but requires cloud upload
• 60%+ of apiaries have unreliable cell coverage during inspections
• Smart beekeeping market growing 20% CAGR to $1.8B by 2032

Sources: React Native ExecuTorch docs, HiveSense offline features, Best Beekeeping Apps 2026

User Signal

Existing idea #346 proposes on-device voice transcription specifically. This proposal extends far beyond STT to the full inference pipeline: voice → structured data → photo analysis → AI recommendations, all on-device. Core differentiator: zero-tap inspections work fully offline with intelligence, not just data capture.

Technical Opportunity

ExecuTorch has a React Native library (Software Mansion) with Expo config plugin support. Key capabilities:

• Whisper for ASR (1% WER on clean speech, 55% fewer errors on noisy audio)
• CLIP for image embedding and classification
• Small LLMs (SmolLM 2, Qwen 3) for text generation and recommendation reasoning
• iOS delegates to CoreML/Metal; Android to QNN/XNNPACK
• 50KB base runtime, models 142MB–1.5GB depending on capability tier

Broodly's architecture already defines Vertex AI for cloud inference — on-device inference becomes a parallel execution path with graceful cloud fallback. The phased approach: Whisper first, photo ML second, recommendation LLM third.

Assessment

Dimension	Score	Rationale
Feasibility	med	ExecuTorch is production-ready but new to Expo ecosystem. HiveSense validates the on-device Whisper approach. Phased rollout mitigates risk.
Impact	high	Only app with full on-device intelligence pipeline. Privacy-first, works-everywhere architecture is an enduring competitive moat.
Urgency	high	Voice-first inspection is becoming table stakes (6+ competitors). On-device execution is the next differentiator wave.

Adversarial Review

Strongest objection: ExecuTorch is relatively new technology. On-device models are inherently less capable than cloud models (GPT-4 vs SmolLM 2). Engineering effort is significant and may distract from core MVP feature delivery.

Rebuttal: Whisper Large v3 achieves 1% WER on-device — competitive with cloud APIs. Phased rollout delivers standalone value at each phase: Phase 1 ships on-device Whisper (proven, HiveSense validates), Phase 2 adds photo classification (TFLite models are mature), Phase 3 adds recommendation LLM. The privacy-first, works-everywhere architecture is a moat, not a distraction.

Suggested Next Step

Spike: Integrate react-native-executorch with Expo config plugin, benchmark Whisper inference latency on iPhone 12+ and Pixel 6+, validate model download and storage UX. Document minimum device requirements and model size tradeoffs.