hacktech

React + Node demo for the Ironsite-style question: can the worker safely reach the target in this scene?

The app lets you:

• capture a live camera frame
• upload an image or a long video and extract a frame server-side with ffmpeg
• ask a model for a baseline reachability answer plus scene annotations
• run local YOLO detections on the extracted frame
• ask K2 Think V2 to critique the scene reasoning using the structured geometry
• run a deterministic geometry layer for distance and obstruction checks
• compare the raw model answer against the reasoning-layer verdict

Setup

1. Install dependencies: npm install
2. Create a local env file: cp .env.example .env
3. Add your API keys to .env. K2_API_KEY enables the K2 Think V2 reasoning panel. OPENAI_API_KEY is optional if you want the OpenAI baseline as well.

Run

Start the API server:

npm run server

In a second terminal, start the frontend:

npm run dev

Open http://localhost:5173.

Demo flow

1. Upload a construction image or video, or capture a live camera frame.
2. For videos, upload once, enter a timestamp in seconds, and click Extract frame from video.
3. Click Analyze frame.
4. Show the judge: Baseline model answer versus Reasoning layer versus K2 Think V2 assessment
5. Use the overlay to point out YOLO detections, worker/target boxes, path, and blockers.

Test

npm test