Object Detection with Crack Detection

This system uses Gemma 3 via OpenRouter to detect objects from camera feed or images and identify if they have cracks. It also provides surface coordinates of detected objects relative to the top-left of the input frame.

Features

• Real-time camera detection or single image analysis
• Object detection with crack identification
• Surface coordinate extraction (x, y position relative to top-left)
• Bounding box visualization
• Confidence levels for detections
• Supports Gemma 3 vision model via OpenRouter

Setup

1. Install Dependencies

pip install -r requirements.txt

2. Configure API Key

1. Get your OpenRouter API key from https://openrouter.ai/keys
2. Copy .env.example to .env:

   cp .env.example .env

3. Edit .env and add your API key:

   OPENROUTER_API_KEY=your_actual_api_key_here
   

Usage

Camera Mode (Default)

Run the script to start camera detection:

python detect_cracks.py

Controls:

• Press c to capture and analyze the current frame
• Press q to quit

Image Mode

Analyze a specific image file:

python detect_cracks.py --image path/to/image.jpg
# or
python detect_cracks.py -i path/to/image.jpg

Example with provided images:

python detect_cracks.py -i cube.jpg
python detect_cracks.py -i gear.jpg
python detect_cracks.py -i knuckle.jpg

Output

The system provides:

1. Visual Output:

   • Bounding boxes around detected objects
   • Red boxes for objects with cracks
   • Green boxes for objects without cracks
   • Center point marked with purple circle
   • Coordinates displayed below each object

2. Console Output:

   ============================================================
   Object: Gear
   Has Crack: True
   Confidence: high
   Position (x, y): (640, 360)
   Bounding Box: (540, 260) to (740, 460)
   Description: Metal gear with visible crack on surface
   ============================================================
   

3. Saved Results:

   • Camera mode: detection_YYYYMMDD_HHMMSS.jpg
   • Image mode: detected_<original_filename>.jpg

How It Works

1. Captures image from camera or loads from file
2. Encodes image to base64 and sends to OpenRouter API
3. Gemma 3 vision model analyzes the image for:
   • Object identification
   • Crack detection
   • Object location (coordinates relative to top-left)
   • Bounding box dimensions
4. Results are parsed and visualized with OpenCV
5. Coordinates are displayed relative to top-left corner (0,0)

Coordinate System

• Origin (0, 0) is at the top-left corner of the image
• X increases from left to right
• Y increases from top to bottom
• Center point (x, y) represents the object's center
• Bounding box shows the full extent of the object

Troubleshooting

• Camera not opening: Check if camera is available and not used by another application
• API errors: Verify your OPENROUTER_API_KEY in .env file
• Import errors: Ensure all dependencies are installed with pip install -r requirements.txt

Notes

• The Gemma 3 model used is google/gemma-2-9b-it:free
• Detection accuracy depends on image quality and object visibility
• Camera resolution is set to 1280x720 by default
• API requests may take a few seconds to process