Cybergrip: Industrial Robotic arm with machine vision

Overview

An automated robotic arm for industrial use that performs quality checks and sorting via machine vision. Includes manual hand-tracking mode. Won 1st place in Egypt at the International Codeavour Competition.

This repository contains code for Arduino and Python projects that control a robotic arm, perform color detection and object sorting, and monitor environmental sensors. It also includes a Pictoblox .sb3 file required by the Codeavour 6.0 competition.

Note: The .sb3 files in this repository are Pictoblox project files, included as required by Codeavour 6.0.

Files and Purpose

autoMultiColor.py

• Python automation script using OpenCV, NumPy, PyFirmata, and threading.
• Connects to an Arduino board on COM9 and controls a stepper motor plus servo motors for a robotic arm.
• Uses live camera capture to detect blue, red, and green objects using HSV color filtering.
• Stops the stepper motor when a blue or red object is centered in the frame, then executes a pick-and-place motion sequence.
• Moves the shoulder, elbow, base, and gripper servos to pick up and deposit detected objects.
• Includes a sleep countdown feature that can stop the program and release the board after a timeout.

vision arm hand.py

• Python gesture control script using OpenCV, NumPy, PyFirmata, and the cvzone HandDetector.
• Captures video from a second camera (VideoCapture(1)) and tracks a single hand.
• Uses finger state detection to control the robotic arm:
  • Certain gestures close or open the gripper.
  • X/Y coordinates of a hand landmark are mapped to servo positions for shoulder and elbow.
  • A potentiometer connected to analog pin A0 is mapped to base rotation.
• Provides on-screen feedback with rectangles and text showing current servo angles.
• Runs until the user presses the c key.

Sensors/Sensors.ino

• Arduino sketch for environmental monitoring and SMS/call alerts.
• Uses a DHT11 temperature and humidity sensor on pin 5.
• Reads a gas sensor on pin 9 and a flame sensor on pin 10.
• Uses a SIM800L GSM module on SoftwareSerial(3, 2) to send SMS messages and make calls.
• Activates a buzzer on pin 11 when a gas leak or fire is detected.
• Sends an initial check message during setup and sends alerts when hazardous conditions are found.

Sensors_Ultrasonic/Sensors_Ultrasonic.ino

• Arduino sketch that extends the sensor system with ultrasonic object detection.
• Uses two ultrasonic sensors:
  • trigPin/echoPin on pins 6/7.
  • trigPin2/echoPin2 on pins 11/12.
• Reads DHT11 temperature and humidity values, plus gas and flame sensors.
• Tracks object counts and skipped objects based on distance thresholds.
• Sends SMS/call alerts via SIM800L for gas or fire events, and prints sensor and object data to Serial.

Sustainability/Sustainability.ino

• Arduino prototype sketch for a robotic arm and ultrasonic distance sensing.
• Configures four servos on pins 9, 10, 11, and 12.
• Sets up an ultrasonic rangefinder on pins 4 and 5.
• Defines a distance-reading helper function, but the loop() function is currently unfinished.

Sensors.sb3

• Pictoblox project file included for visual programming.
• This file is included to satisfy the Codeavour 6.0 competition requirement for .sb3 assets.

Requirements and Dependencies

Python scripts

• pyfirmata
• opencv-python
• numpy
• cvzone (for hand tracking in vision arm hand.py)

Arduino sketches

• DHT.h library for DHT11 temperature/humidity sensor support.
• SoftwareSerial.h for SIM800L communication.
• Servo.h for servo control in Sustainability/Sustainability.ino.

Usage Notes

• The Python files assume the Arduino board is available on COM9.
• The camera indexes (0 for autoMultiColor.py and 1 for vision arm hand.py) may need to be adjusted for your system.
• Update the phone number in the SIM800L code sections before using the SMS/call features.
• autoMultiColor.py uses threaded camera and stepper control loops to run detection and mechanical motion concurrently.

Project Structure

• /autoMultiColor.py — color-based robotic arm automation and stepper control.
• /vision arm hand.py — gesture-based control of the arm using hand tracking.
• /Sensors/Sensors.ino — DHT, gas, flame, buzzer, and SIM800L alert system.
• /Sensors_Ultrasonic/Sensors_Ultrasonic.ino — sensor system with added ultrasonic object counting.
• /Sustainability/Sustainability.ino — basic servo and ultrasonic distance reading prototype.
• /Sensors.sb3 — Pictoblox project file required by Codeavour 6.0.