Skip to content

Interface with the Roboflow API and Python package for running inference (receiving predictions) and customizing result images from your Roboflow Train computer vision models.

License

Notifications You must be signed in to change notification settings

roboflow/roboflow-computer-vision-utilities

Repository files navigation

roboflow-computer-vision-utilities

Interface with the Roboflow API and Python SDK for running inference (receiving predictions) with AutoML and custom-trained models compatible with Roboflow Deploy.

Roboflow Logo

Contact Us!

What is Roboflow?

Roboflow makes managing, preprocessing, augmenting, and versioning datasets for computer vision seamless. This repo utilizes the official Roboflow Python SDK that interfaces with the Roboflow Hosted API Inference Server and REST API.

Key features of Roboflow:

Available Plans & Features

https://roboflow.com/pricing

Personal and Research Projects - Applying for Additional Account Features:

Business Projects and POC's - Requesting Additional Account Features:

Popular Repositories and Learning Resources:

Roboflow Python SDKNotebooks (Custom Model Training Tutorials) • Learn Computer VisionSupervisionPolygon Zone

Installation (Dependencies):

Python Version: 3.10>=Python>=3.7. Install from Source:

git clone https://github.com/roboflow-ai/roboflow-computer-vision-utilities.git
cd roboflow-computer-vision-utilities
python3 -m venv env
source env/bin/activate 
pip3 install -r requirements.txt

🤖 📹 Inference Utilities:

functionality images video stream
Draw Boxes GitHub
Colab
GitHub GitHub
Write Text GitHub
Colab
GitHub GitHub
Fill Boxes GitHub
Colab
GitHub GitHub
Crop Boxes GitHub
Blur Boxes GitHub GitHub GitHub
Two Stage Detection

Object Detection ->
Crop Bounding Box
-> Single-Label Classification

GitHub
Object Counting GitHub Roboflow Documentation (Object Counting)
Measure Object GitHub Roboflow Blog (Measure Object)
Send Email GitHub Roboflow Blog (Send Email)
Send Text GitHub Roboflow Blog: Send Text

Video Inference (Classification)

  • Model predictions for Classification models running via hosted or local deployment supported by Roboflow Deploy.

GitHub

Webcam Inference (Classification)

  • Model predictions for Classification models running via hosted or local deployment supported by Roboflow Deploy.

GitHub

Sample Video Frames

  • Extract and save video frames by a specified frames per second of video.

GitHub

💻 🛜 API Utilities:

Upload Images to a Specified Dataset Split (Train/Valid/Test)

GitHub

  • Upload images from a directory (folder) to a specified dataset split (train, valid, or test set) in your Roboflow project.
  • Base functionality also includes the ability to set a specific percentage of images to upload to the dataset from the specified directory, and the option to name the uploaded dataset batch

More on the Roboflow REST API: https://docs.roboflow.com/rest-api/

🔁 📈 Active Learning Utilities:

Automate improvement of your dataset by using computer vision and conditional upload logic to determine which images should be directly uploaded to your Roboflow workspace.

model type images video
Object Detection GitHub GitHub

Conditionals - Source Code:

  • roboflow-python/roboflow/util/active_learning_utils.py: GitHub
  • roboflow-python/roboflow/core/workspace.py Line 245: GitHub
# set the conditionals values as necessary for your active learning needs
# NOTE - not all conditional fields are required
conditionals = {
    "required_objects_count" : 1,
    "required_class_count": 1,
    "target_classes": ["class_name"],
    "minimum_size_requirement" : float('-inf'),
    "maximum_size_requirement" : float('inf'),
    "confidence_interval" : [10,90],
    "similarity_confidence_threshold": .3,
    "similarity_timeout_limit": 3
}

The Roboflow API JSON Response Object Format

Visualized Roboflow API JSON Response Object Format

twoPass.py: Code for running inference (model predictions) in "two stages" on images.

# default root save location: './inference_images'
# also moves all images in /roboflow-computer-vision-utilities to /roboflow-computer-vision-utilities/inference_images before inferrence
cd images
python3 twoPass.py
  • Ex. Stage 1: object detection (find faces) --> crop the detected areas and send to --> Stage 2: classification (is this a real image or illustrated?)
  • To be used after updating the roboflow_config.json file in the main directory with your Model Info (Workpsace ID, Model/Project ID, Private API Key and Model Version Number)
  • Available in roboflow-computer-vision-utilities/images

trigger_power_automate.py: make predictions on images, save the results, and send an email to the email specified in roboflow_config.json

# default confidence and overlap for predictions: confidence = 40, overlap = 30
cd images
python3 trigger_power_automate.py
  • To be used after updating the roboflow_config.json file in the main directory with your Model Info (Workpsace ID, Model/Project ID, Private API Key and Model Version Number), and email address to send the inference results to.

Streamlit App for Testing Roboflow Object Detection Models

This app allows you to upload an image to be inferenced by an Object Detection model trained with Roboflow Train

The app example app can be found here:

The app will work as-is for inference on individual image files (png, jpeg, jpg formats). If you want to build your own model, you'll need your own API key. Create a Roboflow account to get your own API key.

The app was created using Roboflow and Streamlit.

Example Code Snippets

Receive model predictions from a single image file:

img_path = 'INSERT_PATH_TO_IMG' # .jpg, .jpeg, .png
img = cv2.imread(img_path)

# perform inference on the selected image
predictions = model.predict(img_path, confidence=40,
    overlap=30)

Receive model predictions from images contained in a folder (directory):

raw_data_location = "INSERT_PATH_TO_DIRECTORY"

for raw_data_extension in ['.jpg', '.jpeg', 'png']:
## using the following line for raw_data_externsion results in inference on
## specified file types only
  raw_data_extension = ".jpg" # e.g jpg, jpeg, png
  globbed_files = glob.glob(raw_data_location + '/*' + raw_data_extension)
  for img_path in globbed_files:
      img = cv2.imread(img_path)
      predictions = model.predict(img_path, confidence=40, overlap=30)

Drawing Bounding Boxes

# main bounding box coordinates from JSON response object
# https://docs.roboflow.com/inference/hosted-api#response-object-format
x0 = bounding_box['x'] - bounding_box['width'] / 2
x1 = bounding_box['x'] + bounding_box['width'] / 2
y0 = bounding_box['y'] - bounding_box['height'] / 2
y1 = bounding_box['y'] + bounding_box['height'] / 2

# position coordinates: start = (x0, y0), end = (x1, y1)
# color = RGB-value for bounding box color, (0,0,0) is "black"
# thickness = stroke width/thickness of bounding box
# draw and place bounding boxes
start_point = (int(x0), int(y0))
end_point = (int(x1), int(y1))

cv2.rectangle(img, start_point, end_point, color=(0,0,0), thickness=2)

Writing and Placing Text:

# write and place text
cv2.putText(
    img, # PIL.Image object to place text on
    'placeholder text',#text to place on image
    (12, 12),#location of text in pixels
    fontFace = cv2.FONT_HERSHEY_SIMPLEX, #text font
    fontScale = 0.6,#font scale
    color = (255, 255, 255),#text color in RGB
    thickness=2#thickness/"weight" of text
    )

Drawing "Filled" Bounding Boxes:

# main bounding box coordinates from JSON response object
# https://docs.roboflow.com/inference/hosted-api#response-object-format
x0 = bounding_box['x'] - bounding_box['width'] / 2
x1 = bounding_box['x'] + bounding_box['width'] / 2
y0 = bounding_box['y'] - bounding_box['height'] / 2
y1 = bounding_box['y'] + bounding_box['height'] / 2

# position coordinates: start = (x0, y0), end = (x1, y1)
# color = RGB-value for bounding box color, (0,0,0) is "black"
# thickness = stroke width/thickness of bounding box
# draw and place bounding boxes
start_point = (int(x0), int(y0))
end_point = (int(x1), int(y1))

# setting thickness to -1 --> filled bounding box with the specified color
cv2.rectangle(img, start_point, end_point, color=(0,0,0), thickness=-1)

Blurring the Contents of Bounding Boxes:

# rip bounding box coordinates from current detection
# note: infer returns center points of box as (x,y) and width, height
# ----- but pillow crop requires the top left and bottom right points to crop
x0 = prediction['x'] - prediction['width'] / 2
x1 = prediction['x'] + prediction['width'] / 2
y0 = prediction['y'] - prediction['height'] / 2
y1 = prediction['y'] + prediction['height'] / 2
box = [(x0, y0), (x1, y1)]
blur_x = int(prediction['x'] - prediction['width'] / 2)
blur_y = int(prediction['y'] - prediction['height'] / 2)
blur_width = int(prediction['width'])
blur_height = int(prediction['height'])

# region of interest (ROI), or area to blur
roi = img[blur_y:blur_y+blur_height, blur_x:blur_x+blur_width]

# ADD BLURRED BBOXES
# set blur to (31,31) or (51,51) based on amount of blur desired
blur_image = cv2.GaussianBlur(roi,(51,51),0)
img[blur_y:blur_y+blur_height, blur_x:blur_x+blur_width] = blur_image

Cropping the Contents of Bounding Boxes:

for bounding_box in predictions:
  # defining crop area [height_of_cropArea:width_of_cropArea]
  # croppedArea = img[start_row:end_row, start_col:end_col]
  x0 = bounding_box['x'] - bounding_box['width'] / 2#start_column
  x1 = bounding_box['x'] + bounding_box['width'] / 2#end_column
  y0 = bounding_box['y'] - bounding_box['height'] / 2#start row
  y1 = bounding_box['y'] + bounding_box['height'] / 2#end_row
  class_name = bounding_box['class']
  croppedArea = img[int(y0):int(y1), int(x0):int(x1)]

  # position coordinates: start = (x0, y0), end = (x1, y1)
  # color = RGB-value for bounding box color, (0,0,0) is "black"
  # thickness = stroke width/thickness of the box, -1 = fill box
  cv2.rectangle(croppedArea,
    (int(10), int(10)), (int(80), int(40)), color=(0,0,0),
    thickness=-1)

  # write class name on image, and print class name
  cv2.putText(
      croppedArea, # cv2 image object to place text on
      class_name,# text to place on image
      (20, 20),# location of text in pixels
      fontFace = cv2.FONT_HERSHEY_SIMPLEX,# text font
      fontScale = 0.4,# font scale
      color = (255, 255, 255),#text color in RGB
      thickness=2# thickness/"weight" of text
      )

  # SAVE CROPPED IMAGES
  cv2.imwrite(f'crop{i}_' + os.path.basename(img_path), croppedArea)
  i+=1

About

Interface with the Roboflow API and Python package for running inference (receiving predictions) and customizing result images from your Roboflow Train computer vision models.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published