How it works

Template Matching and Edge Detection Wiki

This document provides an overview of how the two Python scripts work together for edge detection and template matching using OpenCV.

---

Overview

The project combines edge detection techniques and template matching to identify specific patterns (e.g., a currency note) in a video stream. It uses:

• Edge Detection: Canny, Sobel, or Laplacian methods for preprocessing.
• Template Matching: ORB (Oriented FAST and Rotated BRIEF) to find and match key points between a template image and a live video stream.

---

Edge Detection

File: EdgeDetection Class

The EdgeDetection class provides an interface to switch between different edge detection algorithms.

Key Components

Initialization

class EdgeDetection:
    def __init__(self, method='canny'):
        self.method = method

• Default edge detection method is canny.

Set Method

def set_method(self, method):
    self.method = method

• Updates the current edge detection method dynamically during runtime.

Edge Detection Methods

• Canny

def canny_edge_detection(self, gray_frame):
    edges = cv2.Canny(gray_frame, 50, 150)
    return edges

• Sobel

def sobel_edge_detection(self, gray_frame):
    grad_x = cv2.Sobel(gray_frame, cv2.CV_64F, 1, 0, ksize=3)
    grad_y = cv2.Sobel(gray_frame, cv2.CV_64F, 0, 1, ksize=3)
    abs_grad_x = cv2.convertScaleAbs(grad_x)
    abs_grad_y = cv2.convertScaleAbs(grad_y)
    edges = cv2.addWeighted(abs_grad_x, 0.5, abs_grad_y, 0.5, 0)
    return edges

• Laplacian

def laplacian_edge_detection(self, gray_frame):
    edges = cv2.Laplacian(gray_frame, cv2.CV_64F)
    edges = cv2.convertScaleAbs(edges)
    return edges

Usage in Main Loop

• Read frames from the webcam.
• Convert the frame to grayscale.
• Apply edge detection using the selected method.

edges = edge_detector.detect_edges(gray)
cv2.imshow('Edges', edges)

Switch Methods at Runtime

Users can dynamically switch between edge detection methods with keyboard inputs:

• Press c: Switch to Canny.
• Press s: Switch to Sobel.
• Press l: Switch to Laplacian.

---

Template Matching

File: TemplateMatcher Class

The TemplateMatcher class detects specific objects in a live video stream by matching keypoints in a template image and the video stream using ORB.

Key Components

Initialization

class TemplateMatcher:
    def __init__(self, template_path, edge_method='canny', popUpActive=False, camera_index=0):

• Loads a grayscale template image.
• Initializes the edge detection method and ORB parameters.
• Configures the camera source.

Match Found Popup

def match_found(self):
    app = QApplication(sys.argv)
    self.popup = SimplePopup("Match to Currency was found.")
    self.popup.show()

• Displays a popup notification when a match is found (requires PyQt6).

Edge Detection in Matching

def process_frame(self, frame, orb):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    edges = self.edge_detection.detect_edges(gray)

• Applies the edge detection method to the video stream before keypoint matching.
• Highlights detected edges in red.

Keypoint Matching

matches = self.bf.knnMatch(self.des_template, des_frame, k=2)
good_matches = [m for m, n in matches if len(matches) > 1 and m.distance < 0.75 * n.distance]

• Finds matches between template and frame descriptors using a brute-force matcher.
• Filters good matches based on distance ratio (Lowe's ratio test).

Homography and Bounding Box

if len(good_matches) > 150:
    src_pts = np.float32([self.kp_template[m.queryIdx].pt for m in good_matches]).reshape(-1, 1, 2)
    dst_pts = np.float32([kp_frame[m.trainIdx].pt for m in good_matches]).reshape(-1, 1, 2)
    M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)
    h, w = self.template.shape
    pts = np.float32([[0, 0], [0, h], [w, h], [w, 0]]).reshape(-1, 1, 2)
    dst = cv2.perspectiveTransform(pts, M)
    frame = cv2.polylines(frame, [np.int32(dst)], True, (0, 255, 0), 3, cv2.LINE_AA)

• Uses homography to map the detected object in the video stream and draws a green bounding box around it.

---

Tuning ORB Parameters

Finding the Best Parameters

def find_best_parameters(self):
    nfeatures_list = [500, 1000, 1500]
    scaleFactor_list = [1.1, 1.2, 1.3]
    nlevels_list = [8, 10, 12]
    ...
    for nfeatures in nfeatures_list:
        for scaleFactor in scaleFactor_list:
            ...
            orb = cv2.ORB_create(nfeatures=nfeatures, scaleFactor=scaleFactor, ...)
            self.kp_template, self.des_template = orb.detectAndCompute(self.template, None)

• Iteratively tests different ORB parameters to find the configuration with the highest number of good matches.
• Saves the best parameters to a JSON file.

Loading Parameters

def load_and_use_parameters(self):
    with open(r'best_orb_params.json', 'r') as f:
        params = json.load(f)
    orb = cv2.ORB_create(nfeatures=params['nfeatures'], scaleFactor=params['scaleFactor'], ...)

• Loads the best ORB parameters from the JSON file for consistent detection.

---

Runtime Controls

• Edge Detection Modes:

  • Canny: Press c.
  • Sobel: Press s.
  • Laplacian: Press l.

• Exit Application:

  • Press q to quit.

---

How It Works Together

1. Initialization:

   • EdgeDetection class preprocesses video frames using edge detection.
   • TemplateMatcher sets up ORB and matches the template against the video stream.

2. Runtime Matching:

   • Frames are captured, processed for edges, and keypoints are extracted.
   • ORB matches keypoints between the template and the frame.
   • If sufficient matches are found, a bounding box is drawn around the detected object.

3. Dynamic Tuning:

   • Find optimal ORB parameters using find_best_parameters.
   • Save and reuse the parameters for consistent results.

---

Dependencies

• OpenCV (cv2)
• NumPy (numpy)
• PyQt6 (for popups)

---

Example Usage

1. Find Best ORB Parameters

python template_matcher.py find

2. Run with Optimal Parameters

python template_matcher.py use

---