SuperPoint + LightGlue Batch Inference with ONNX

This repository provides a minimal and efficient implementation for running batch inference with the SuperPoint + LightGlue pipeline using ONNX Runtime. It is optimized for CUDA/GPU inference and supports dynamic batching with interleaved image pairs.

---

🚀 Features

• ✅ Dynamic batching of image pairs
• ✅ ONNX Runtime with CUDA (GPU) acceleration
• ✅ Interleaved input format for LightGlue ONNX compatibility
• ✅ SuperPoint preprocessing built-in
• ✅ Keypoint and match visualization using matplotlib

---

🧠 Model Info

Use one of the following models provided by LightGlue:

Model Type	File Name Suffix	Input Shape	Output Shapes
ONNX-only	*_lightglue_pipeline.onnx	(2B, 1, H, W)	(2B, 1024, 2), (M, 3), (M,)
ONNX Runtime (recommended)	*_lightglue_pipeline.ort.onnx	(2B, 1, H, W)	Same as above
TensorRT (static shape)	*_lightglue_pipeline.trt.onnx	(2, 1, 1024, 1024)	(2, 1024, 2), (M, 3), (M,)

All models were exported with --num-keypoints 1024.

---

🖼️ Input Format

Input images must be provided as pairs of left and right views, for example:

image_pairs = [
    ("assets/image_left1.jpg", "assets/image_right1.jpg"),
    ("assets/image_left2.jpg", "assets/image_right2.jpg"),
    ...
]

These images will be interleaved as:

[L0, R0, L1, R1, L2, R2, ...]

This interleaving is essential for correct inference.

---

🛠 Installation

1. Clone the repo

git clone https://github.com/your-username/superpoint-lightglue-batch.git
cd superpoint-lightglue-batch

2. Install dependencies

pip install -r requirements.txt

Or install manually:

pip install onnxruntime-gpu opencv-python numpy matplotlib

---

▶️ Run Inference

1. Add your image pairs to image_pairs in run_batch_inference.py.

2. Run the script:

python run_batch_inference.py

This will:

• Preprocess the images
• Run inference on the ONNX model
• Visualize matches for each image pair

---

📂 Project Structure

.
├── weights/
│   └── superpoint_lightglue_pipeline.onnx      # Your ONNX model
├── assets/
│   ├── image_left1.jpg
│   ├── image_right1.jpg
│   └── ...                                     # Input images
├── run_batch_inference.py                      # Main script
├── README.md
└── requirements.txt

---

📊 Optional: Enable Profiling

Set profile = True in the script to generate an ONNX Runtime profiling file:

profile = True

This will produce a JSON file viewable in Chrome DevTools for performance analysis.

---

🖼️ Output Visualization

Each image pair will be shown side-by-side with matched keypoints visualized using colored lines (default: lime green).

Example:

Use viz.plt.savefig() instead of viz.plt.show() to export results.

---

⚙️ Advanced Notes

• Input shape must be (2B, 1, H, W) — do not use (B, 2, 1, H, W).
• Matches are returned as (M, 3) where each row is:

  [batch_index, index_in_left_keypoints, index_in_right_keypoints]
  

• match_scores is aligned with matches, shape (M,).

---

🤝 Acknowledgements

• LightGlue by CVG, ETH Zürich
• SuperPoint
• ONNX Runtime by Microsoft
• Visualization from lightglue_dynamo utilities

---

📄 License

This project is licensed under the MIT License. See LICENSE for details.