Skip to content

Make Rendered Database using Rendered Image Obtained by Trained Gaussian Splatting

Notifications You must be signed in to change notification settings

SungJaeShin/Rendered_DB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Rendered Database

[Goal] Make Rendered Database using Rendered Image Obtained by Trained 3D Gaussian Splatting

Get Rendered Image using 3D Gaussian Splatting Rasterization

  • Benchmark Dataset → Tanks and Temples
Render Img1 (Novel View Synthesis) Render Img2 (Novel View Synthesis) Render Img3 (Novel View Synthesis)

Prerequisites

  1. 3D Gaussian Splatting
  2. NetVLAD Pytorch
  3. Faiss
  4. Anaconda
  5. OpenCV with Contrib
  6. RoMa
  7. LightGlue
  8. SuperPoint
  9. Feature Matching

Download pre-trained model

Download pre-trained model


Preparing Dataset (in example folder)

(1) cameras.txt → [IMAGE_ID, CAMERA TYPE, IMAGE Width, IMAGE HEIGHT, ~]
(2) images.txt → [IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME]
(3) image folder


Combined Preparing Dataset (Make KeyFrame DB)

  • Using NetVLAD Method to make global descriptor

    • KeyFrame = {Index, Image, Keypoints, Descriptor(local or global), Camera Pose, Camera Type, Camera Params}
  • Final Database Extension file is HDF5


Method of Keypoint and Descriptor (in config.yaml)

  • extractor_method (extractor_utils.py):

    • 0 → apply ORB
    • 1 → apply SIFT
    • 2 → apply AKAZE
    • 3 → apply SuperPoint Model
    • roma_based_extractor function → apply RoMa Model
  • descriptor_method (extractor_utils.py):

    • 0 → apply ORB
    • 1 → apply SURF
    • 2 → apply DAISY
    • 3 → apply AKAZE
    • 4 → apply SuperPoint Model
  • matcher_mode (matching_utils.py):

    • calculate_score function → apply BF Matcher (Brute-Force) + KNN Matches
    • roma_based_extractor function → apply RoMa Model
    • lightglue_matcher function → apply LightGlue Model
  • REJECTION_MODE (using cv2):

    • apply cv2.fundamentalMatrix

Example Results

  • (1) Find the Nearest Distance using Faiss obtained by VLAD vector (Find Candidates)

  • (2) Find Feature Matching using Local Descriptor (Finda the Best Candidates)

    Matching Results (ORB + DAISY + distance=0.99)
    Matching Results (RoMa indoor + reproj=0.01)

About

Make Rendered Database using Rendered Image Obtained by Trained Gaussian Splatting

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published