colmap2nerf.py (colmap feature_extractor): qt.qpa.xcb: XKeyboard extension not present on the X server

[compressionist]

Hi everyone,

I am trying to run colmap2nerf.py for my videos and images. I am running it from a remote server via VNC, and unfortunately, I am getting the following error.

:~/instant-ngp$ python scripts/colmap2nerf.py --video_in data/closet2/611A6706.MP4 --video_fps 1 --run_colmap --aabb_scale 16
...
running colmap with:
	db=colmap.db
	images=data/closet2/images
	sparse=colmap_sparse
	text=colmap_text
warning! folders 'colmap_sparse' and 'colmap_text' will be deleted/replaced. continue? (Y/n)Y
==== running: colmap feature_extractor --ImageReader.camera_model OPENCV --ImageReader.single_camera 1 --database_path colmap.db --image_path data/closet2/images
qt.qpa.xcb: XKeyboard extension not present on the X server
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-ubuntu'
QEGLPlatformContext: eglMakeCurrent failed: 3009

Does anyone have experience running CОLMAP via VNC? Or do I need another tool instead of VNC like TurboVNC mentioned on the VirtualGL project home page: https://virtualgl.org?

[compressionist]

From the Colmap documentation, if there is no attached display, the Colmap should automatically switch to the GPU. But so far, it doesn't work on my GPU instance. I get errors, as I reported in my previous post or other errors if I switch to tigervnc-standalone-server instead of tightvncserver.

==== running: colmap feature_extractor --ImageReader.camera_model OPENCV --ImageReader.single_camera 1 --database_path colmap.db --image_path data/closet2/images
qt.qpa.xcb: could not connect to display localhost:0.0
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

So I added to colmap2nerf.py the --CPU_based _feature_extraction_and_matching parameter to be able to run the Colmap feature extraction and matching with CPU by setting the --SiftExtraction.use_gpu 0 and --SiftMatching.use_gpu 0 options as follows:

   if args.CPU_based_feature_extraction_and_matching:
                do_system(f"colmap feature_extractor --SiftExtraction.use_gpu 0 --ImageReader.camera_model OPENCV --ImageReader.single_camera 1 --database_path {db} --image_path {images}")
                do_system(f"colmap {args.colmap_matcher}_matcher --SiftMatching.use_gpu 0 --database_path {db}")
   else:
                do_system(f"colmap feature_extractor --ImageReader.camera_model OPENCV --ImageReader.single_camera 1 --database_path {db} --image_path {images}")
                do_system(f"colmap {args.colmap_matcher}_matcher --database_path {db}")

After this fix, colmap2nerf.py ran without any issues for my dataset. Still, unfortunately, it's not the solution I'm expecting, especially when I have a powerful GPU that suppose to be much faster than a CPU to handle these kinds of tasks.

[Tom94]

It's probably worth opening an issue over at COLMAP's github repo to see whether they're able to come up with a fix. Cheers!

[jhormigo]

Did you ever open this isue at COLMAP's repo or found a solution for it that uses GPU?

[GuintherKovalski]

Seems that it is a conflict between cv2 (opencv-python) and QT5 that causes this error. I was able to run this on GPU with:

pip uninstall opencv-python
pip install opencv-python-headless

right after pip install -r requirements.txt.

[RobinWitch]

I used to install colmap by using command apt install colmap ,
Using this method, I meet a similar problem with you .
Then I try to build colmap from source code ,by the guide of this colmap installl ,
This question disappears...
It's quite strange , but useful for me .
Hope this method can help you!