Params to adjust for high-res input?

[sweeneychris]

I'm running ORB-SLAM2 on some videos captured with a drone. The drone can easily capture 1080P video at 60 fps, but when I run it through ORB-SLAM2 it obviously does not work anywhere near real-time any more. I do not need the system to actually run in real time, but I was wondering if you could point me to some parameters that I should look to tune for this high-resolution input? Currently I have only increased the number of features to 4000. Also, is there a way to increase the number of times Global BA is run? Since we are fine with not-real-time performance, it would be great to have the highest quality possible.

[raulmur]

If you do not care about real-time you can even extract more features. The default is 1000 for 640x480 at 8 scales. I would say you could extract 6000 for 1920x1080. I would also suggest to compute more scales, let say 13 or 14 scales to get a similar lowest resolution as 640x480 with 8 scales. If you compute more scales you will need to extract more features (as they are distributed along scales).

About Global BA, you can launch it whenever you want. Just look at how you can check if Global BA is running here: https://github.com/raulmur/ORB_SLAM2/blob/master/src/LoopClosing.cc#L412-L421
and how you can launch a new Global BA here: https://github.com/raulmur/ORB_SLAM2/blob/master/src/LoopClosing.cc#L572-L575

You could also expand the number of keyframes and points in the Local BA (you will need to ensure that the tracking processes frames, an therefore the keyframe insertion rate, is low enough so that the Local BA can do some iterations). To increase the number of keyframes in the Local BA, you can just retrieve all neighbor keyframes at a distance N in the covisibility graph (the current implementiation N=1) and put them in lLocalKeyFrames: https://github.com/raulmur/ORB_SLAM2/blob/master/src/Optimizer.cc#L461-L468

[sweeneychris]

Thanks so much for the pointers! And very much thanks for an excellent and robust open-source library

[sweeneychris]

Could you explain what the variable nLoopKF does in the LoopClosure::RunGlobalBundleAdjustment? I am having a hard time understanding it from the code alone

[raulmur]

The global BA saves the results of the optimization on the keyframe and mappoints structures (KeyFrame::mTcwGBA, KeyFrame::mTcwBefGBA, MapPoint::mPosGBA) and associated to nLoopKF (KeyFrame::mnBAGlobalForKF, MapPoint::mnBAGlobalForKF), which is the id of the keyframe when the loop was detected. After the optimization, local mapping is stopped and the current state of the map and the output of the global BA are merged: https://github.com/raulmur/ORB_SLAM2/blob/master/src/LoopClosing.cc#L672-L733 . There you need to check if a keyframe or a mappoint has been optimized by global BA by using nLoopKF.

[BW25]

I am attempting to use a higher resolution input myself (1280-720).
However, whenever I use ORBSLAM2 with higher resolution video or live input, it always fails to initialize.

I have adjusted the camera settings file TUM1 with our camera parameters for f, c, k, and p, but cannot find where to change the resolution of the input image.

How can I adjust ORBSLAM2 to use higher resolution video?
Failing that, it it possible to scale live video with ROS?
Thank you.