Skip to content

From a recorded session to COLMAP

S edited this page Aug 20, 2026 · 3 revisions

From a recorded session to COLMAP and Gaussian splatting

A side effect of recording AR sessions is that you end up holding something the 3D reconstruction world would like: camera frames that are already posed, captured on a walk, with GPS attached.

Why posed frames matter

Structure-from-motion pipelines like COLMAP normally begin by working out where each photo was taken. That step — feature matching, then bundle adjustment across every image pair — is the slow and fragile part. It is where reconstructions fail on textureless walls, repeated patterns and slow-moving capture, and it scales badly with image count.

An AR session skips it. The device's visual-inertial odometry already estimated the camera pose for every frame, at frame rate, using both camera and IMU. When the recorder captures a camera frame it captures the pose alongside it.

So the input to reconstruction is not "a folder of photos" but "a folder of photos with known extrinsics" — a substantially easier problem. Poses can be used as initialisation, which converges faster and more reliably, or as constraints.

And a GPS track that could make it georeferenced

The second thing the recording carries is a GPS track aligned to those poses. That is the raw material for something photogrammetry usually lacks entirely: absolute position and scale.

A conventional SfM reconstruction is correct only up to an unknown similarity transform — right shape, arbitrary scale, arbitrary position, arbitrary orientation. Getting it into world coordinates normally means ground control points or manual alignment. Here the transform between local and world coordinates is the thing the whole system estimates anyway, so the ingredients are already in the ZIP.

Being precise about the current state: the export is posed, not georeferenced. The camera poses are written in the AR session's own frame, and no alignment is applied to them. The GPS track ships alongside, so a consumer holding both can georeference the result themselves — but the exporter does not do it for them, and saying otherwise would be describing a plan as a feature.

Doing it properly turns out to have one non-obvious constraint worth knowing before anyone tries: do not export earth-centred coordinates. Every common Gaussian-splatting trainer stores geometry in 32-bit floats, and at earth-radius magnitudes the gap between two representable values is half a metre — positions snap to that lattice, and the gradients that should nudge them are smaller than one step, so the optimiser stalls. A local east-north-up frame with a recorded origin is the representation that survives the whole chain, and COLMAP's model files have nowhere to record that origin, so it has to travel in a sidecar of your own.

What people do with it

  • Gaussian splatting, which wants many posed images from varied viewpoints — precisely the shape of data a walk produces. Recorded ZIPs can be opened directly in third-party tooling such as LichtFeld-Studio, and viewed with colmapview.github.io.
  • Site documentation — walk a site once and keep a 3D record of how it looked on that date, with the GPS track alongside it to place it.
  • Content authoring — reconstruct the place and author AR content against the reconstruction indoors rather than on site. Getting the result back to the real location at the right coordinates needs the georeferencing step above, so today this works within a session's own frame rather than across devices.
  • Evaluation — a reconstruction is an independent check on the alignment. If your fused trajectory and the SfM-recovered trajectory disagree, one of them is wrong, and that is information you did not have before.
  • ML training data — synchronised camera, depth, pose and GPS is a respectable multimodal dataset, and hard to collect at scale by other means.

Honest limits

  • Phone frames captured while walking are not survey photography. Motion blur, rolling shutter and automatic exposure changes all hurt. The image-quality gate helps here: it drops the worst frames before they are ever stored, which is one of the few places where the gate's saving is banked rather than theoretical.
  • Odometry poses drift. They are excellent locally and accumulate error over a long walk, so they are a strong initialisation rather than ground truth to be trusted blindly.
  • GPS gives metres, not centimetres. Georeferencing inherits GNSS accuracy, which is the right precision for placing a building and the wrong one for aligning to a doorframe.
  • Coverage is what the user walked. Reconstruction quality follows viewpoint variety, and a person walking a path provides much less of it than someone deliberately orbiting a subject.

The point

None of this required building a capture rig or a reconstruction pipeline. It falls out of recording an AR session honestly — every channel, time-synchronised, in a portable file.

Which is a reasonable argument for capturing more than the immediate task demands. The recordings made to debug alignment turn out to be a dataset, and datasets outlive the reason they were collected.

Clone this wiki locally