Question about c2w

[kam1107]

Hi, just to confirm the format of c2w matrix. Is it c2w[:3,:3] corresponds to rot matrix in order [x,y,z] and c2w[:3,-1] the camera position in order [z,x,y]? I noticed in _truncate_with_plane_intersection you use rays_o[:, :, 0] < altitude.

[hturki]

It's the same format as in the original NeRF paper for LLFF, which I think is "down, right, backwards" with the rotation matrix then reordered as in https://github.com/bmild/nerf/blob/master/load_llff.py#L250

So yes, the first dimension of the camera position is altitude, which goes downwards.

[kam1107]

So in this code, the rotation matrix is not reordered?

[hturki]

The 'c2w' field in the metadata files in the provided datasets are already rotated with the operation in https://github.com/bmild/nerf/blob/master/load_llff.py#L250 - if you're using your own custom dataset you'll want to apply this operation before writing your metadata files

[kam1107]

Hmmmm I'm still confused. According to LLFF repo (https://github.com/Fyusion/LLFF#using-your-own-poses-without-running-colmap), before reordering, the three axes are [down, right, backwards], so I think after reordering it should be [right, up, backwards]?

[kam1107]

If 'c2w' in metadata is already reordered, should it be in order [right, up, backwards]?

[hturki]

If you look at https://github.com/bmild/nerf/blob/master/load_llff.py#L250, you'll notice that the column reordering ends up becoming (1, -0, 2, 3), ie: drb -> rub for the rotation matrix. But the position itself (col 3), is unchanged, and is hence still in drb coordinates

np.concatenate([poses[:, 1:2, :], -poses[:, 0:1, :], poses[:, 2:, :]], 1)

[kam1107]

Ahh I see. Thanks for your patience!