This lets certain operations, especially matrix inversion, be
quite a bit faster.  And we don't need anything beyond affine
transformations anyway.