SmallVec is mostly a drop in replacement for Vec. There are two methods on std::vec::IntoIter that allow random access on the remainder of the iterator: as_slice and as_mut_slice.
These are missing in SmallVec, but implementing them would be rather easy.