Input Data format for spconv modules

[FrancescoMandru]

I have a question regarding how to structure the point cloud in order to feed it into the spconv different modules. As the official repository seems to be inactive I'm asking you how to prepara a point cloud. That' my attempt:

    ptc = load_velo_scan(data[0])[:, 0:3] # Load the point cloud

    # ------------------ #
    #  Data Preparation  #
    # ------------------ #
    xyzmin = np.floor(np.percentile(ptc, 0, axis=0)).astype(np.int32)
    xyzmax = np.ceil(np.percentile(ptc, 100, axis=0)).astype(np.int32)
    ori_points = torch.tensor(ptc, requires_grad=True)
    voxel_generator = spconv.utils.VoxelGeneratorV2(voxel_size=[0.5] * 3,
                                                    point_cloud_range=[xyzmin[0], xyzmin[1], xyzmin[2], xyzmax[0], xyzmax[1], xyzmax[2]],
                                                    max_num_points=50,
                                                    max_voxels=20000,
                                                    full_mean=False)

    # ------------------------------- #
    #  Associated with compute graph  #
    # ------------------------------- #
    # 1. retain raw indice.
    ori_points_ = ori_points.detach()
    batch_point_indice = torch.arange(0, ori_points_.shape[0]).float().unsqueeze(1)  # [N, 1]
    # [N,3] -> [N,4], the add '1' is the indice in N
    ori_points_ = torch.cat((ori_points_, batch_point_indice), dim=1).view(-1, 4)
    # 2. generate voxels
    # features[num_voxels, point_per_voxel, 3+1] 1(indice), coords: [num_voxels, 1+3] 1(batch)
    # res contains:
    #   voxels: [M, max_points, ndim] float tensor. only contain points.
    #   coordinates: [M, 3] int32 tensor. zyx format.
    #   num_points_per_voxel: [M] int32 tensor.
    res = voxel_generator.generate(ori_points_.numpy())

    # https://github.com/traveller59/spconv/issues/142
    # https://github.com/traveller59/spconv/issues/188

    # features: [num_voxels, max_num_points_per_voxel, 3]
    gen_fea = torch.tensor(res['voxels'], dtype=torch.float32)
    # coors: [num_voxels, 3]
    gen_coords = torch.tensor(res['coordinates'], dtype=torch.int32)
    batch_idx = torch.zeros((gen_coords.shape[0], 1), dtype=torch.int32)  # batch=0
    gen_coords = torch.cat((batch_idx, gen_coords), dim=1)
    # # num_points: [num_voxels]
    gen_npoints = torch.tensor(res['num_points_per_voxel'], dtype=torch.int32)
    # 3. reconnect with compute graph
    gen_fea[:, :, :3] = ori_points[gen_fea[:, :, 3].long(), :]
    # # features[num_voxels, point_per_voxel, 3], N.B. set pos value to 0 in indice 0 (filling value)
    gen_fea = gen_fea[:, :, :3]
    # 4. you can do anything
    # voxel_vfe: [num_voxels, 3]
    voxel_vfe = torch.mean(gen_fea, dim=1)
    detached_vox = voxel_vfe.detach().numpy()
    spatial_shape = np.ceil(np.percentile(detached_vox, 100, axis=0) - np.percentile(detached_vox, 0, axis=0)).astype(np.int32)
    voxel_feature = spconv.SparseConvTensor(features=voxel_vfe, indices=gen_coords,
                                            spatial_shape=[400,400,400], # Don't know how to set it !!!!
                                            batch_size=1)

    simple_net = spconv.SubMConv3d(in_channels=3, out_channels=32,
                                   kernel_size=3, stride=1)
    output = simple_net(voxel_feature).dense()

If this procedure is not straightforward to you, cound you please provide an example of data preparing for a very simple network? Just to clarify it, thanks

[xinge008]

voxel_feature = spconv.SparseConvTensor(features=voxel_vfe, indices=gen_coords,
                                            spatial_shape=[400,400,400], # Don't know how to set it !!!!
                                            batch_size=1)

This is the keypoint, and spatial_shape is the range of coords (larger or equal to range of coordinate).

[FrancescoMandru]

@xinge008 In your opinion, suppose that I didn't voxelized my point cloud. What is voxel_fce and gen_coord ?? If I have onlu a point cloud (N, 3)