Skip to content

3D augmentation and transforms of 2D/3D sparse data, such as 3D triangle meshes or point clouds in Euclidean space. Extension of the Fast.ai library to train Sub-manifold Sparse Convolution Networks

License

Notifications You must be signed in to change notification settings

goodok/fastai_sparse

Repository files navigation

fastai_sparse

This is an extension of the fast.ai library to train Submanifold Sparse Convolution Networks that apply to 2D/3D sparse data, such as 3D geometric meshes or point clouds in euclidian space

Currently, this library has SparseConvNet under the hood which is the best in 3D (ScanNet benchmark, ShapeNet workshop) so far.

Installation

fastai_sparse is compatible with: Python 3.6, PyTorch 1.0+

Some key dependences:

  • Fast.ai
  • PyTorch sparse convolution models: SparseConvNet.
  • PLY file reader and 3D geometry mesh transforms are implemented by trimesh.
  • ipyvolume is used for interactive visualisation in jupyter notebooks examples.

See details in INSTALL.md

Features:

  • fast.ai train/inference loop concept (Model + DataBunch --> Learner) Classes overview
  • model training best practices provided by fast.ai (Learning Rate Finder, One Cycle policy)
  • 3D transforms for data preprocessing and augmentation:
    • mesh-level transforms and features extraction (surface normals, triangle area,...)
    • points-level spatial transforms (affine, elastic,...)
    • points-level features (color, brightness)
    • mesh to points
    • points to sparse voxels
  • metrics (IoU, avgIoU, ) calculation and tracking
  • visualization utils (batch generator output)

Notebooks with examples

TODO

Priority 1:

  • Separate 3D augmentation library with key points, spatial targets
  • Prediction pipeline
  • Classification/regression examples
  • Spatial targets (bounding box, key points, axes)

Priority 2:

  • TTA
  • Multi-GPU
  • PointNet-like feature extraction layer ("VoxelNet" architecture)
  • Confidence / heatmap / kernels visualization

About

3D augmentation and transforms of 2D/3D sparse data, such as 3D triangle meshes or point clouds in Euclidean space. Extension of the Fast.ai library to train Sub-manifold Sparse Convolution Networks

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published