RPTU-AI-Toolkit

A toolkit for preprocessing, training and inference
Explore the docs »

Report Bug or Request Feature

Table of Contents

1. About The Project
   • Built With
2. Installation
3. Features
4. Requirements
5. Program structure
   • File hierarchy
6. Usage
7. Roadmap
8. Useful links
9. Contact
10. License

About The Project

This is a toolkit to train a ML model for semantic segmentation of point clouds.

It covers preprocessing, training and inference, and in this repository it is used to make BIM models based on IfcOpenShell standard from lidar scans.

Built With

Installation

1. You can install the Minkowski Engine with pip, with anaconda, or on the system directly.

• PIP installation
• Conda installation
• Python installation
• Docker installation

2. Install Poetry:

   curl -sSL https://install.python-poetry.org | python3 -
   

3. Install project dependencies: poetry install

4. Set config parameters in config/config.yaml

5. poetry run python src/main.py

Features

• Unlimited high-dimensional sparse tensor support
• All standard neural network layers (Convolution, Pooling, Broadcast, etc.)
• Dynamic computation graph
• Custom kernel shapes
• Multi-GPU training
• Multi-threaded kernel map
• Multi-threaded compilation
• Highly-optimized GPU kernels

Requirements

• Ubuntu >= 14.04
• CUDA >= 10.1.243 and the same CUDA version used for pytorch (e.g. if you use conda cudatoolkit=11.1, use CUDA=11.1 for MinkowskiEngine compilation)
• pytorch >= 1.7 To specify CUDA version, please use conda for installation. You must match the CUDA version pytorch uses and CUDA version used for Minkowski Engine installation. conda install -y -c nvidia -c pytorch pytorch=1.8.1 cudatoolkit=10.2)
• python >= 3.6
• ninja (for installation)
• GCC >= 7.4.0

Program structure

Data Preparation

1. Raw Data Analysis:
   Generate report of each asc and pcd file, check for skewed distribution.

2. Preprocessing:
   Data Preparation: Feature selection, data cleansing, transforming, outlier removal, scaling (normalization/standardization). Feature Engineering: Imputation (managing missing data), one-hot encoding of categorical features.

3. Training:
   A classification model will be trained based on a CNN architecture

4. Inference:
   Using Blender's python wrapper a BIM-Model will be generated from the trained ML-Model

5. Reports:
   Training and Evaluation
   Iteration (hyperparameter tuning)
   Test and Verification
   Visualizing

File hierarchy

data/                   # Raw and processed data (pcd, asc, ply), mapping dictionaries (yaml)
logs/
models/             # Trained models
notebooks/       # Jupyter notebooks for exploration and visualization (sanity check of raw and processed data)
tests/
src/
| ── config/
│           |── config.yaml
│           |── preprocessing.yaml
│           |── training.yaml
│           └── inference.yaml
| ── data/
│           └── preprocessing.py
| ── models/
│           └── model.bin
| ── training/
│           |── train.py
│           └── evaluate.py
| ── inference/
│           └── infer.py
| ── utils/
│           └── utils.py
└ ── main.py

Usage

Please refer to the Documentation

Roadmap

• Add logs
• Add Demo
• Add Notebooks
• Multi-preprocessing Support
  • cpu
  • gpu

Useful links and other projects using Minkowski Engine

• Point clouds
• Open3d
• [Minkowski Engine][https://github.com/NVIDIA/MinkowskiEngine]

• Segmentation: 3D and 4D Spatio-Temporal Semantic Segmentation, CVPR'19
• Representation Learning: Fully Convolutional Geometric Features, ICCV'19
• 3D Registration: Learning multiview 3D point cloud registration, CVPR'20
• 3D Registration: Deep Global Registration, CVPR'20
• Pattern Recognition: High-Dimensional Convolutional Networks for Geometric Pattern Recognition, CVPR'20
• Detection: Generative Sparse Detection Networks for 3D Single-shot Object Detection, ECCV'20
• Image matching: Sparse Neighbourhood Consensus Networks, ECCV'20

Contact

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)