This repository provides the implementation and documentation necessary to run a deep learning-based grasp-proposition network. The network is designed to propose optimal grasp points for objects located in proximity to a robotic arm, facilitating accurate and efficient manipulation of objects in diverse settings. This is especially crucial in environments where precision and reliability are paramount, such as in manufacturing, logistics, and healthcare.
The codebase includes custom implementations of PointNet and PointNet++, two influential architectures for point cloud processing, all built using PyTorch for ease of use and flexibility. Users can leverage this repository to generate grasp points on everyday objects by inputting point cloud data captured from a LiDAR camera.
- Custom PointNet Implementations: Tailored versions of PointNet and PointNet++ designed for grasp point detection.
- Comprehensive Documentation: Detailed instructions and explanations to guide users through installation, usage, and understanding of the system.
- High Accuracy: Our models demonstrate an impressive 89.5% accuracy in predicting grasp points, ensuring reliability in practical applications.
- Visualization Tools: Built-in functionality to visualize point clouds and grasp points for better understanding and analysis.
Visualize the processed point clouds and proposed grasp points with our built-in tools:
# build C++ code for visualization
cd visualizer
bash build.sh
# run one example
python show3d_balls.py
Ensure that you have all the required dependencies installed. You can install them using:
pip install -r requirements.txt
To prepare your data for training and evaluation, follow the steps below:
- Download 3D indoor parsing dataset (S3DIS) here and save in
data/s3dis/Stanford3dDataset_v1.2_Aligned_Version/.
cd data_utils
python collect_indoor3d_data.py
- Processed data will save in
data/s3dis/stanford_indoor3d/.
Our deep learning grasp-proposition network demonstrates state-of-the-art performance, reaching up to 89.5% accuracy in proposing grasp points on various objects. This showcases the model's capability in understanding complex spatial arrangements and making intelligent decisions for robotic manipulation.
python train_classification.py --model pointnet2_cls_ssg --log_dir pointnet2_cls_ssg
python test_classification.py --log_dir pointnet2_cls_ssg
- Real World Testing: Plans are underway to deploy the network on actual robotic arms to validate its performance in real-world settings.
- Real World Testing: Plans are underway to deploy the network on actual robotic arms to validate its performance in real-world settings. Improved DataLoader: Enhancements to the DataLoader will be made to incorporate a more physics-aware understanding of objects, further improving the accuracy and reliability of grasp point predictions.Improved DataLoader: Enhancements to the DataLoader will be made to incorporate a more physics-aware understanding of objects, further improving the accuracy and reliability of grasp point predictions.
- Broader Dataset: We aim to extend the training dataset to include over 1000 different everyday objects, ensuring that the model generalizes well across various contexts.
- Continuous Learning: Implementing mechanisms for the model to continually learn and adapt to new objects and environments over time.