[TOC]

Deep-Learning-On-Point-Clouds

A curated list of primary sources involving papers, books, blogs on the research theme applying deep learning on point cloud data. Moreover, I will try to summarize these primary sources with a note.

For papers, each note will try to summarize the basic background, main proposals, key components of the proposals, architecture, code implementation, methodology part, potential use of the paper and etc. For books, briefly list out which parts are important to the research topic.

Papers for Point clouds

classification, semantic segmentation and instance segmentation for point clouds

1. classification and semantic segmentation 1.1 primary sources

• PointNet (CVPR 2017 oral) | my note
• PointNet++ (NIPS 2017) | my note
• Relation-CNN (CVPR 2019 Oral)) | my note
• RandLA-Net (CVPR 2020 oral) | my note
• PointCNN (NIPS2018)
• PointConv (CVPR2018)
• DGCNN (ACM Transactions on Graphics 2019)
• KpConv (ICCV 2019)
• superpoint graph (CVPR2018 & CVPR2019)

1.2 other source

• Volumetric and multi-view CNNs (CVPR 2016 spotlight)
• Render for cnn (ICCV 2015)
• SPLATNet (CVPR2018)
• So-Net (CVPR 2018)
• OctNet (CVPR 2017)
• PU-Net (CVPR 2018)
• PointRCNN (CVPR 2019)
• Scan2CAD (CVPR 2019)
• PartNet (CVPR 2019)
• GeoNet (CVPR2018)
• SPFN (CVPR 2019 Oral)
• VoxelNet (CVPR 2017)
• Tangent Convolutions (CVPR 2018)
• PCN (3DV 2018 (Oral))
• 3DMatch (CVPR 2017 Oral)
• 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks, ScanNet No.1 on the leatherboard(26th,sep,19)
• 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks and Submanifold Sparse Convolutional Networks, ScanNet No.2 on the leatherboard(26th,sep,19)

2.instance segmentation 1. primary sources

• SGPN (CVPR 2018)
• ASIS, CVPR19
• 3D-SIS (CVPR 2019)
• 3D-BoNet
• JSIS3D
• GSPN (CVPR 2019)

2. other sources todo

detection and flow analysis for point clouds

1. detection 1.1 primary sources

• F-PointNet (CVPR 2018)
• VoteNet (ICCV 2019)
• 3D Fully Convolutional Network for Vehicle Detection in Point Cloud

2.scene flow analysis

• FlowNet3D (CVPR 2019)

generation/synthesis for point clouds

1. primary sources

• r-GAN/l-GAN/GMM-AE (ICML 2018)

Achlioptas, Panos, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. “Learning Representations and Generative Models for 3D Point Clouds(ICML-2018).” ArXiv:1707.02392 [Cs], June 12, 2018. http://arxiv.org/abs/1707.02392.

• PointFlow (ICCV 2019 Oral)

Yang, Guandao, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. “PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows,” June 28, 2019. https://arxiv.org/abs/1906.12320v3.

• Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction – Lin et al. (AAAI 2018 )

2. other sources

• PointSetGeneration (CVPR 2017)
• 3D adversarial point clouds (CVPR 2018)
• 3D-point-cloud-generation (AAAI 2018 oral)

reconstruction and model retrieval for point clouds

1. reconstruction

1.1 primary sources

• 3D-point-cloud-generation (AAAI 2018)

1.2 other sources

2. model retrieval 2.1 primary sources 2.2 other sources

Papers for images, sequential data

review papers

• LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." Nature 521.7553 (2015): 436-444. pdf (Three Giants' Survey) ⭐⭐⭐⭐⭐
• Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. "A fast learning algorithm for deep belief nets." Neural computation 18.7 (2006): 1527-1554. pdf(Deep Learning Eve) ⭐⭐⭐
• Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. "Reducing the dimensionality of data with neural networks." Science 313.5786 (2006): 504-507. pdf (Milestone, Show the promise of deep learning) ⭐⭐⭐
• Domingos, Pedro. “A Few Useful Things to Know about Machine Learning.” Communications of the ACM 55, no. 10 (October 1, 2012): 78. pdf ⭐⭐
• Jordan, M. I., and T. M. Mitchell. “Machine Learning: Trends, Perspectives, and Prospects.” Science 349, no. 6245 (July 17, 2015): 255–60. pdf ⭐⭐

ImageNet Evolution

• Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012. pdf (AlexNet, Deep Learning Breakthrough) ⭐️⭐️⭐️⭐️⭐️

• Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014). pdf (VGGNet,Neural Networks become very deep!) ⭐️⭐️⭐️

• Szegedy, Christian, et al. "Going deeper with convolutions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. pdf (GoogLeNet) ⭐️⭐️⭐️

• He, Kaiming, et al. "Deep residual learning for image recognition." arXiv preprint arXiv:1512.03385 (2015). pdf (ResNet, Very very deep networks, CVPR best paper) ⭐️⭐️⭐️⭐️⭐️

• xception

• densenet

• LSTM, RNN, and some variants

image semantic segmentation

[1] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation.” in CVPR, 2015. [pdf] (FCN) ⭐⭐⭐⭐⭐

[2] L.C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. "Semantic image segmentation with deep convolutional nets and fully connected crfs." In ICLR, 2015. [pdf] () ⭐⭐⭐⭐⭐

[3] Pinheiro, P.O., Collobert, R., Dollar, P. "Learning to segment object candidates." In: NIPS. 2015. [pdf] ⭐⭐⭐⭐

• PSPNet
• Deeplab v3+

image detection

[1] Szegedy, Christian, Alexander Toshev, and Dumitru Erhan. "Deep neural networks for object detection." Advances in Neural Information Processing Systems. 2013. [pdf] ⭐⭐⭐

[2] Girshick, Ross, et al. "Rich feature hierarchies for accurate object detection and semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2014. [pdf] (RCNN) ⭐⭐⭐⭐⭐

[3] He, Kaiming, et al. "Spatial pyramid pooling in deep convolutional networks for visual recognition." European Conference on Computer Vision. Springer International Publishing, 2014. [pdf] (SPPNet) ⭐⭐⭐⭐

[4] Girshick, Ross. "Fast r-cnn." Proceedings of the IEEE International Conference on Computer Vision. 2015. [pdf] ⭐⭐⭐⭐

[5] Ren, Shaoqing, et al. "Faster R-CNN: Towards real-time object detection with region proposal networks." Advances in neural information processing systems. 2015. [pdf] ⭐⭐⭐⭐

[6] Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." arXiv preprint arXiv:1506.02640 (2015). [pdf] (YOLO,Oustanding Work, really practical, currently already YOLO v4) ⭐⭐⭐⭐⭐

[7] Liu, Wei, et al. "SSD: Single Shot MultiBox Detector." arXiv preprint arXiv:1512.02325 (2015). [pdf] ⭐⭐⭐

[8] Dai, Jifeng, et al. "R-FCN: Object Detection via Region-based Fully Convolutional Networks." arXiv preprint arXiv:1605.06409 (2016). [pdf] ⭐⭐⭐⭐

[9] He, Gkioxari, et al. "Mask R-CNN" arXiv preprint arXiv:1703.06870 (2017). [pdf] ⭐⭐⭐⭐

• SSD
• Yolo
• Fast R-CNN
• Faster R-CNN

other papers

todo

books

deep learning and machine learning

• Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition, Aurélien Géron, Sep, 2019. ⭐️⭐️⭐️⭐️⭐️️

a practical DL book with lots of examples and exercises using Scikit-learn, TF 2.x (keras included)

• Deep learning with python, François Chollet, Dec, 2017. ⭐️⭐️⭐️⭐️⭐️

an extremely classic and concise book on deep learning using keras.

• Deep Learning, Ian Goodfellow, etc, 2017 ⭐️⭐️⭐️⭐️⭐️

Deep Learning Bible

• Machine Learning Yearning, Andrew Ng, 2018. ⭐️⭐️⭐️⭐️

a collection of practical machine learning techniques to build yr intelligent system.

• Neural networks and deep learning, Michael Nielsen, around 2015. ⭐⭐️⭐️⭐️

• An introduction to statistical learning : with applications in R. Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, 2013. ⭐️⭐️⭐️

point clouds

• Topographic Laser Ranging and Scanning: Principles and Processing, 2nd Edition: Jie Shan, Charles K. Toth.

• Airborne and Terrestrial Laser Scanning, Vosselman George.

Acknowledgment

• Part of sources refer to floodsung's Deep-Learning-Papers-Reading-Roadmap

todos

notes for sem. seg.

• pointnet
• pointnet++
• RandLA-Net
• RS-CNN
• PointCNN
• KPConv

brief summary for 2d images

• classical nets, e.g.: AlexNet
• nets for img sem. seg.
• nets for img det.

notes for det.

• votenet