Awesome-Efficient-Deep-Learning

Courses

TinyML and Efficient Deep Learning Computing (6.5940 • Fall • 2024) • https://efficientml.ai

Knowledge Distillation

• 2022-ECCV-CMKD: Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection

• 2023-CVPR-UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye View

• 2024-CVPR-CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation

• 2024-ECCV-LabelDistill: Label-guided Cross-modal Knowledge Distillation for Camera-based 3D Object Detection

• 2024-CVPR-CrossKD: Cross-Head Knowledge Distillation for Dense Object Detection

TinyML

• 2020-NeurIPS-TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning

• 2020-NeurIPS-MCUNet: Tiny Deep Learning on IoT Devices ‍

• 2021-NeurIPS-MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning ‍

• 2022-NeurIPS-MCUNetV3: On-Device Training Under 256KB Memory‍

Computer Vision

• 2023-ICCV-EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction

Large Language Model

• 2023-ICML-SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
• 2024-MLSys (Best Paper)-AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration