Awesome Video Coding Papers

A curated list of neural network-based video/image coding papers, datasets, and tools.

Tip

📚✍️ Contributions welcome! Submit a PR to add papers/datasets/tools.
Let's build a comprehensive resource together! 🚀

[!TIP] [!TIP]
📚✍️ All original paper files and code files are available upon request. Please feel free to email me at xiongweixiao@foxmail.com, and I will reply within 24 hours.

---

Table of Contents

• End-to-End Neural Video Coding
• Hybrid Neural-Traditional Coding
• Perceptual Video Coding
• Learned Image Compression
• Rate & Complexity Control
• Low-Latency & Real-Time Coding
• Datasets & Metrics
• Tools & Libraries

---

End-to-End Neural Video Coding

Fully learned video codecs without traditional block-based frameworks.

• [CVPR 2023] FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos, Chen et al.

  • Key Idea: Frame-wise implicit neural representation with motion-guided training.
  • Code | Bibtex

• [ICCV 2023] EVC: Enhanced Neural Video Compression with Per-Pixel Flow-Guided Alignment, Hu et al.

  • Key Idea: Optical flow-guided alignment for inter-frame coding.
  • Code | Bibtex

• [NeurIPS 2022] DCVC-DC: Deep Contextual Video Compression with Dynamic Convolutions, Li et al.

  • Key Idea: Dynamic convolution kernels for adaptive feature extraction.
  • Code | Bibtex

---

Hybrid Neural-Traditional Coding

Combining neural networks with traditional coding tools (e.g., HEVC/VVC).

• Frame Intra Prediction
• Frame Inter Prediction
• Transform Coding
• Quantization
• Entropy Coding
• Loop Filtering
• Rate-Distortion Optimization
• Rate Control
• Complexity Control
• Datasets
• Tools

Frame Intra Prediction

• [VCIP 2024] Vectorized Angular Intra Prediction for Practical VVC Encoding CODE

• [Multimedia Tools and Applications 2024] Fast CU partition strategy based on texture and neighboring partition information for Versatile Video Coding Intra Coding CODE

• [TCSVT 2024] STRANet: soft-target and restriction-aware neural network for efficient VVC intra coding CODE

• [ISCAS 2023] Ultra-Lightweight CNN Based Fast Intra Prediction for VVC Screen Content Coding CODE

• [TCSVT 2023] Machine Learning Based Efficient QT-MTT Partitioning Scheme for VVC Intra Encoders CODE

• [TCSVT 2023] Deep Multi-Task Learning Based Fast Intra-Mode Decision for Versatile Video Coding CODE

• [TIP 2023] Partition Map Prediction for Fast Block Partitioning in VVC Intra-Frame Coding CODE

• [TIP 2023] Adaptive Chroma Prediction Based on Luma Difference for H.266/VVC CODE

• [TMM 2022] Fast Intra Mode Decision Algorithm for Versatile Video Coding CODE

• [TMM 2022] Efficient VVC Intra Prediction Based on Deep Feature Fusion and Probability Estimation CODE

• [VCIP 2022] Autoencoder-based intra prediction with auxiliary feature CODE

• [VCIP 2022] Neural Frank-Wolfe Policy Optimization for Region-of-Interest Intra-Frame Coding with HEVC/H.265 CODE

• [PCS 2022] Effective VVC Intra Prediction Based on Ensemble Learning CODE

• [TCSVT 2022] HG-FCN: Hierarchical Grid Fully Convolutional Network for Fast VVC Intra Coding CODE

• [TCSVT 2022] Configurable Fast Block Partitioning for VVC Intra Coding Using Light Gradient Boosting Machine CODE

• [DCC 2022] Graph-based Transform based on 3D Convolutional Neural Network for Intra-Prediction of Imaging Data CODE

• [2022] An efficient low-complexity block partition scheme for VVC intra coding CODE

• [PCS 2021] Contour-based Intra Coding Using Gaussian Processes and Neural Networks CODE

• [VCIP 2021] Learning-Based Complexity Reduction Scheme for VVC Intra-Frame Prediction CODE

• [ICIP 2021] Intra To Inter: Towards Intra Prediction for Learning-Based Video Coders Using Optical Flow CODE

• [TIP 2021] DeepQTMT: A Deep Learning Approach for Fast QTMT-Based CU Partition of Intra-Mode VVC CODE

• [DCC 2021] Fast Partitioning for VVC Intra-Picture Encoding with a CNN Minimizing the Rate-Distortion-Time Cost CODE

• [ICMEW 2021] Open-Source: Attention-Based Neural Networks For Chroma Intra Prediction In Video Coding CODE

• [TMM 2020] Fast Multi-Type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network CODE

• [ICIP 2020] CNN Oriented Complexity Reduction Of VVC Intra Encoder CODE

• [ICIP 2020] Chroma Intra Prediction With Attention-Based CNN Architectures CODE

• [ICIP 2020] Multi-Mode Intra Prediction for Learning-Based Image Compression CODE

• [ICIP 2020] Optimized Convolutional Neural Networks for Video Intra Prediction CODE

• [VCIP 2020] Introducing Latent Space Correlation to Conditional Autoencoders for Intra Prediction CODE

• [VCIP 2020] Fully Neural Network Mode Based Intra Prediction of Variable Block Size CODE

• [ISCAS 2020] Fast partitioning decision scheme for versatile video coding intra-frame prediction CODE

• [TCSVT 2020] Low complexity CTU partition structure decision and fast intra mode decision for versatile video coding CODE

• [TCSVT 2020] Deep Learning-Based Chroma Prediction for Intra Versatile Video Coding CODE

• [Multimedia Systems 2020] Fast CU partition decision for H.266/VVC based on the improved DAG-SVM classifier model CODE

• [Multimedia Tools and Applications 2020] A fast CU size decision algorithm for VVC intra prediction based on support vector machine CODE

• [TBC 2019] Fast HEVC Intra Mode Decision Based on RDO Cost Prediction CODE

• [VCIP 2019] Adaptive CU Split Decision with Pooling-variable CNN for VVC Intra Encoding CODE

• [TCSVT 2019] Video Compression Using Generalized Binary Partitioning, Trellis Coded Quantization, Perceptually Optimized Encoding, and Advanced Prediction and Transform Coding CODE

• [TCSVT 2019] Multi-scale Convolutional Neural Network Based Intra Prediction for Video Coding CODE

• [TCSVT 2019] CNN-based Intra-Prediction for Lossless HEVC CODE

• [TIP 2019] Context-adaptive neural network-based prediction for image compression CODE

• [TCSVT 2019] A deep convolutional neural network approach for complexity reduction on intra-mode HEVC CODE

• [TMM 2019] Generative Adversarial Network-Based Intra Prediction for Video Coding CODE

• [TMM 2019] Enhanced Intra Prediction for Video Coding by Using Multiple Neural Networks CODE

• [JVCIR 2019] Intra Mode Prediction for H. 266/FVC Video Coding based on Convolutional Neural Network CODE

• [ISCAS 2019] CNN-Based Bi-Prediction Utilizing Spatial Information for Video Coding CODE

• [ICASSP 2019] Nonlinear Prediction of Multidimensional Signals via Deep Regression with Applications to Image Coding CODE

• [ICASSP 2019] Convolutional neural networks for video intra prediction using cross-component adaptation CODE

• [DCC 2019] Deep learning based angular intra-prediction for lossless HEVC video coding CODE

• [DCC 2019] Intra picture prediction for video coding with neural networks CODE

• [ICIP 2019] Look-Ahead Prediction Based Coding Unit Size Pruning for VVC Intra Coding CODE

• [TIP 2018] Fully Connected Network-Based Intra Prediction for Image Coding CODE

• [ICME 2017] A deep convolutional neural network approach for complexity reduction on intra-mode HEVC CODE

• [PCS 2016] Deep learning-based intra prediction mode decision for HEVC CODE

Frame Inter Prediction

• [ICMEW 2024] Adaptive Intra Period Size for Deep Learning-Based Screen Content Video Coding CODE

• [Pattern Recognition 2024] IBVC: Interpolation-driven B-frame video compression CODE

• [TIP 2024] Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding CODE

• [MMsys 2024] Inter-Frame Parallelization in an Open Optimized VVC Encoder CODE

• [AAAI 2023] Video Compression Artifact Reduction by Fusing Motion Compensation and Global Context in a Swin-CNN Based Parallel Architecture CODE

• [ICASSP 2023] Learned Video Coding with Motion Compensation Mixture Model CODE

• [DCC 2023] Butterfly: Multiple Reference Frames Feature Propagation Mechanism for Neural Video Compression CODE

• [TCSVT 2022] Neural Network-Based Enhancement to Inter Prediction for Video Coding CODE

• [TCSVT 2022] Deep Affine Motion Compensation Network for Inter Prediction in VVC CODE

• [TIP 2022] Neural Reference Synthesis for Inter Frame Coding CODE

• [VCIP 2022] Deep Reference Frame Interpolation based Inter Prediction Enhancement for Versatile Video Coding CODE

• [VCIP 2022] Deep Reference Frame Interpolation based Inter Prediction Enhancement for Versatile Video Coding CODE

• [ICIP 2022] Intra-Inter Prediction for Versatile Video Coding Using a Residual Convolutional Neural Network CODE

• [DCC 2021] Bi-Prediction Enhancement with Deep Frame Prediction Network for Versatile Video Coding CODE

• [DCC 2021] Deformable Convolution Network based Invertibility-Driven Interpolation Filter for HEVC CODE

• [ICIP 2021] Deep Video Compression for Interframe Coding CODE

• [VCIP 2021] Deep Inter Prediction via Reference Frame Interpolation for Blurry Video Coding CODE

• [PCS 2021] Switchable Motion Models for Non-Block-Based Inter Prediction in Learning-Based Video Coding CODE

• [ICCV 2021] Extending Neural P-frame Codecs for B-frame Coding CODE

• [TCSVT 2021] Motion Vector Coding and Block Merging in the Versatile Video Coding Standard CODE

• [TCSVT 2021] Subblock-Based Motion Derivation and Inter Prediction Refinement in the Versatile Video Coding Standard CODE

• [TCSVT 2020] Convolutional Neural Network Based Bi-prediction Utilizing Spatial and Temporal Information in Video Coding CODE

• [TCSVT 2020] A Robust Quality Enhancement Method Based on Joint Spatial-Temporal Priors for Video Coding CODE

• [TCSVT 2020] Compression Priors Assisted Convolutional Neural Network for Fractional Interpolation CODE

• [TCSVT 2020] Deep Network-Based Frame Extrapolation With Reference Frame Alignment CODE

• [TCSVT 2020] A Distortion-Aware Multi-task Learning Framework for Fractional Interpolation in Video Coding CODE

• [TCSVT 2020] Neural Video Coding Using Multiscale Motion Compensation and Spatiotemporal Context Model CODE

• [TIP 2020] Optical Flow Based Co-Located Reference Frame for Video Compression CODE

• [CVPR Workshops 2020] Joint Motion and Residual Information Latent Representation for P-Frame Coding CODE

• [CVPR Workshops 2020] P-Frame Coding Proposal by NCTU: Parametric Video Prediction Through Backprop-Based Motion Estimation CODE

• [ISCAS 2020] Memory-Augmented Auto-Regressive Network for Frame Recurrent Inter Prediction CODE

• [ICIP 2020] Interpreting CNN for Low Complexity Learned Sub-pixel Motion Compensation in Video Coding CODE

• [ICIP 2020] Deep Virtual Reference Frame Generation For Multiview Video Coding CODE

• [VCIP 2020] Deep Inter Coding with Interpolated Reference Frame for Hierarchical Coding Structure CODE

• [ICMEW 2020] CNN-Based Inter Prediction Refinement for AVS3 CODE

• [TCSVT 2019] Deep frame prediction for video coding CODE

• [TCSVT 2019] A Multi-Scale Position Feature Transform Network for Video Frame Interpolation CODE

• [TIP 2019] Three-Zone Segmentation-Based Motion Compensation for Video Compression CODE

• [TMM 2019] Deep reference generation with multi-domain hierarchical constraints for inter prediction CODE

• [ISCAS 2019] Switch mode based deep fractional interpolation in video coding CODE

• [ISCAS 2019] CNN-Based Bi-Prediction Utilizing Spatial Information for Video Coding CODE

• [DCC 2019] Deep frame interpolation for video compression CODE

• [ICIP 2019] Advanced cnn based motion compensation fractional interpolation CODE

• [PCS 2019] HEVC Inter Coding using Deep Recurrent Neural Networks and Artificial Reference Pictures CODE

• [PCS 2019] An Extended Skip Strategy for Inter Prediction CODE

• [PCS 2019] Recent development of AVS video coding standard: AVS3 CODE

• [TCSVT 2018] Enhanced Bi-Prediction With Convolutional Neural Network for High-Efficiency Video Coding CODE

• [TCSVT 2018] Triple-Frame-Based Bi-Directional Motion Estimation for Motion-Compensated Frame Interpolation CODE

• [TCSVT 2018] Weighted Convolutional Motion-Compensated Frame Rate Up-Conversion Using Deep Residual Network CODE

Transform Coding

• [TBC 2024] Fast Transform Kernel Selection Based on Frequency Matching and Probability Model for AV1 CODE
• [PCS 2024] Nonlinear Transform Coding for VVC Intra Coding CODE
• [DCC 2024] Decoder-side Secondary Transform Derivation for Video Coding beyond AVS3 CODE
• [DCC 2024] Construction of Fast Data-driven Transforms for Image Compression via Multipath Coordinate Descent on Orthogonal Matrix Manifold CODE
• [ICASSP 2024] Adaptive Secondary Transform Sets for Video Coding Beyond AV1 CODE
• [ICCCAS 2024] A Fast Transform Algorithm for VVC Intra Coding CODE
• [ACM TOM 2024] Graph Based Cross-Channel Transform for Color Image Compression CODE
• [TCSVT 2024] Revisiting All-Zero Block Detection for Versatile Video Coding CODE
• [TCSVT 2023] Learning-Based Early Transform Skip Mode Decision for VVC Screen Content Coding CODE
• [ISCAS 2023] High-Throughput Design for a Multi-Size DCT-II Targeting the AV1 Encoder CODE
• [ISCAS 2022] A High-Throughput Design for the H.266/VVC Low-Frequency Non-Separable Transform CODE
• [IEEE Journal of Selected Topics in Signal Processing 2021] Nonlinear Transform Coding CODE
• [ICIP 2021] Machine-Learning Based Secondary Transform for Improved Image Compression in JPEG2000 CODE
• [DCC 2021] Graph Based Transforms based on Graph Neural Networks for Predictive Transform Coding CODE
• [PCS 2021] Combined neural network-based intra prediction and transform selection CODE
• [TCSVT 2021] Transform Coding in the VVC Standard CODE
• [ICIP 2020] Augmenting JPEG2000 With Wavelet Coefficient Prediction CODE
• [VCIP 2020] Deep Learning-Based Nonlinear Transform for HEVC Intra Coding CODE
• [PCS 2019] Low frequency non-separable transform CODE

Quantization

• [TBC 2023] Complexity-Efficient Dependent Quantization for Versatile Video Coding CODE

• [ICIP 2022] Learning Frequency-Specific Quantization Scaling in VVC for Standard-Compliant Task-Driven Image Coding CODE

• [DCC 2021] JQF: Optimal JPEG Quantization Table Fusion by Simulated Annealing on Texture Images and Predicting Textures CODE

• [VCIP 2020] Towards Quantized DCT Coefficients Restoration for Compressed Images CODE

• [DCC 2020] Deep Learning-based Image Compression with Trellis Coded Quantization CODE

• [ECCV 2020] Task-Aware Quantization Network for JPEG Image Compression CODE

• [DCC 2019] Hard-Decision Quantization Algorithm Based on Deep Learning in Intra Video Coding CODE

Entropy Coding

• [IET Image Processing 2024] Dynamic estimator selection for double-bit-range estimation in VVC CABAC entropy coding CODE
• [TCSVT 2021] Quantization and Entropy Coding in the Versatile Video Coding (VVC) Standard CODE
• [TCSVT 2019] Convolutional Neural Network-Based Arithmetic Coding for HEVC Intra-Predicted Residues CODE

In-Loop Filtering

• [DCC 2024] Residual Block Fusion in Low Complexity Neural Network-Based In-loop Filtering for Video Compression CODE

• [ICIP 2024] IN-Loop Filter for Object Mask Coding in Versatile Video Coding CODE

• [PCS 2024] Swin Transformer-Based In-Loop Filter for VVC Intra Coding CODE

• [ACM TOM 2024] A Reconfigurable Framework for Neural Network Based Video In-Loop Filtering CODE

• [VCIP 2024] PFT-ILF: In-loop Filter with Partition Feature Transform for Versatile Video Coding CODE

• [TIP 2024] Combining Progressive Rethinking and Collaborative Learning: A Deep Framework for In-Loop Filtering CODE

• [VCIP 2024] Fast Adaptive Loop Filter Algorithm Based on the Optimization of Class Merging CODE

• [VCIP 2024] In-Loop Filtering via Trained Look-Up Tables CODE

• [ICIP 2024] IN-Loop Filter for Object Mask Coding in Versatile Video Coding CODE

• [ICIP 2024] NN-Based In-Loop Filtering With Inputs Transformed CODE

• [TCSVT 2024] Neural Network Based Multi-Level In-Loop Filtering for Versatile Video Coding CODE

• [TCSVT 2024] Screen Content-Aware Video Coding through Non-Local Model embedded with Intra-Inter In-Loop Filtering CODE

• [TCSVT 2024] Area-Efficient Processing Elements-Based Adaptive Loop Filter Architecture With Optimized Memory for VVC CODE

• [Pattern Recognition 2024] DWT-SALF: Subband Adaptive Neural Network Based In-Loop Filter for VVC Using Cyclic DWT CODE

• [Pattern Recognition 2024] Progressive Learning Based on QP Distance for Enhancing HOP In-Loop Filter CODE

• [ISCAS 2023] Towards Next Generation Video Coding: from Neural Network Based Predictive Coding to In-Loop Filtering CODE

• [ICME 2023] Meta-ILF: In-Loop Filter with Customized Weights For VVC Intra Coding CODE

• [ICML 2023] Low Complexity Neural Network-Based In-loop Filtering with Decomposed Split Luma-Chroma Model for Video Compression CODE

• [TCSVT 2023] Lightweight Multiattention Recursive Residual CNN-Based In-Loop Filter Driven by Neuron Diversity CODE

• [TCSVT 2023] Deep In-Loop Filtering via Multi-Domain Correlation Learning and Partition Constraint for Multiview Video Coding CODE

• [ICIG 2023] Content-Adaptive Block Clustering for Improving VVC Adaptive Loop Filtering CODE

• [ACM TOM 2023] iDAM: Iteratively Trained Deep In-loop Filter with Adaptive Model Selection CODE

• [ICIP 2023] Lightweight CNN-Based in-Loop Filter for VVC Intra Coding CODE

• [TMM 2023] Joint Rate-Distortion Optimization for Video Coding and Learning-Based In-Loop Filtering CODE

• [DCC 2023] A Low Complexity Convolutional Neural Network with Fused CP Decomposition for In-Loop Filtering in Video Coding CODE

• [IFTC 2023] Temporal Dependency-Oriented Deep In-Loop Filter for VVC CODE

• [TMM 2022] Textural and Directional Information Based Offset In-Loop Filtering in AVS3 CODE

• [ICASSP 2022] Low-Complexity Multi-Model CNN in-Loop Filter for AVS3 CODE

• [CVPR 2022] Perceptual In-Loop Filter for Image and Video Compression CODE

• [ACM TOM 2022] NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding CODE

• [ICME 2022] Neural Network Based in-Loop Filter with Constrained Memory CODE

• [TIP 2022] Deformable Wiener Filter for Future Video Coding CODE

• [TIP 2022] QA-filter: A QP-adaptive convolutional neural network filter for video coding CODE

• [TCE 2022] Adaptive Loop Filter Hardware Design for 4K ASIC VVC Decoders CODE

• [VCIP 2022] Multi-stage Locally and Long-range Correlated Feature Fusion for Learned In-loop Filter in VVC CODE

• [ICIP 2022] Switchable CNN-Based Same-Resolution and Super-Resolution In-Loop Restoration for Next Generation Video Codecs CODE

• [ICIP 2022] Non-Separable Filtering with Side-Information and Contextually-Designed Filters for Next Generation Video Codecs CODE

• [ICIP 2022] Adaptive Loop Filter with a CNN-Based Classification CODE

• [PCS 2022] Efficient HW Design of Adaptive Loop Filter for 4k ASIC VVC Encoder CODE

• [PCS 2022] Multi-Stage Spatial and Frequency Feature Fusion using Transformer in CNN-Based In-Loop Filter for VVC CODE

• [PCS 2022] Optimize neural network based in-loop filters through iterative training CODE

• [PCS 2022] Generalized deblocking filter for AVM CODE

• [ISCAS 2022] An Attention Based CNN with Temporal Hierarchical Deployment for AVS3 Inter In-loop Filtering CODE

• [ISCAS 2022] Complexity Reduction of Learned In-Loop Filtering in Video Coding CODE

• [ISCAS 2022] A QP-adaptive Mechanism for CNN-based Filter in Video Coding CODE

• [ISCAS 2022] Joint Luma and Chroma Multi-Scale CNN In-loop Filter for Versatile Video Coding CODE

• [DCC 2022] Joint Rate Distortion Optimization with CNN-based In-Loop Filter For Hybrid Video Coding CODE

• [DCC 2022] Parametric Non-local In-loop Filter for Future Video Coding CODE

• [DCC 2022] An Improved Multi-reference Frame Loop Filter Algorithm Based on Transformer for VVC CODE

• [JVCIR 2022] PTR-CNN for in-loop filtering in video coding CODE

• [TCSVT 2022] One-for-all: An efficient variable convolution neural network for in-loop filter of VVC CODE

• [SPIC 2021] Deep learning based HEVC in-loop filter and noise reduction CODE

• [SPIC 2021] A progressive CNN in-loop filtering approach for inter frame coding CODE

• [TIP 2021] Adaptive deep reinforcement learning-based in-loop filter for VVC CODE

• [TCSVT 2021] VVC In-Loop Filters CODE

• [CVPR 2021] Deep Learning Based Spatial-Temporal In-Loop Filtering for Versatile Video Coding CODE

• [DCC 2021] Multi-Density Convolutional Neural Network for In-Loop Filter in Video Coding CODE

• [DCC 2021] An Efficient QP Variable Convolutional Neural Network Based In-loop Filter for Intra Coding CODE

• [DCC 2021] 3D-CVQE: An Effective 3D-CNN Quality Enhancement for Compressed Video Using Limited Coding Information CODE

• [DCC 2021] Spatial-Temporal Fusion Convolutional Neural Network for Compressed Video Enhancement in HEVC CODE

• [DCC 2021] Video Enhancement Network Based on Max-Pooling and Hierarchical Feature Fusion CODE

• [PCS 2021] Bonnineau C, Hamidouche W, Travers J F, et al. Multitask Learning for VVC Quality Enhancement and Super-Resolution CODE

• [PCS 2021] Bordes P, Galpin F, Dumas T, et al. Revisiting the Sample Adaptive Offset post-filter of VVC with Neural-Networks CODE

• [PCS 2021] Nasiri F, Hamidouche W, Morin L, et al. Model Selection CNN-based VVC QualityEnhancement CODE

• [PCS 2021] Convolutional neural network-based post-filtering for compressed YUV420 images and video CODE

• [VCIP 2021] Distortion-based Neural Network for Compression Artifacts Reduction in VVC CODE

• [ICIP 2021] Convolutional Neural Network Based In-Loop Filter For VVC Intra Coding CODE

• [DCC 2021] Flow-Guided Temporal-Spatial Network for HEVC Compressed Video Quality Enhancement CODE

• [DCC 2021] Densely Connected Unit based Loop Filter for Short Video Coding CODE

• [ICIP 2021] Enhancing quality for VVC compressed videos by jointly exploiting spatial details and temporal structure CODE

• [ICIP 2021] Partition tree guided progressive rethinking network for in-loop filtering of HEVC CODE

• [ICIP 2021] Deep Enhancement for 3D HDR Brain Image Compression CODE

• [ICIP 2021] Esmaeilzehi A, Ahmad M O, Swamy M N S. Development Of New Fractal And Non-Fractal Deep Residual Networks For Deblocking Of Jpeg Decompressed Images CODE

• [ICIP 2021] Xia J, Wen J. Asymmetric Convolutional Residual Network for AV1 Intra in-Loop Filtering CODE

• [ICIP 2021] Li B, Liang J, Wang Y. Compression Artifact Removal with Stacked Multi-Context Channel-Wise Attention Network CODE

• [ICIP 2021] Kim T, Lee H, Son H, et al. SF-CNN: A Fast Compression Artifacts Removal via Spatial-To-Frequency Convolutional Neural Networks CODE

• [ICIP 2021] Esmaeilzehi A, Ahmad M O, Swamy M N S. Deep Jpeg Image Deblocking Using Residual Maxout Units CODE

• [ICME 2020] Multi-Gradient Convolutional Neural Network Based In-Loop Filter For Vvc CODE

• [TCSVT 2020] A switchable deep learning approach for in-loop filtering in video coding CODE

• [VCIP 2020] Ma H, Liu D, Wu F. Improving Compression Artifact Reduction via End-to-End Learning of Side Information CODE

• [VCIP 2020] A Mixed Appearance-based and Coding Distortion-based CNN Fusion Approach for In-loop Filtering in Video Coding CODE

• [VCIP 2020] Nasiri F, Hamidouche W, Morin L, et al. Prediction-Aware Quality Enhancement of VVC Using CNN CODE

• [VCIP 2020] Yue J, Gao Y, Li S, et al. A Mixed Appearance-based and Coding Distortion-based CNN Fusion Approach for In-loop Filtering in Video Coding CODE

• [VCIP 2019] Yu Y, Yang X, Chen J, et al. Deep Learning Based In-Loop Filter for Video Coding CODE

• [ICIP 2020] Esmaeilzehi A, Ahmad M O, Swamy M N S. Development Of New Fractal And Non-Fractal Deep Residual Networks For Deblocking Of Jpeg Decompressed Images CODE

• [ICIP 2020] Li B, Liang J, Wang Y. Compression Artifact Removal with Stacked Multi-Context Channel-Wise Attention Network CODE

• [ICIP 2020] Kim T, Lee H, Son H, et al. SF-CNN: A Fast Compression Artifacts Removal via Spatial-To-Frequency Convolutional Neural Networks CODE

• [ICIP 2020] Esmaeilzehi A, Ahmad M O, Swamy M N S. Deep Jpeg Image Deblocking Using Residual Maxout Units CODE

• [ICIP 2019] Esmaeilzehi A, Ahmad M O, Swamy M N S. Development Of New Fractal And Non-Fractal Deep Residual Networks For Deblocking Of Jpeg Decompressed Images CODE

• [ICIP 2019] Li B, Liang J, Wang Y. Compression Artifact Removal with Stacked Multi-Context Channel-Wise Attention Network CODE

• [ICIP 2019] Kim T, Lee H, Son H, et al. SF-CNN: A Fast Compression Artifacts Removal via Spatial-To-Frequency Convolutional Neural Networks CODE

• [ICIP 2019] Esmaeilzehi A, Ahmad M O, Swamy M N S. Deep Jpeg Image Deblocking Using Residual Maxout Units CODE

• [ICIP 2019] Gao S, Xiong Z. Deep Enhancement for 3D HDR Brain Image Compression CODE

• [ICIP 2019] Meng X, Deng X, Zhu S, et al. Enhancing quality for VVC compressed videos by jointly exploiting spatial details and temporal structure CODE

• [ICIP 2019] Partition tree guided progressive rethinking network for in-loop filtering of HEVC CODE

• [ICIP 2019] [Esmaeilzehi A, Ahmad M O, Swamy M N S. Deep Jpeg Image Deblocking Using Residual Maxout Units](https://doi.org/10.1109/ICIP.2019.880337

Rate-Distortion Optimization

• [TIP 2023] Toward the Achievable Rate-Distortion Bound of VVC Intra Coding: A Beam Search-Based Joint Optimization Scheme CODE
• [TMM 2023] Joint Rate-Distortion Optimization for Video Coding and Learning-Based In-Loop Filtering CODE
• [TBC 2023] A Quality-of-Experience-Aware Framework for Versatile Video Coding-Based Video Transmission CODE
• [TIP 2020] Rate distortion optimization: A joint framework and algorithms for random access hierarchical video coding CODE

Rate Control

• [JVCIP 2024] Exploring the rate-distortion-complexity optimization in neural image compression☆ CODE

• [ISCAS 2024] Rate Control for Slimmable Video Codec using Multilayer Perceptron CODE

• [TVT 2024] An Efficient Neural Network Based Rate Control for Intra-frame in Versatile Video Coding CODE

• [PCS 2024] Multi-Agent Reinforcement Learning based Bit Allocation for Gaming Video Coding CODE

• [TIP 2024] λ-Domain Rate Control via Wavelet-Based Residual Neural Network for VVC HDR Intra Coding CODE

• [TCSVT 2024] Recent Advances in Rate Control: From Optimization to Implementation and Beyond CODE

• [TCSVT 2024] Content-Adaptive Rate Control Method for User-Generated Content Videos CODE

• [TMM 2024] Content-Adaptive Rate-Distortion Modeling for Frame-Level Rate Control in Versatile Video Coding CODE

• [TBC 2024] Spatial Coupling Strategy and Improved BFGS-Based Advanced Rate Control for VVC CODE

• [VCIP 2023] Reinforcement Learning-based Frame-level Bit Allocation for VVC CODE

• [MMSP 2023] Rate control for VVC intra coding with simplified cubic rate-distortion model CODE

• [TIP 2023] Joint decision tree and visual feature rate control optimization for VVC UHD coding CODE

• [TCSVT 2023] λ -domain VVC rate control based on Nash equilibrium CODE

• [TCSVT 2023] Neural Network Based Rate Control for Versatile Video Coding CODE

• [TCSVT 2023] Efficient Rate Control in Versatile Video Coding With Adaptive Spatial–Temporal Bit Allocation and Parameter Updating CODE

• [TCSVT 2023] A CTU-Level Screen Content Rate Control for Low-Delay Versatile Video Coding CODE

• [TBC 2023] Precise Encoding Complexity Control for Versatile Video Coding CODE

• [VCIP 2023] Rate Control Optimization for Joint Geometry and Attribute Coding of LiDAR Point Clouds CODE

• [TIP 2023] SUR-Driven Video Coding Rate Control for Jointly Optimizing Perceptual Quality and Buffer Control CODE

• [Information Sciences 2023] Rate distortion optimization with adaptive content modeling for random-access versatile video coding CODE

• [MMSP 2022] Reinforcement Learning based Low Delay Rate Control for HEVC Region of Interest Coding CODE

• [TBC 2022] Optimum Quality Control Algorithm for Versatile Video Coding CODE

• [TIP 2022] Joint Decision Tree and Visual Feature Rate Control Optimization for VVC UHD Coding CODE

• [TBC 2022] Perceptual quality consistency oriented CTU level rate control for HEVC intra coding CODE

• [TIP 2022] Learning-Based Rate Control for Video-Based Point Cloud Compression CODE

• [ArXiv 2022] MuZero with Self-competition for Rate Control in VP9 Video Compression CODE

• [ArXiv 2022] λ-domain VVC Rate Control Based on Game Theory CODE

• [MIPR 2022] Machine-Learning Based High Efficiency Rate Control for AV1 CODE

• [Multimedia Tools and Applications 2021] New bufferless rate control for high efficiency video coding CODE

• [ACM MM 2021] Game theory-driven rate control for 360-degree video coding CODE

• [TIP 2021] Multi-Objective Optimization of Quality in VVC Rate Control for Low-Delay Video Coding CODE

• [TCC 2021] Hybrid Distortion-Based Rate-Distortion Optimization and Rate Control for H.265/HEVC CODE

• [DCC 2021] A Viewport-Adaptive Rate Control Approach for Omnidirectional Video Coding CODE

• [DCC 2021] A Dual-Critic Reinforcement Learning Framework for Frame-Level Bit Allocation in HEVC/H.265 CODE(The code page may be out of date, you can access the code by mail me xiongweixiao@foxmail.com)

• [ICME 2021] Machine learning-based rate distortion modeling for VVC/H.266 intra-frame CODE

• [TCSVT 2021] Rate control for predictive transform screen content video coding based on RANSAC CODE

• [TCSVT 2021] High Efficiency Rate Control for Versatile Video Coding Based on Composite Cauchy Distribution CODE

• [TBC 2021] A Bit Allocation Method Based on Inter-View Dependency and Spatio-Temporal Correlation for Multi-View Texture Video Coding CODE

• [TII 2021] Consistent Quality Oriented Rate Control in HEVC Via Balancing Intra and Inter Frame Coding CODE

• [VCIP 2020] Neural Rate Control for Video Encoding using Imitation Learning CODE

• [TCSVT 2020] Global rate-distortion optimization-based rate control for HEVC HDR coding CODE

• [TBC 2020] A novel rate control scheme for video coding in HEVC-SCC CODE

• [TIP 2020] Rate Control for Video-Based Point Cloud Compression CODE

• [TMM 2020] Rate Control Method Based on Deep Reinforcement Learning for Dynamic Video Sequences in HEVC CODE

• [DCC 2020] A Rate Control Scheme for HEVC Intra Coding Using Convolution Neural Network (CNN) CODE

• [ICIP 2020] Yoco: Light-Weight Rate Control Model Learning CODE

• [ICASSP 2020] Intra Frame Rate Control for Versatile Video Coding with Quadratic Rate-Distortion Modelling CODE

• [Multimedia Tools and Applications 2020] FJND-based fuzzy rate control of scalable video for streaming applications CODE

• [TBC 2019] Data-driven rate control for rate-distortion optimization in HEVC based on simplified effective initial QP learning CODE

• [TIP 2019] Optimize x265 rate control: An exploration of lookahead in frame bit allocation and slice type decision CODE

• [PCS 2019] Predicting Rate Control Target Through A Learning Based Content Adaptive Model CODE

Complexity Control

• [TBC 2024] An Efficient and Flexible Complexity Control Method for Versatile Video Coding CODE

• [TBC 2024] Fast Mode and CU Splitting Decision for Intra Prediction in VVC SCC CODE

• [TBC 2024] Learning-Based Efficient Quantizer Selection for Fast HEVC Encoder CODE

• [JVCIP 2024] Exploring the rate-distortion-complexity optimization in neural image compression☆ CODE

• [VCIP 2024] Fast Machine Learning Aided Intra Mode Decision for Real-Time VVC Intra Coding CODE

• [SPL 2024] VVC Intra Coding Complexity Optimization Based on Early Skipping of the Secondary Transform CODE

• [TBC 2023] Precise Encoding Complexity Control for Versatile Video Coding CODE

• [TBC 2023] Complexity-Efficient Dependent Quantization for Versatile Video Coding CODE

• [TCSVT 2023] Learning-Based Early Transform Skip Mode Decision for VVC Screen Content Coding CODE

• [PCS 2022] Performance-Complexity Analysis of Adaptive Loop Filter with a CNN-based Classification CODE

• [TCSVT 2022] Configurable Fast Block Partitioning for VVC Intra Coding Using Light Gradient Boosting Machine CODE

• [ISCAS 2022] Intra encoding complexity control with a time-cost model for versatile video coding CODE

• [ISCAS 2022] Fast Affine Motion Estimation for VVC using Machine-Learning-Based Early Search Termination CODE

• [ICIP 2022] CNN-Based Fast CU Partitioning Algorithm for VVC Intra Coding CODE

• [IEEE Open Journal of Circuits and Systems 2021] AV1 and VVC Video Codecs: Overview on Complexity Reduction and Hardware Design CODE

• [SPIC 2021] SVM based approach for complexity control of HEVC intra coding CODE

• [TBC 2021] Early intra CU size decision for versatile video coding based on a tunable decision model CODE

• [ISCAS 2021] SVM based fast cu partitioning algorithm for VVC intra coding CODE

• [TCSVT 2021] VVC Complexity and Software Implementation Analysis CODE

• [TCSVT 2021] Fast DST-VII/DCT-VIII with dual implementation support for versatile video coding CODE

• [TCSVT 2021] Modeling acceleration properties for flexible INTRA HEVC complexity control CODE

• [ICIP 2020] Complexity Analysis Of Next-Generation VVC Encoding And Decoding CODE

• [TIP 2020] Accelerate CTU Partition to Real Time for HEVC Encoding With Complexity Control CODE

• [TCSVT 2020] Low-complexity CTU partition structure decision and fast intra mode decision for versatile video coding CODE

• [TIP 2020] Online learning-based multi-stage complexity control for live video coding CODE

• [MMSP 2019] Complexity reduction opportunities in the future VVC intra encoder CODE

• [TIP 2019] Complexity control in the HEVC intracoding for industrial video applications CODE

• [TIP 2019] Tunable VVC frame partitioning based on lightweight machine learning CODE

• [ICME 2019] Two-stage fast multiple transform selection algorithm for VVC intra coding CODE

• [ICIP 2019] Fast partitioning decision strategies for the upcoming versatile video coding (VVC) standard CODE

• [TMM 2019] CTU-level complexity control for high efficiency video coding CODE

• [JRTIP 2017] Complexity control of HEVC encoders targeting real-time constraints CODE

• [TCSVT 2016] Pareto-based method for high efficiency video coding with limited encoding time CODE

• [TCSVT 2016] Subjective-driven complexity control approach for HEVC CODE

• [ACCESS 2016] Hierarchical complexity control of HEVC for live video encoding CODE

• [TMM 2016] Complexity control based on a fast coding unit decision method in the HEVC video coding standard CODE

Datasets

Tools

• [TCSVT 2021] Overview of the Screen Content Support in VVC: Applications, Coding Tools, and Performance CODE

Survey

• [Proceedings of the IEEE 2021] Developments in International Video Coding Standardization After AVC, With an Overview of Versatile Video Coding (VVC) CODE

Learned Image Compression

Neural network-based image codecs.

2014

• [CoRR 2014] Auto-encoders: reconstruction versus compression
  • Key Idea: Theoretical exploration of autoencoders for compression by minimizing code length.
  • Code: Not Provided

---

2016

• [CoRR 2016] Towards Conceptual Compression

  • Key Idea: Recurrent VAE for latent space learning in conceptual compression.
  • Code: Not Provided

• [ICLR 2016] Autoencoding beyond pixels using a learned similarity metric

  • Key Idea: KL divergence-based VAE for image generation.
  • Code: Not Provided

• [ICLR 2016] Variable Rate Image Compression with Recurrent Neural Networks

  • Key Idea: Adaptive RNN-based compression on Kodak dataset.
  • Code: https://github.com/mr-mikmik/VRIC-RNN

---

2017

• [CVPR 2017] Full Resolution Image Compression with Recurrent Neural Networks

  • Key Idea: Combines RNN and entropy encoding for high MS-SSIM.
  • Code: https://github.com/1zb/pytorch-image-comp-rnn

• [DCC 2017] Semantic Perceptual Image Compression using Deep Convolution Networks

  • Key Idea: Semantic-aware compression with CNN.
  • Code: https://github.com/iamaaditya/image-compression-cnn

---

2018

• [CVPR 2018] Variational Autoencoder for Low Bit-rate Image Compression

  • Key Idea: Nonlinear encoder + uniform quantizer for low-bitrate compression.
  • Code: Not Provided | Bibtex: See [Ref 47]

• [NIPS 2018] Joint Autoregressive and Hierarchical Priors for Learned Image Compression

  • Key Idea: Hybrid autoregressive and hierarchical priors for entropy modeling.
  • Code

• [CVPR 2018] Deep Image Compression via End-to-End Learning

  • Key Idea: End-to-end CNN framework with CLIC 2018 dataset.
  • Code: Not Provided

• [NIPS 2018] Non-local Recurrent Network for Image Restoration

  • Key Idea: Integrates non-local operations with RNN for artifact reduction.
  • Code

---

2019

• [ICCV 2019] Generative adversarial networks for extreme learned image compression
  • Key Idea: GAN-based compression targeting <0.1 bpp with selective generative modules.
  • Code: GitHub

---

2020

• [WACV 2020] CompressNet: Generative Compression at Extremely Low Bitrates

  • Key Idea: Combines MSE, adversarial, and layer-wise losses for ultra-low bpp.
  • Code

• [ICIP 2020] Channel-Wise Autoregressive Entropy Models for Learned Image Compression

  • Key Idea: Channel-wise autoregressive entropy model for ultra-low bpp (0.01921).
  • Code

---

2021

• [TIP 2021] End-to-End Learnt Image Compression via Non-Local Attention Optimization

  • Key Idea: Non-local attention modules (NLAM) for global context modeling.
  • Code: GitHub | Bibtex: See [Ref 48]

• [TCSVT 2021] Learned Block-based Hybrid Image Compression

  • Key Idea: Block-based partitioning to address OOM issues in high-resolution images.
  • Code: Not Provided

---

2022

• [CVPR 2022 Oral] ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Context
  • Key Idea: Advanced entropy modeling with channel-wise and spatial context grouping.
  • Code | Bibtex: See [Ref ELIC]

2023

2024

• [CVPR 2024] ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Context
  • Key Idea: Advanced entropy modeling with channel-wise context.
  • Code
• [ICLR 2024] Towards Image Compression with Perfect Realism at Ultra-Low Bitrates
  • Key Idea: Using Stable Diffusion v2.1 (Rombach et al., CVPR 2022) as latent diffusion model and hence refer to our work as PerCo (SD)
  • Code

---

Rate & Complexity Control

Rate & Complexity Control codecs.

---

Low-Latency & Real-Time Coding

Solutions for live streaming/teleconferencing.

• [ACM MM 2023] LVC: Latency-Aware Neural Video Compression with Frame-Level Parallelism, Chen et al.
  • Key Idea: Frame-level parallelism to reduce encoding latency.
  • Code | Bibtex

---

Datasets & Metrics

Video Coding Datasets

Name	Content	Resolution	Annotations	Link
UVG	16 diverse video sequences	Up to 4K	PSNR, MS-SSIM	Download
CLIC 2023	User-generated videos	1080p/4K	Subjective scores	Website

Metrics

Metric	Type	Description	Code
LPIPS-V	FR	Perceptual similarity for videos	GitHub
VMAF	FR	Netflix's video quality metric	GitHub

---

Tools & Libraries

Name	Description	Link
CompressAI	PyTorch library for learned image/video compression	GitHub
NNVCC	Toolkit for neural video coding research	GitHub

---

持续更新中...欢迎提交PR补充！ 🎉