Intel® Neural Compressor v1.8.1 Release

Features

• Knowledge distillation
  • Supported knowledge distillation on TensorFlow
• Pruning
  • Support Multi-node training on TensorFlow
• Acceleration library
  • Supported Hugging Face minilm_l6_h384_uncased_sst2, bert_base_cased_mrpc, and bert_base_nli_mean_tokens_stsb models

Validated Configurations

• Python 3.6 & 3.7 & 3.8 & 3.9
• Centos 8.3 & Ubuntu 18.04
• TensorFlow 2.6.2 & 2.7
• Intel TensorFlow 2.4.0, 2.5.0 and 1.15.0 UP3
• PyTorch 1.8.0+cpu, 1.9.0+cpu, IPEX 1.8.0
• MxNet 1.6.0, 1.7.0, 1.8.0
• ONNX Runtime 1.6.0, 1.7.0, 1.8.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/neural-compressor.git	$ git clone https://github.com/intel/neural-compressor.git
Binary	Pip	https://pypi.org/project/neural-compressor	$ pip install neural-compressor
Binary	Conda	https://anaconda.org/intel/neural-compressor	$ conda install neural-compressor -c conda-forge -c intel

Contact:

Please feel free to contact inc.maintainers@intel.com, if you get any questions.

Intel® Neural Compressor v1.8 Release

Features

• Knowledge distillation
  • Implemented the algorithms of paper “Pruning Once For All” accepted by NeurIPS 2021 ENLSP workshop
  • Supported optimization pipelines (knowledge distillation & quantization-aware training) on PyTorch
• Quantization
  • Added the support of ONNX RT 1.7
  • Added the support of TensorFlow 2.6.2 and 2.7
  • Added the support of PyTorch 1.10
• Pruning
  • Supported magnitude pruning on TensorFlow
• Acceleration library
  • Supported Hugging Face top 10 downloaded NLP models

Productivity

• Added performance profiling feature to INC UI service.
• Improved ease-of-use user interface for quantization with few clicks

Ecosystem

• Added notebook of using HuggingFace optimization library (Optimum) to Transformers
• Enabled top 20 downloaded Hugging Face NLP models with Optimum
• Upstreamed more INC quantized models to ONNX Model Zoo

Validated Configurations

• Python 3.6 & 3.7 & 3.8 & 3.9
• Centos 8.3 & Ubuntu 18.04
• TensorFlow 2.6.2 & 2.7
• Intel TensorFlow 2.4.0, 2.5.0 and 1.15.0 UP3
• PyTorch 1.8.0+cpu, 1.9.0+cpu, IPEX 1.8.0
• MxNet 1.6.0, 1.7.0, 1.8.0
• ONNX Runtime 1.6.0, 1.7.0, 1.8.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/neural-compressor.git	$ git clone https://github.com/intel/neural-compressor.git
Binary	Pip	https://pypi.org/project/neural-compressor	$ pip install neural-compressor
Binary	Conda	https://anaconda.org/intel/neural-compressor	$ conda install neural-compressor -c conda-forge -c intel

Contact:

Please feel free to contact inc.maintainers@intel.com, if you get any questions.

Intel® Neural Compressor v1.7.1 Release

Intel® Neural Compressor(formerly known as Intel® Low Precision Optimization Tool) v1.7 release is featured by:

Features

• Acceleration library
  • Support unified buffer memory allocation policy

Ecosystem

• Upstreamed INC quantized models (alexnet/caffenet/googlenet/squeezenet) to ONNX Model Zoo

Documentation

• Performance and accuracy data update

Validated Configurations

• Python 3.6 & 3.7 & 3.8 & 3.9
• Centos 8.3 & Ubuntu 18.04
• TensorFlow 2.6.0
• Intel TensorFlow 2.4.0, 2.5.0 and 1.15.0 UP3
• PyTorch 1.8.0+cpu, 1.9.0+cpu, IPEX 1.8.0
• MxNet 1.6.0, 1.7.0, 1.8.0
• ONNX Runtime 1.6.0, 1.7.0, 1.8.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/neural-compressor.git	$ git clone https://github.com/intel/neural-compressor.git
Binary	Pip	https://pypi.org/project/neural-compressor	$ pip install neural-compressor
Binary	Conda	https://anaconda.org/intel/neural-compressor	$ conda install neural-compressor -c conda-forge -c intel

Contact:

Please feel free to contact INC Maintainers, if you get any questions.

Intel® Neural Compressor v1.7 Release

Intel® Neural Compressor(formerly known as Intel® Low Precision Optimization Tool) v1.7 release is featured by:

Features

• Quantization
  • Improved quantization accuracy in SSD-Reset34 and MobileNet v3 on TensorFlow
• Pruning
  • Supported magnitude pruning on TensorFlow
• Knowledge distillation
  • Supported knowledge distillation on PyTorch
• Multi-node support
  • Supported multi-node pruning with distributed dataloader on PyTorch
  • Supported multi-node inference for benchmark on PyTorch
• Acceleration library
  • Added a domain-specific acceleration library for NLP models

Productivity

• Supported the configuration-free (pure Python) quantization
• Improved ease-of-use user interface for quantization with few clicks

Ecosystem

• Integrated into HuggingFace optimization library (Optimum)
• Upstreamed INC quantized models (RN50, VGG16) to ONNX Model Zoo

Documentation

• Add tutorial and examples for knowledge distillation
• Add tutorial and examples for multi-node training
• Add tutorial and examples for acceleration library

Validated Configurations

• Python 3.6 & 3.7 & 3.8 & 3.9
• Centos 8.3 & Ubuntu 18.04
• TensorFlow 2.6.0
• Intel TensorFlow 2.4.0, 2.5.0 and 1.15.0 UP3
• PyTorch 1.8.0+cpu, 1.9.0+cpu, IPEX 1.8.0
• MxNet 1.6.0, 1.7.0, 1.8.0
• ONNX Runtime 1.6.0, 1.7.0, 1.8.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/neural-compressor.git	$ git clone https://github.com/intel/neural-compressor.git
Binary	Pip	https://pypi.org/project/neural-compressor	$ pip install neural-compressor
Binary	Conda	https://anaconda.org/intel/neural-compressor	$ conda install neural-compressor -c conda-forge -c intel

Contact:

Please feel free to contact lpot.maintainers@intel.com, if you get any questions.

Intel® Low Precision Optimization Tool v1.6 Release

Intel® Low Precision Optimization Tool v1.6 release is featured by:

Pruning:

• Support pruning and post-training quantization pipeline on PyTorch
• Support pruning during quantization-aware training on PyTorch

Quantization:

• Support post-training quantization on TensorFlow 2.6.0, PyTorch 1.9.0, IPEX 1.8.0, and MXNet 1.8.0
• Support quantization-aware training on TensorFlow 2.x (Keras API)

User Experience:

• Improve quantization productivity with new UI
• Support quantized model recovery from tuning history

New Models:

• Support ResNet50 on ONNX model zoo

Documentation:

• Add pruned models
• Add quantized MLPerf models

Validated Configurations:

• Python 3.6 & 3.7 & 3.8 & 3.9
• Centos 8.3 & Ubuntu 18.04
• TensorFlow 2.6.0
• Intel TensorFlow 2.4.0, 2.5.0 and 1.15.0 UP3
• PyTorch 1.8.0+cpu, 1.9.0+cpu, IPEX 1.8.0
• MxNet 1.6.0, 1.7.0, 1.8.0
• ONNX Runtime 1.6.0, 1.7.0, 1.8.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/lpot.git	$ git clone https://github.com/intel/lpot.git
Binary	Pip	https://pypi.org/project/lpot	$ pip install lpot
Binary	Conda	https://anaconda.org/intel/lpot	$ conda install lpot -c conda-forge -c intel

Contact:

Please feel free to contact lpot.maintainers@intel.com, if you get any questions.

Intel® Low Precision Optimization Tool v1.5.1 Release

Intel® Low Precision Optimization Tool v1.5.1 release is featured by:

• Gradient-sensitivity pruning for CNN model
• Static quantization support for ONNX NLP model
• Dynamic seq length support in NLP dataloader
• Enrich quantization statistics

Validated Configurations:

• Python 3.6 & 3.7 & 3.8 & 3.9
• Centos 8.3 & Ubuntu 18.04
• Intel TensorFlow 1.15.2, 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0 and 1.15.0 UP1 & UP2 & UP3
• PyTorch 1.5.0+cpu, 1.6.0+cpu, 1.8.0+cpu, ipex
• MxNet 1.6.0, 1.7.0
• ONNX Runtime 1.6.0, 1.7.0, 1.8.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/lpot.git	$ git clone https://github.com/intel/lpot.git
Binary	Pip	https://pypi.org/project/lpot	$ pip install lpot
Binary	Conda	https://anaconda.org/intel/lpot	$ conda install lpot -c conda-forge -c intel

Contact:

Please feel free to contact lpot.maintainers@intel.com, if you get any questions.

Intel® Low Precision Optimization Tool v1.5 Release

Intel® Low Precision Optimization Tool v1.5 release is featured by:

• Add pattern-lock sparsity algorithm for NLP fine-tuning tasks
  • Up to 70% unstructured sparsity and 50% structured sparsity with <2% accuracy loss on 5 Bert finetuning tasks
• Add NLP head pruning algorithm for HuggingFace models
  • Performance speedup up to 3.0X within 1.5% accuracy loss on HuggingFace BERT SST-2
• Support model optimization pipeline
• Integrate SigOPT with multi-metrics optimization
  • Complementary as basic strategy to speed up the tuning
• Support TensorFlow 2.5, PyTorch 1.8, and ONNX Runtime 1.8

Validated Configurations:

• Python 3.6 & 3.7 & 3.8 & 3.9
• Centos 8.3 & Ubuntu 18.04
• Intel TensorFlow 1.15.2, 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0 and 1.15.0 UP1 & UP2 & UP3
• PyTorch 1.5.0+cpu, 1.6.0+cpu, 1.8.0+cpu, ipex
• MxNet 1.6.0, 1.7.0
• ONNX Runtime 1.6.0, 1.7.0, 1.8.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/lpot.git	$ git clone https://github.com/intel/lpot.git
Binary	Pip	https://pypi.org/project/lpot	$ pip install lpot
Binary	Conda	https://anaconda.org/intel/lpot	$ conda install lpot -c conda-forge -c intel

Contact:

Please feel free to contact lpot.maintainers@intel.com, if you get any questions.

Intel® Low Precision Optimization Tool v1.4.1 Release

Intel® Low Precision Optimization Tool v1.4.1 release is featured by:

1. Support TensorFlow 2.5.0
2. Support PyTorch 1.8.0
3. Support TensorFlow Object Detection YOLO-V3 model

Validated Configurations:

• Python 3.6 & 3.7 & 3.8
• Centos 7 & Ubuntu 18.04
• Intel TensorFlow 1.15.2, 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0 and 1.15.0 UP1 & UP2
• PyTorch 1.5.0+cpu, 1.6.0+cpu, ipex
• MxNet 1.7.0
• ONNX Runtime 1.6.0, 1.7.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/lpot.git	$ git clone https://github.com/intel/lpot.git
Binary	Pip	https://pypi.org/project/lpot	$ pip install lpot
Binary	Conda	https://anaconda.org/intel/lpot	$ conda install lpot -c conda-forge -c intel

Contact:

Please feel free to contact lpot.maintainers@intel.com, if you get any questions.

Intel® Low Precision Optimization Tool v1.4 Release

Intel® Low Precision Optimization Tool v1.4 release is featured by:

Quantization

1. PyTorch FX-based quantization support
2. TensorFlow & ONNX RT quantization enhancement

Pruning

1. Pruning/sparsity API refinement
2. Magnitude-based pruning on PyTorch

Model Zoo

1. INT8 key models updated (BERT on TensorFlow, DLRM on PyTorch, etc.)
2. 20+ HuggingFace model quantization

User Experience

1. More comprehensive logging message
2. UI enhancement with FP32 optimization, auto-mixed precision (BF16/FP32), and graph visualization
3. Online document: https://intel.github.io/lpot

Extended Capabilities

1. Model conversion from QAT to Intel Optimized TensorFlow model

Validated Configurations:

• Python 3.6 & 3.7 & 3.8
• Centos 7 & Ubuntu 18.04
• Intel TensorFlow 1.15.2, 2.1.0, 2.2.0, 2.3.0, 2.4.0 and 1.15.0 UP1 & UP2
• PyTorch 1.5.0+cpu, 1.6.0+cpu, ipex
• MxNet 1.7.0
• ONNX Runtime 1.6.0, 1.7.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/lpot.git	$ git clone https://github.com/intel/lpot.git
Binary	Pip	https://pypi.org/project/lpot	$ pip install lpot
Binary	Conda	https://anaconda.org/intel/lpot	$ conda install lpot -c conda-forge -c intel

Contact:

Please feel free to contact lpot.maintainers@intel.com, if you get any questions.

Intel® Low Precision Optimization Tool v1.3.1 Release

Intel® Low Precision Optimization Tool v1.3 release is featured by:

1. Improve graph optimization without explicit input/output setting

Validated Configurations:

• Python 3.6 & 3.7 & 3.8
• Centos 7 & Ubuntu 18.04
• Intel TensorFlow 1.15.2, 2.1.0, 2.2.0, 2.3.0, 2.4.0 and 1.15.0 UP1 & UP2
• PyTorch 1.5.0+cpu, 1.6.0+cpu, ipex
• MxNet 1.7.0
• ONNX Runtime 1.6.0, 1.7.0

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/lpot.git	$ git clone https://github.com/intel/lpot.git
Binary	Pip	https://pypi.org/project/lpot	$ pip install lpot
Binary	Conda	https://anaconda.org/intel/lpot	$ conda install lpot -c conda-forge -c intel

Contact:

Please feel free to contact lpot.maintainers@intel.com, if you get any questions.