Skip to content

Commit

Permalink
fix ilit miss issue: rename the ilit to lpot in code, ut and descript…
Browse files Browse the repository at this point in the history
…ion (#456)

* create

* rm wrong file

* push missed files

* add ci prepare cmd

* add sudo in env

* fix the env by clone to private env

* fix env setting

* mv the ilit to new folder, clear the output of ipy

* rm temp files

* Lqnguyen branch3 (#210)

* Add bitonic-sort sample.

* Add a note about common file in README.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Move 1d_HeatTransfer sample to open source GitHub.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Updating License file to remove date

* Adding Buffer Object approach.

* Add comment about the location of dpc_common.hpp.

* New sample: Prefix Sum.

* Remove new sample.

* New code sample PrefixSum in ParallelPatterns.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Integrate MPI code sample with dpc_reduce code sample.

* Update README.md

* Update main.cpp

* Integrate MPI with latest dpc_reduce for beta09.

* Update README.md

* Update main.cpp

* Update main.cpp

* Update README.md

* Update CXX to icpx and compiler option for beta09.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Add "export I_MPI_CXX=dpcpp" in sample.json file.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Update json file.

* Sync with master.

* Update bitonic-sort code sample according to the latest guideline.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>

* Lqnguyen branch1 (#201)

* Add bitonic-sort sample.

* Add a note about common file in README.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Move 1d_HeatTransfer sample to open source GitHub.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Updating License file to remove date

* Adding Buffer Object approach.

* Add comment about the location of dpc_common.hpp.

* New sample: Prefix Sum.

* Remove new sample.

* New code sample PrefixSum in ParallelPatterns.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Integrate MPI code sample with dpc_reduce code sample.

* Update README.md

* Update main.cpp

* Integrate MPI with latest dpc_reduce for beta09.

* Update README.md

* Update main.cpp

* Update main.cpp

* Update README.md

* Update CXX to icpx and compiler option for beta09.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Add "export I_MPI_CXX=dpcpp" in sample.json file.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Update json file.

* Sync with master.

* Update the PrefixSum code sample according to the latest guidelines.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Update based on comments from reviewer.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Restructure the Usage function.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>

* Lqnguyen branch2 (#209)

* Add bitonic-sort sample.

* Add a note about common file in README.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Move 1d_HeatTransfer sample to open source GitHub.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Updating License file to remove date

* Adding Buffer Object approach.

* Add comment about the location of dpc_common.hpp.

* New sample: Prefix Sum.

* Remove new sample.

* New code sample PrefixSum in ParallelPatterns.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Integrate MPI code sample with dpc_reduce code sample.

* Update README.md

* Update main.cpp

* Integrate MPI with latest dpc_reduce for beta09.

* Update README.md

* Update main.cpp

* Update main.cpp

* Update README.md

* Update CXX to icpx and compiler option for beta09.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Add "export I_MPI_CXX=dpcpp" in sample.json file.

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

* Update json file.

* Sync with master.

* Update 1d_HeatTransfer code sample according to the new guideline.

* Add comment about dpc_common.hpp .

Signed-off-by: Loc Nguyen <loc.q.nguyen@intel.com>

Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>

* namespace change for montecarlo (#208)

* Adding mandelbrot sample to the repository

Signed-off-by: vmadanan <varsha.madananth@intel.com>

* Adding changes to mandelbrot to remove libsycl-complex.so dependency

* namespace change for Monte Carlo

* Updated samples to newest coding guidelines

* Updating samples- Mandelbrot, DCT and MonteCarlo with newest coding guidelines

* Adding changes to buffer and accessor declarations (#214)

* Initial commit for iso3dfd_dpcpp code sample

Signed-off-by: Gogar, Sunny L <sunny.l.gogar@intel.com>

* Update License.txt

* Update sample.json

* Adding iso3dfd_omp_offload and changing dpc++ compile for windows to dpcpp

* Delete .nfs000000043228fc3f00000140

* Removing build directory accidently checked in

* Update sample.json

Fixing a missing comma

* Adding couple of changes as per Paul's recommendation

* Updating some variable names as per guidelines

* Moving iso3dfd_omp_offload to C++ folder

* Fixing a windows related error about missing std:: for tranform

* Adding algorithm header explicity in iso3dfd.h

* Fixing the sample.json to eliminate recent errors

* Adding changes to buffer and accessor declarations

* Update samples for beta10 release (#207)

* Update simple add sample

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Update make files

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Update fpga make file

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Add dpc_common.hpp

* Update sample.json

* Fix Makefile.win

* Update Makefile.win

* Update sample.json

* Remove dpc_common.hpp

* Update VS project file

* Update README.md

* Update sample.json

* Add stb

* Update read me file

* Initial commit

* Update License.txt

* Change location of matrix multiplication sample

* Fix matrix mul sample VS project file

* Update samples for beta10 release

* Fix for Windows

* Fix for FPGA

* Fix for FPGA

* Fix for FPGA to support both beta09 and beta10

* Add header comment

Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>

* folder structures changes following saumya's request (#217)

* Beta10 GZIP performance update (#204)

* Beta10 GZIP update -- use USM for data transfer

Signed-off-by: Audrey Kertesz <audrey.kertesz@intel.com>

* Trivial change to re-trigger CI

Signed-off-by: Audrey Kertesz <audrey.kertesz@intel.com>

* Update top level README (#222)

* Update top-level README and improve format

Signed-off-by: Audrey Kertesz <audrey.kertesz@intel.com>

* Minor formatting update

Signed-off-by: Audrey Kertesz <audrey.kertesz@intel.com>

* Fix path to oneDPL for Beta10  (#224)

* initial commit of openMP example.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* Initial commit of the dpc_reduce

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* added guid to sample.json

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* fixed sample.json files.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* fixed the include files.  Somehow I copied a slightly old repo and it still had <chrono> and the omp_common.hpp file.  They have been removed.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* added license.txt file ran through formating tool one more time removed all calls to "std::endl" and replaced with " \n"

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* renamed license.txt to License.txt

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* added "ciTests" to the sample.json file.  It passed the check.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* fixed make error

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* fixed sample.json

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* removed "2020" from the License.txt file due to update guidelines.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* added comment regarding where you can find dpc_common in both files per Paul's comments.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* Modified names of the functions to represent what they do (ie. calc_pi_*) per suggestion from Paul.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* initial check-in to the C++ repo

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* put correct comment on dpc_common.hpp

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* added commenting indicating where they can find corresponding include files.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* added comment line

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* removed openMP repo from DPC++ as it will be moved to C++ directory

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* fixed category line in sample.json to match exact text expected.

* removing openMP from the DPC directory.  It has been moved to C++ directory.

* fixed tf_init call

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* removed all calls into PSTL internal logic.  This is what was causing fails between beta08 and beta09.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* fixed env variable to run on CPU

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* update Readme file to include information about setting
env variable to allocate more memory for any runs
on the cpu

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* added option in Cmake file to support unnamed lambda option.   You need this to compile if the environment doesn't have this set by default.

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* path to output file from compile has changed.  it no longer seems to create the src directory.

* started to remove get_access and change it to accessor name()

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* fixed remaining get_access

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* removed commented out old code

Signed-off-by: todd.erdner <todd.erdner@intel.com>

* Fixed path in Cmakelists.txt to suport both beta10 and beta09.  The location of the oneDPL
library changed between the two releases.

* Update CMakeLists.txt

Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>

* Added new Sample (TensorFlow Multinode Training with Horovod) (#197)

* Added new Sample (TensorFlow Multinode Training with Horovod)

Signed-off-by: Shailen Sobhee <shailen.sobhee@intel.com>

* Fixed assert reported by bandit code checker tool.

Signed-off-by: Shailen Sobhee <shailen.sobhee@gmail.com>

* Fix CI issue (MPI bug) - Upload to new folder structure

Signed-off-by: Shailen Sobhee <shailen.sobhee@gmail.com>

* Minor little fix in sample.json; A comma was missing.

Signed-off-by: Shailen Sobhee <shailen.sobhee@gmail.com>

* Removed old references to old folder structure

Signed-off-by: Shailen Sobhee <shailen.sobhee@gmail.com>

* Update third_party_programs.txt (#221)

* Updating License  file to no date in the title /*
 * Copyright (c) 2020 Intel Corporation
 *
 * This program and the accompanying materials are made available under the
 * terms of the The MIT License which is available at
 * https://opensource.org/licenses/MIT.
 *
 * SPDX-License-Identifier: MIT
 */

* Update README.md

* Fix FPGA entries

* Update README.md

Updates per request of sranikonda

* Update README.md

* removing duplicate samples after transfering to dwarves folders

* Update Makefile.win

changing compiler name from "dpcpp-cl" to "dpcpp"

* Update Makefile.win

* Update Makefile.win.fpga

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update README.md

* Update README.md

* Update from Legal Approval of 10/05/2020

Co-authored-by: akertesz <67655634+akertesz@users.noreply.github.com>

* Update Buffers/Accessors according to latest coding guidelines (Matrix_multiply Advisor and VTune). (#215)

* TBB Samples Migration

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Addressing PR Change Requests

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Fill in "Purpose" Section of both README files.

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Remove binary and build files

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* include dpc_common header, remove exception handler, fix json files. (all changes apply to both samples)

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* include dpc_common headers, remove exception handlers (both samples)

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Fix README files, include header files for windows

* Remove namespace, end files, use "std::iota", fix README

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* fix README

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Fix "matrix_multiply" samples failures on Windows.

* buffer/accessor updates for coding guidelines (matrix mul).

Co-authored-by: root <root@dtc-nuc-03l.jf.intel.com>

* oneMKL sample updates for beta10 (#213)

* Jupyter notebooks update as per the latest guidelines (#223)

* updated the simplied version of the accessors, used auto for parallel_for
index

* using vector.size() instead of the global variables as per the comments

* fixed the typo. Also check the output vector size

* Updated Readme to add the include files path for dpc_common.hpp
Updated the cpp file with the comments on dev_utilities folder

* Updated the Jupyter notebooks as per the beta10 guidelines <praveen.k.kundurthy@intel.com>

* removed sample.json as these are jupyter notebooks <praveen.k.kundurthy@intel.com>

* removed some checkpoint files that are not necessary <praveen.k.kundurthy@intel.com>

* removed unwanted files <praveen.k.kundurthy@intel.com>

* removed unwanted checkpoint files <praveen.k.kundurthy@intel.com>

* Samples: block APSP and merge SPMV (#219)

* Update simple add sample

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Update make files

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Update fpga make file

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Add dpc_common.hpp

* Update sample.json

* Fix Makefile.win

* Update Makefile.win

* Update sample.json

* Remove dpc_common.hpp

* Update VS project file

* Update README.md

* Update sample.json

* Add stb

* Update read me file

* Initial commit

* Update License.txt

* Change location of matrix multiplication sample

* Fix matrix mul sample VS project file

* Update samples for beta10 release

* Fix for Windows

* Fix for FPGA

* Fix for FPGA

* Fix for FPGA to support both beta09 and beta10

* Add header comment

* Samples: block apsp and merge spmv

* Add readme files

* Update readme file

* Update sample.json

Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>

* move TF GS sample to new folder structure according to Saumya's direction (#227)

* Update sample.json (#228)

* Update simple add sample

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Update make files

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Update fpga make file

Signed-off-by: Maria, Moushumi <moushumi.maria@intel.com>

* Add dpc_common.hpp

* Update sample.json

* Fix Makefile.win

* Update Makefile.win

* Update sample.json

* Remove dpc_common.hpp

* Update VS project file

* Update README.md

* Update sample.json

* Add stb

* Update read me file

* Initial commit

* Update License.txt

* Change location of matrix multiplication sample

* Fix matrix mul sample VS project file

* Update samples for beta10 release

* Fix for Windows

* Fix for FPGA

* Fix for FPGA

* Fix for FPGA to support both beta09 and beta10

* Add header comment

* Samples: block apsp and merge spmv

* Add readme files

* Update readme file

* Update sample.json

* Update sample.json

Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>

* Edit for readme and some json files (#229)

* Updating License  file to no date in the title /*
 * Copyright (c) 2020 Intel Corporation
 *
 * This program and the accompanying materials are made available under the
 * terms of the The MIT License which is available at
 * https://opensource.org/licenses/MIT.
 *
 * SPDX-License-Identifier: MIT
 */

* Update README.md

* Fix FPGA entries

* Update README.md

Updates per request of sranikonda

* Update README.md

* removing duplicate samples after transfering to dwarves folders

* Update Makefile.win

changing compiler name from "dpcpp-cl" to "dpcpp"

* Update Makefile.win

* Update Makefile.win.fpga

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update README.md

* Update README.md

* Update from Legal Approval of 10/05/2020

* Create README.md

* Add files via upload

* Update README.md

minor modifications to content, purpose and key implementation details.

* Update sample.json

aligned description with readme

* Update README.md

reshuffled parts of the purpose and implementation details and abstracted a few key concepts into better summaries.

* Update sample.json

synched description with readme.

* Update README.md

Co-authored-by: akertesz <67655634+akertesz@users.noreply.github.com>
Co-authored-by: tomlenth <tom.f.lenth@intel.com>

* Changed folder structure (#220)

* Moved model zoo sample to new directory (#216)

* moved model zoo sample to new directory

* added runipy dependency installation

* added error handling

* minor fix

* Updating buffers/accessors for TBB Samples according to coding guidelines. Update CMake files to use defaults. (#230)

* TBB Samples Migration

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Addressing PR Change Requests

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Fill in "Purpose" Section of both README files.

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Remove binary and build files

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* include dpc_common header, remove exception handler, fix json files. (all changes apply to both samples)

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* include dpc_common headers, remove exception handlers (both samples)

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Fix README files, include header files for windows

* Remove namespace, end files, use "std::iota", fix README

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* fix README

Signed-off-by: root <root@dtc-nuc-03l.jf.intel.com>

* Fix "matrix_multiply" samples failures on Windows.

* buffer/accessor updates for coding guidelines (matrix mul).

* Update buffers/accessors for TBB Samples. Update CMake files to use defaults.

Co-authored-by: root <root@dtc-nuc-03l.jf.intel.com>

* Update oneVPL samples for Beta10 (#218)

* Add computed_tomography sample (#212)

* create

* rm wrong file

* push missed files

* add ci prepare cmd

* add sudo in env

* fix the env by clone to private env

* fix env setting

* mv the ilit to new folder, clear the output of ipy

* rm temp files

* change structure

* rebase the update

* rm .gitkeep

* update for new API and config for ilit 1.0 in golden release

* update the script to prepare running env

* optimize for CPU to fix the latency of int8 low issue

* rm unused code

* fix the latency issue by script

* correct the file name in text

* refactor to rename ilit to lpot in code, ut and description

* fix error to quantize model, due to miss rename ilit to lpot

Co-authored-by: lqnguyen <loc.q.nguyen@intel.com>
Co-authored-by: JoeOster <52936608+JoeOster@users.noreply.github.com>
Co-authored-by: vmadananth <12753028+vmadananth@users.noreply.github.com>
Co-authored-by: slgogar <33332238+slgogar@users.noreply.github.com>
Co-authored-by: Moushumi <55515077+moushumi-maria@users.noreply.github.com>
Co-authored-by: Louie Tsai <louie.tsai@intel.com>
Co-authored-by: akertesz <67655634+akertesz@users.noreply.github.com>
Co-authored-by: terdner <todd.erdner@intel.com>
Co-authored-by: Shailen Sobhee <shailen.sobhee@gmail.com>
Co-authored-by: clevels <59889830+clevels@users.noreply.github.com>
Co-authored-by: root <root@dtc-nuc-03l.jf.intel.com>
Co-authored-by: petercad <48329794+petercad@users.noreply.github.com>
Co-authored-by: praveenkk123 <praveen.k.kundurthy@intel.com>
Co-authored-by: tomlenth <tom.f.lenth@intel.com>
Co-authored-by: Jing Xu <jing.xu@intel.com>
Co-authored-by: Jitendra Patil <jitendra.patil@intel.com>
Co-authored-by: Marc Valle <30421017+mav-intel@users.noreply.github.com>
  • Loading branch information
18 people committed Mar 23, 2021
1 parent 3a7f85f commit 591ef67
Show file tree
Hide file tree
Showing 11 changed files with 64 additions and 51 deletions.
@@ -1,13 +1,13 @@
# `Intel(R) Low Precision Optimization Tool (iLiT)` Sample for Tensorflow
# `Intel(R) Low Precision Optimization Tool (LPOT)` Sample for Tensorflow

## Background
Low-precision inference can speed up inference obviously, by converting the fp32 model to int8 or bf16 model. Intel provides Intel(R) Deep Learning Boost technology in the Second Generation Intel(R) Xeon(R) Scalable Processors and newer Xeon(R), which supports to speed up int8 and bf16 model by hardware.

Intel(R) Low Precision Optimization Tool (iLiT) helps the user to simplify the processing to convert the fp32 model to int8/bf16.
Intel(R) Low Precision Optimization Tool (LPOT) helps the user to simplify the processing to convert the fp32 model to int8/bf16.

At the same time, iLiT will tune the quanization method to reduce the accuracy loss, which is a big blocker for low-precision inference.
At the same time, LPOT will tune the quanization method to reduce the accuracy loss, which is a big blocker for low-precision inference.

iLiT is released in Intel(R) AI Analytics Toolkit and works with Intel(R) Optimization of Tensorflow*.
LPOT is released in Intel(R) AI Analytics Toolkit and works with Intel(R) Optimization of Tensorflow*.

Please refer to the official website for detailed info and news: [https://github.com/intel/lp-opt-tool](https://github.com/intel/lp-opt-tool)

Expand All @@ -19,18 +19,18 @@ Code samples are licensed under the MIT license. See
Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt)

## Purpose
This sample will show a whole process to build up a CNN model to recognize handwriting number and speed up it by iLiT.
This sample will show a whole process to build up a CNN model to recognize handwriting number and speed up it by LPOT.

We will learn how to train a CNN model based on Keras with Tensorflow, use iLiT to quantize the model and compare the performance to understand the benefit of iLiT.
We will learn how to train a CNN model based on Keras with Tensorflow, use LPOT to quantize the model and compare the performance to understand the benefit of LPOT.

## Key Implementation Details

- Use Keras on Tensorflow to build and train the CNN model.


- Define function and class for iLiT to quantize the CNN model.
- Define function and class for LPOT to quantize the CNN model.

The iLiT can run on any Intel(R) CPU to quantize the AI model.
The LPOT can run on any Intel(R) CPU to quantize the AI model.

The quantized AI model has better inference performance than the FP32 model on Intel CPU.

Expand All @@ -42,12 +42,12 @@ We will learn how to train a CNN model based on Keras with Tensorflow, use iLiT

## Pre-requirement

| Optimized for | Description
| Optimized for | Description
|:--- |:---
| OS | Linux* Ubuntu* 18.04
| Hardware | The Second Generation Intel(R) Xeon(R) Scalable processor family or newer
| Software | Intel(R) oneAPI AI Analytics Toolkit
| What you will learn | How to use iLiT tool to quantize the AI model based on Tensorflow and speed up the inference on Intel(R) Xeon(R) CPU
| What you will learn | How to use LPOT tool to quantize the AI model based on Tensorflow and speed up the inference on Intel(R) Xeon(R) CPU
| Time to complete | 10 minutes

## Running Environment
Expand Down Expand Up @@ -75,7 +75,7 @@ https://software.intel.com/content/www/us/en/develop/articles/installation-guide

Intel(R) Optimization of Tensorflow* are included in Intel(R) AI Analytics Toolkit. So, no need to install them separately.

This sample depends on **Tensorflow* 2.2**.
This sample depends on **Tensorflow* 2.2** or newer.

### Activate Intel(R) AI Analytics Toolkit

Expand All @@ -99,11 +99,11 @@ conda info -e
#
base * /opt/intel/oneapi/intelpython/latest
pytorch /opt/intel/oneapi/intelpython/latest/envs/pytorch
pytorch-1.5.0 /opt/intel/oneapi/intelpython/latest/envs/pytorch-1.5.0
pytorch-1.7.0 /opt/intel/oneapi/intelpython/latest/envs/pytorch-1.7.0
tensorflow /opt/intel/oneapi/intelpython/latest/envs/tensorflow
tensorflow-2.2.0 /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.2.0
/opt/intel/oneapi/pytorch/1.5.0
/opt/intel/oneapi/tensorflow/2.2.0
tensorflow-2.3.0 /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.3.0
/opt/intel/oneapi/pytorch/1.7.0
/opt/intel/oneapi/tensorflow/2.3.0
```

2. Activate Tensorflow Env.
Expand All @@ -114,11 +114,17 @@ conda activate tensorflow
```

### Install iLiT by Local Channel
### Install LPOT by Local Channel

```
cd /opt/intel/oneapi/LPOT/latest
sudo ./install_LPOT.sh
```
or
```
cd /opt/intel/oneapi/iLiT/latest
sudo ./install_iLiT.sh
```

### Install Jupyter Notebook
Expand Down Expand Up @@ -166,7 +172,7 @@ conda activate /opt/intel/oneapi/intelpython/latest/envs/tensorflow

### Open Sample Code File

In a web browser, open link: **http://yyy:8888/?token=146761d9317552c43e0d6b8b6b9e1108053d465f6ca32fca**. Click 'ilit_sample_tensorflow.ipynb' to start up the sample.
In a web browser, open link: **http://yyy:8888/?token=146761d9317552c43e0d6b8b6b9e1108053d465f6ca32fca**. Click 'lpot_sample_tensorflow.ipynb' to start up the sample.

### Run

Expand Down
@@ -1,4 +1,8 @@
import ilit
try:
import lpot
except:
import ilit as lpot

import alexnet
import math
import mnist_dataset
Expand Down Expand Up @@ -30,7 +34,7 @@ def __iter__(self):

def auto_tune(input_graph_path, yaml_config, batch_size):
fp32_graph = alexnet.load_pb(input_graph_path)
quan = ilit.Quantization(yaml_config)
quan = lpot.Quantization(yaml_config)
dataloader = Dataloader(batch_size)

q_model = quan(
Expand All @@ -47,4 +51,4 @@ def auto_tune(input_graph_path, yaml_config, batch_size):
int8_pb_file = "alexnet_int8_model.pb"

q_model = auto_tune(fp32_frezon_pb_file, yaml_file, batch_size)
save_int8_frezon_pb(q_model, int8_pb_file)
save_int8_frezon_pb(q_model, int8_pb_file)
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Intel® Low Precision Optimization Tool (iLiT) Sample for Tensorflow"
"# Intel® Low Precision Optimization Tool (LPOT) Sample for Tensorflow"
]
},
{
Expand All @@ -13,17 +13,17 @@
"source": [
"## Agenda\n",
"- Train a CNN Model Based on Keras\n",
"- Quantize Keras Model by ilit\n",
"- Quantize Keras Model by LPOT\n",
"- Compare Quantized Model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### iLiT Release and Sample \n",
"### LPOT Release and Sample \n",
"\n",
"This sample code is always updated for the iLiT release in latest oneAPI release.\n",
"This sample code is always updated for the LPOT release in latest oneAPI release.\n",
"\n",
"If you want to get the sample code for old oneAPI release, please checkout the old sample code release by git tag.\n",
"\n",
Expand Down Expand Up @@ -51,7 +51,9 @@
"source": [
"Import python packages and check version.\n",
"\n",
"Make sure the Tensorflow is **2.x** and iLiT, matplotlib are installed."
"Make sure the Tensorflow is **2.2** or newer and LPOT, matplotlib are installed.\n",
"\n",
"Note, LPOT has an old name **ilit**. Following script supports to old package name **ilit**."
]
},
{
Expand All @@ -61,14 +63,16 @@
"outputs": [],
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)\n",
"\n",
"import matplotlib.pyplot as plt\n",
"\n",
"import ilit\n",
"print(ilit.__path__)\n",
"print(\"Tensorflow version {}\".format(tf.__version__))\n",
"\n",
"try:\n",
" import lpot\n",
" print(\"LPOT version {}\".format(lpot.__version__)) \n",
"except:\n",
" import ilit as lpot\n",
" print(\"iLiT version {}\".format(lpot.__version__)) \n",
"\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np"
]
},
Expand Down Expand Up @@ -192,12 +196,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Quantize FP32 Model by iLiT\n",
"## Quantize FP32 Model by LPOT\n",
"\n",
"iLiT supports to quantize the model with a validation dataset for tuning.\n",
"LPOT supports to quantize the model with a validation dataset for tuning.\n",
"Finally, it returns an frezon quantized model based on int8.\n",
"\n",
"We prepare a python script \"**ilit_quantize_model.py**\" to call iLiT to finish the all quantization job.\n",
"We prepare a python script \"**LPOT_quantize_model.py**\" to call LPOT to finish the all quantization job.\n",
"Following code sample is used to explain the code.\n",
"\n",
"### Define Dataloader\n",
Expand Down Expand Up @@ -259,7 +263,7 @@
"source": [
"### Define Yaml File\n",
"\n",
"We define alexnet.yaml to save the necessary parameters for iLiT.\n",
"We define alexnet.yaml to save the necessary parameters for LPOT.\n",
"In this case, we only need to change the input/output according to the fp32 model.\n",
"\n",
"In this case, the input node name is '**x**'.\n",
Expand Down Expand Up @@ -292,11 +296,10 @@
},
"outputs": [],
"source": [
"import ilit\n",
"\n",
"def auto_tune(input_graph_path, yaml_config, batch_size): \n",
" fp32_graph = alexnet.load_pb(input_graph_path)\n",
" quan = ilit.Quantization(yaml_config)\n",
" quan = lpot.Quantization(yaml_config)\n",
" dataloader = Dataloader(batch_size)\n",
" assert(dataloader)\n",
" q_model = quan(\n",
Expand Down Expand Up @@ -324,7 +327,7 @@
"source": [
"### Call Function to Quantize the Model\n",
"\n",
"Show the code in \"**ilit_quantize_model.py**\"."
"Show the code in \"**lpot_quantize_model.py**\"."
]
},
{
Expand All @@ -333,14 +336,14 @@
"metadata": {},
"outputs": [],
"source": [
"!cat ilit_quantize_model.py"
"!cat lpot_quantize_model.py"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will execute the \"**ilit_quantize_model.py**\" to show the whole process of quantizing a model."
"We will execute the \"**lpot_quantize_model.py**\" to show the whole process of quantizing a model."
]
},
{
Expand All @@ -351,7 +354,7 @@
},
"outputs": [],
"source": [
"!python ilit_quantize_model.py"
"!python lpot_quantize_model.py"
]
},
{
Expand Down Expand Up @@ -555,9 +558,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python [conda env:miniconda3-vlpot]",
"language": "python",
"name": "python3"
"name": "conda-env-miniconda3-vlpot-py"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -569,7 +572,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
"version": "3.6.12"
}
},
"nbformat": 4,
Expand Down
Expand Up @@ -2,9 +2,9 @@
"guid": "82e7612f-2810-4d12-9c75-c17fcbb946fa",
"name": "Intel® Low Precision Optimization Tool Tensorflow Getting Started",
"categories": ["Toolkit/oneAPI AI And Analytics/AI Getting Started Samples"],
"description": "This sample illustrates how to run ILIT to quantize the FP32 model trained by Keras on Tensorflow to INT8 model to speed up the inference.",
"description": "This sample illustrates how to run LPOT to quantize the FP32 model trained by Keras on Tensorflow to INT8 model to speed up the inference.",
"languages": [{"python":{}}],
"dependencies": ["tensorflow","iLiT"],
"dependencies": ["tensorflow","lpot"],
"os": ["linux"],
"builder": ["cli"],
"targetDevice": ["CPU"],
Expand All @@ -15,12 +15,12 @@
"conda env remove -n user_tensorflow",
"conda create --name user_tensorflow -y",
"conda activate user_tensorflow",
"conda install -c ${ONEAPI_ROOT}/conda_channel tensorflow numpy pyyaml scikit-learn schema ilit matplotlib -y --offline",
"conda install runipy notebook -y"
],
"id": "ilit tensorflow",
"conda install -c ${ONEAPI_ROOT}/conda_channel tensorflow numpy pyyaml scikit-learn schema lpot ilit matplotlib -y",
"conda install runipy notebook -y"
],
"id": "lpot tensorflow",
"steps": [
"runipy ilit_sample_tensorflow.ipynb"
"runipy lpot_sample_tensorflow.ipynb"
]
}
]
Expand Down

0 comments on commit 591ef67

Please sign in to comment.