Skip to content

Support different backends for adaptors #157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 82 commits into from
Dec 9, 2022
Merged

Support different backends for adaptors #157

merged 82 commits into from
Dec 9, 2022

Conversation

mengniwang95
Copy link
Contributor

Signed-off-by: Mengni Wang mengni.wang@intel.com

Type of Change

feature

Description

detail description
Support different backends for ort adaptor

Expected Behavior & Potential Risk

For default backend, ONNX models' accuracy & perf keep unchanged.
For onnxrt_trt_ep backend, the generated int8 onnx model can run with trt ep.

How has this PR been tested?

extension test

Dependency Change?

no

@mengniwang95 mengniwang95 marked this pull request as ready for review December 1, 2022 02:26
@mengniwang95
Copy link
Contributor Author

mengniwang95 commented Dec 1, 2022 •
edited
Loading

test part of models:https://inteltf-jenk.sh.intel.com/job/intel-lpot-validation-top-mr-extension/3520/

@mengniwang95 mengniwang95 force-pushed the mengni/backend branch 2 times, most recently from 98aeb78 to eb022e1 Compare December 2, 2022 08:51
@mengniwang95 mengniwang95 changed the title Support different backends for ort adaptor Support different backends for adaptors Dec 5, 2022
ftian1
ftian1 reviewed Dec 5, 2022
View reviewed changes
ftian1
ftian1 reviewed Dec 5, 2022
View reviewed changes
mengniwang95
mengniwang95 commented Dec 5, 2022
View reviewed changes
mengniwang95 and others added 20 commits December 7, 2022 09:57
@mengniwang95 @PenghuiCheng
fix scale zp calculation
bb91ae0 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@mengniwang95 @PenghuiCheng
fix import error
8fe8ac0 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@mengniwang95 @PenghuiCheng
fix torch bug
82a59f4 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@mengniwang95 @PenghuiCheng
add backend/output_format
09cd750 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@mengniwang95 @PenghuiCheng
fix bug
03bba71 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@mengniwang95 @PenghuiCheng
add backend to benchamrk
0b8ef86 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@PenghuiCheng
Change the behavior of backend for PyTorch
925b342 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@PenghuiCheng
fixed UT error
804a27f 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@PenghuiCheng
fixed UT error
300167f 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@PenghuiCheng
Fixed UT error for PyTorch dynamic quantization
69cb49c 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@lvliang-intel @PenghuiCheng
Add backend for tensorflow
abce1c1 
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
@mengniwang95 @PenghuiCheng
fix trt acc
f2778c9 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@lvliang-intel @PenghuiCheng
fit tf ut issues
fdb12a2 
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
@PenghuiCheng
Change approach value and move the path of set_workspace...and so on.
350b2b4 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@mengniwang95 @PenghuiCheng
remove fp16 and output_format
65e1425 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@mengniwang95 @PenghuiCheng
add file
307d9ac 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@lvliang-intel @PenghuiCheng
remove all None parameters and update with quant_format
5b77e1c 
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
@mengniwang95 @PenghuiCheng
fix ut
6790465 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@PenghuiCheng
Fixed some UT error
90fd69d 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@mengniwang95 @PenghuiCheng
update get/set_backend
34ca56d 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
mengniwang95 and others added 10 commits December 7, 2022 22:56
@mengniwang95 @PenghuiCheng
fix for TRT EP
e5084c4 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@mengniwang95 @PenghuiCheng
fix onnx ut failure
af075cb 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@PenghuiCheng
fixed benchmark error with b_func
f7f6fe4 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@lvliang-intel @PenghuiCheng
fix itex ut issue
0be8062 
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
@PenghuiCheng
Fixed export UT error
73ef020 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@lvliang-intel @PenghuiCheng
ut update for model change
06b12be 
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
@PenghuiCheng
Fixed examples error and UT error
fae4872 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@PenghuiCheng
Change the export API for compression_manager
98b8e36 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@lvliang-intel @PenghuiCheng
update ut for code change
f4475ac 
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
@PenghuiCheng
rebase master branch and changed code style
1fd9407 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@PenghuiCheng
Update nlp text-classification examples with new API
74cf9fc 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@lvliang-intel lvliang-intel mentioned this pull request Dec 8, 2022
Merged
lvliang-intel and others added 5 commits December 8, 2022 14:35
@lvliang-intel
fix strategy ut issue
a2fde02 
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
@mengniwang95
fix pylint
a28d2bf 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@PenghuiCheng
Fixed PyTroch UT errors
9cc9792 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@mengniwang95
fix mxnet example
948cd4e 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
@PenghuiCheng
Fixed PyTorch UT error
578098d 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@chensuyue chensuyue added the enhancement New feature or request label Dec 9, 2022
@chensuyue chensuyue added this to the v2.0 milestone Dec 9, 2022
@PenghuiCheng PenghuiCheng requested a review from ftian1 December 9, 2022 04:24
@PenghuiCheng
Update examples
d7677c3 
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
@chensuyue chensuyue merged commit 2483a84 into master Dec 9, 2022
@chensuyue chensuyue deleted the mengni/backend branch December 9, 2022 05:41
chensuyue pushed a commit that referenced this pull request Dec 12, 2022
@mengniwang95 @PenghuiCheng
@lvliang-intel @chensuyue
Support different backends for adaptors (#157)
03d93f2 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
Co-authored-by: Cheng, Penghui <penghui.cheng@intel.com>
Co-authored-by: Lv, Liang1 <liang1.lv@intel.com>
(cherry picked from commit 2483a84)
zehao-intel pushed a commit that referenced this pull request Dec 20, 2022
@mengniwang95 @PenghuiCheng
@lvliang-intel @zehao-intel
Support different backends for adaptors (#157)
2af07a4 
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com>
Signed-off-by: Lv, Liang1 <liang1.lv@intel.com>
Co-authored-by: Cheng, Penghui <penghui.cheng@intel.com>
Co-authored-by: Lv, Liang1 <liang1.lv@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
VincyZhang pushed a commit that referenced this pull request Feb 12, 2023
@yi1ding
VNNI kernel blocking (#157)
bfbdd06 
* vnni kernel blocking

* support micro bs

* use template for vnni_data_t

* clang-format

* clearer vnni params

* add perf test cases to ut

* addition private fields for performance

* perf tests and small changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ftian1 ftian1 ftian1 approved these changes

@lvliang-intel lvliang-intel lvliang-intel approved these changes

@xin3he xin3he Awaiting requested review from xin3he

+1 more reviewer

@PenghuiCheng PenghuiCheng PenghuiCheng approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

v2.0

Development

Successfully merging this pull request may close these issues.

5 participants

@mengniwang95 @ftian1 @PenghuiCheng @lvliang-intel @chensuyue