Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Quant][PT2E] Enable qconv for quantization 2.0 export #104580

Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
21eacd8
Enable qconv for quantization 2.0 export
leslie-fang-intel Jul 4, 2023
80a9fb8
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 4, 2023
f7a0604
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 4, 2023
78a2579
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 6, 2023
8f7c1a1
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 6, 2023
8b4eb59
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 6, 2023
01a84a5
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 6, 2023
99c8ecb
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 7, 2023
97bfd1d
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 7, 2023
f7993ca
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 7, 2023
c60a896
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 7, 2023
a60fc29
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 10, 2023
4c409e3
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 10, 2023
3185fb1
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 11, 2023
715658f
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 13, 2023
8f53a09
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 13, 2023
8ae10dc
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 17, 2023
12f5695
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 19, 2023
6136b52
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 19, 2023
6d05daf
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 24, 2023
81a55cf
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 25, 2023
1c58482
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 25, 2023
b83600a
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 28, 2023
772550b
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Jul 30, 2023
2da245a
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 3, 2023
9dac4ee
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 8, 2023
3e3becc
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 11, 2023
4e03a45
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 12, 2023
d156d52
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 16, 2023
70d4f73
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 16, 2023
1679d46
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 18, 2023
583d131
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 21, 2023
0b90367
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 21, 2023
3800cab
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 23, 2023
4da74ec
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 25, 2023
b0dca15
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 25, 2023
e67e9bb
Update on "[Quant][PT2E] Enable qconv for quantization 2.0 export"
leslie-fang-intel Aug 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
51 changes: 44 additions & 7 deletions aten/src/ATen/native/quantized/cpu/OnednnUtils.h
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,18 @@
#include <cpuinfo.h>

#include <c10/util/CallOnce.h>
#endif // #if AT_MKLDNN_ENABLED()

enum PostOps {
NoPostOp,
Relu,
LeakyRelu,
Tanh,
Add,
AddRelu,
};

#if AT_MKLDNN_ENABLED()
using PrimitiveCacheKey = std::tuple<
double, // input_scale
int64_t, // input_zero_point
Expand Down Expand Up @@ -107,13 +118,6 @@ struct DeconvPrimitiveCache : PrimitiveCache {
}
};

enum PostOps {
NoPostOp,
Relu,
LeakyRelu,
Tanh,
};

struct PackedLinearWeightsOnednn : public LinearPackedParamsBase {
PackedLinearWeightsOnednn(
std::unique_ptr<ideep::tensor> weight,
Expand Down Expand Up @@ -379,4 +383,37 @@ static bool should_use_onednn_quant(

} // onednn_utils

at::Tensor _qconv_prepack_pt2e(
at::Tensor weight, // from CPU backend instead of QuantizedCPU
at::Tensor weight_scales, // Weight zero points must be 0 for onednn
torch::List<int64_t> input_shape,
double input_scale,
int64_t input_zero_point,
torch::List<int64_t> stride,
torch::List<int64_t> padding,
torch::List<int64_t> dilation,
int64_t groups);

template <PostOps postOpFused>
static at::Tensor _quantized_convolution_pt2e(
at::Tensor act, // contains quantized values but not QTensor
double act_scale,
int64_t act_zero_point,
at::Tensor weight, // MKLDNN tensor with quantized values
at::Tensor weight_scales,
at::Tensor weight_zero_points,
c10::optional<at::Tensor> bias, // Bias is packed if not None
torch::List<int64_t> stride,
torch::List<int64_t> padding,
torch::List<int64_t> dilation,
bool transposed,
int64_t groups,
double output_scale,
int64_t output_zero_point,
c10::optional<at::Tensor> accum=c10::nullopt, // accum to fused with conv add
double accum_scale=1.0,
int64_t accum_zero_point=0,
bool fp32_output=false,
const c10::optional<c10::ArrayRef<c10::IValue>>& post_op_args=c10::nullopt);

#endif // #if AT_MKLDNN_ENABLED()