Skip to content

ciflow/trunk/21013

@apullin apullin tagged this 18 Jul 15:15
Summary:

ConvertToClampPass normalizes hardtanh, relu and relu6 into aten.clamp. Chains like HardTanh -> ReLU therefore become two adjacent clamps, the second of which is redundant.

Add FuseConsecutiveClampsPass, which folds any chain of adjacent (single-user) clamp.default ops into one clamp using exact composition: clamp(clamp(x,a,b),c,d) == clamp(x, max(a,c), min(b,d)). This subsumes HardTanh->ReLU, ReLU->ReLU, HardTanh->HardTanh, ReLU6->ReLU, explicit clamp->clamp, and longer chains, dropping one op (and one intermediate tensor) per fused pair.

Runs after FoldAndAnnotateQParamsPass so it fires in both the FP and INT paths; in the quantized case the surviving clamp inherits the first clamp's input qparams and the second's output qparams, dropping the intermediate requant (qtol=1, same tradeoff as FuseConsecutiveRescalesPass). Branching clamps (multiple users) are left untouched.

Differential Revision: D112351522
Assets 2
Loading