3. DepGraph & Group
Gongfan Fang edited this page Jul 21, 2023
·
6 revisions
Dependency Graph (DepGraph) is the core feature of Torch-Pruning, which provides an automatic mechanism to group dependent layers. There are two key concepts for DepGraph:
- tp.dependency.Dependency: the dependency between layers.
- tp.dependency.DependencyGraph: A relational graph to model the dependency.
- tp.dependency.Group: A list of dependencies that represents the minimally-removable units.
This tutorial provides an example for ResNet-18.
In Torch-Pruning, DepGraph helps us collect all dependent layers for pruning. First we build the DepGraph with an example inputs, which will be used to trace the computational graph of the model. Then, we choose a layer and select those to-be-pruned channels using the idxs. parameters.
import torch
from torchvision.models import resnet18
import torch_pruning as tp
model = resnet18(pretrained=True).eval()
# 1. Build the DepGraph with an example input.
DG = tp.DependencyGraph().build_dependency(model, example_inputs=torch.randn(1,3,224,224))
# 2. Specify the to-be-pruned channels. Here we prune those channels indexed by [2, 6, 9].
group = DG.get_pruning_group( model.conv1, tp.prune_conv_out_channels, idxs=[2, 6, 9] )
# 3. Prune all coupled layers in this group
group.prune()
output = model(torch.randn(1,3,224,224)) # Testgroup = DG.get_pruning_group( model.conv1, tp.prune_conv_out_channels, idxs=[2, 6, 9] )
# Choose 1. a root layer, 2. a pruning function, 3. a index list
print(group.details()) # use print(group) if you are not interested in the full idxs list.--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)), idxs=[2, 6, 9] (Pruning Root)
[1] prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on bn1 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), idxs=[2, 6, 9]
[2] prune_out_channels on bn1 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_20(ReluBackward1), idxs=[2, 6, 9]
[3] prune_out_channels on _ElementWiseOp_20(ReluBackward1) => prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward), idxs=[2, 6, 9]
[4] prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_out_channels on _ElementWiseOp_18(AddBackward0), idxs=[2, 6, 9]
[5] prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_in_channels on layer1.0.conv1 (Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), idxs=[2, 6, 9]
[6] prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on layer1.0.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), idxs=[2, 6, 9]
[7] prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on _ElementWiseOp_17(ReluBackward1), idxs=[2, 6, 9]
[8] prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_out_channels on _ElementWiseOp_16(AddBackward0), idxs=[2, 6, 9]
[9] prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_in_channels on layer1.1.conv1 (Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), idxs=[2, 6, 9]
[10] prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on layer1.1.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), idxs=[2, 6, 9]
[11] prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on _ElementWiseOp_15(ReluBackward1), idxs=[2, 6, 9]
[12] prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.downsample.0 (Conv2d(61, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)), idxs=[2, 6, 9]
[13] prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.conv1 (Conv2d(61, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)), idxs=[2, 6, 9]
[14] prune_out_channels on layer1.1.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.1.conv2 (Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), idxs=[2, 6, 9]
[15] prune_out_channels on layer1.0.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.0.conv2 (Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), idxs=[2, 6, 9]
--------------------------------
Pruning with the pre-defined idxs:
print(group[0].idxs)
group.prune()[2, 6, 9]
Pruning with new idxs:
new_idxs = [1,2,3,4]
group.prune(new_idxs)for i, (dep, idxs) in enumerate(group):
print("Dep: ", dep, " Idxs:", idxs)Dep: prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) Idxs: [2, 6, 9]
Dep: prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on bn1 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) Idxs: [2, 6, 9]
Dep: prune_out_channels on bn1 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_20(ReluBackward1) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_20(ReluBackward1) => prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_out_channels on _ElementWiseOp_18(AddBackward0) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_in_channels on layer1.0.conv1 (Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on layer1.0.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on _ElementWiseOp_17(ReluBackward1) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_out_channels on _ElementWiseOp_16(AddBackward0) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_in_channels on layer1.1.conv1 (Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on layer1.1.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on _ElementWiseOp_15(ReluBackward1) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.downsample.0 (Conv2d(61, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)) Idxs: [2, 6, 9]
Dep: prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.conv1 (Conv2d(61, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)) Idxs: [2, 6, 9]
Dep: prune_out_channels on layer1.1.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.1.conv2 (Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) Idxs: [2, 6, 9]
Dep: prune_out_channels on layer1.0.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.0.conv2 (Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) Idxs: [2, 6, 9]for i, (dep, idxs) in enumerate(group):
layer = dep.layer
pruning_fn = dep.pruning_fn
print(layer, pruning_fn)Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False) <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f339752f730>>
BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7f339752f7c0>>
_ElementWiseOp_20(ReluBackward1) <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7f3396ec5820>>
_ElementWiseOp_19(MaxPool2DWithIndicesBackward) <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7f3396ec5820>>
_ElementWiseOp_18(AddBackward0) <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7f3396ec5820>>
Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f339752f730>>
BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7f339752f7c0>>
_ElementWiseOp_17(ReluBackward1) <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7f3396ec5820>>
_ElementWiseOp_16(AddBackward0) <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7f3396ec5820>>
Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f339752f730>>
BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7f339752f7c0>>
_ElementWiseOp_15(ReluBackward1) <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7f3396ec5820>>
Conv2d(61, 128, kernel_size=(1, 1), stride=(2, 2), bias=False) <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f339752f730>>
Conv2d(61, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f339752f730>>
Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f339752f730>>
Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f339752f730>>If you are interested in the "Trigger" relation:
for i, (dep, idxs) in enumerate(group):
trigger = dep.trigger
handler = dep.handler
source_layer = dep.source.module
target_layer = dep.target.module
print("For Dep: ", dep)
print(" > Trigger: ", trigger)
print(" > Handler: ", handler)
print(" > Source Layer: ", source_layer)
print(" > Target Layer: ", target_layer)
print("")Here handler & target_layer are identical to dep.pruning_fn & dep.layer respectively.
For Dep: prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False))
> Trigger: <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Handler: <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Source Layer: Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
> Target Layer: Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
For Dep: prune_out_channels on conv1 (Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on bn1 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))
> Trigger: <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Handler: <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7fe4a23cce50>>
> Source Layer: Conv2d(3, 61, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
> Target Layer: BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
For Dep: prune_out_channels on bn1 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_20(ReluBackward1)
> Trigger: <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7fe4a23cce50>>
> Handler: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Source Layer: BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
> Target Layer: _ElementWiseOp_20(ReluBackward1)
For Dep: prune_out_channels on _ElementWiseOp_20(ReluBackward1) => prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward)
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Source Layer: _ElementWiseOp_20(ReluBackward1)
> Target Layer: _ElementWiseOp_19(MaxPool2DWithIndicesBackward)
For Dep: prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_out_channels on _ElementWiseOp_18(AddBackward0)
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Source Layer: _ElementWiseOp_19(MaxPool2DWithIndicesBackward)
> Target Layer: _ElementWiseOp_18(AddBackward0)
For Dep: prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_in_channels on layer1.0.conv1 (Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False))
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Source Layer: _ElementWiseOp_19(MaxPool2DWithIndicesBackward)
> Target Layer: Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
For Dep: prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on layer1.0.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7fe4a23cce50>>
> Source Layer: _ElementWiseOp_18(AddBackward0)
> Target Layer: BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
For Dep: prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on _ElementWiseOp_17(ReluBackward1)
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Source Layer: _ElementWiseOp_18(AddBackward0)
> Target Layer: _ElementWiseOp_17(ReluBackward1)
For Dep: prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_out_channels on _ElementWiseOp_16(AddBackward0)
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Source Layer: _ElementWiseOp_17(ReluBackward1)
> Target Layer: _ElementWiseOp_16(AddBackward0)
For Dep: prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_in_channels on layer1.1.conv1 (Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False))
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Source Layer: _ElementWiseOp_17(ReluBackward1)
> Target Layer: Conv2d(61, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
For Dep: prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on layer1.1.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7fe4a23cce50>>
> Source Layer: _ElementWiseOp_16(AddBackward0)
> Target Layer: BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
For Dep: prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on _ElementWiseOp_15(ReluBackward1)
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Source Layer: _ElementWiseOp_16(AddBackward0)
> Target Layer: _ElementWiseOp_15(ReluBackward1)
For Dep: prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.downsample.0 (Conv2d(61, 128, kernel_size=(1, 1), stride=(2, 2), bias=False))
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Source Layer: _ElementWiseOp_15(ReluBackward1)
> Target Layer: Conv2d(61, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
For Dep: prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.conv1 (Conv2d(61, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False))
> Trigger: <bound method DummyPruner.prune_out_channels of <torch_pruning.ops.ElementWisePruner object at 0x7fe4a1d49eb0>>
> Handler: <bound method ConvPruner.prune_in_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Source Layer: _ElementWiseOp_15(ReluBackward1)
> Target Layer: Conv2d(61, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
For Dep: prune_out_channels on layer1.1.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.1.conv2 (Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False))
> Trigger: <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7fe4a23cce50>>
> Handler: <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Source Layer: BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
> Target Layer: Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
For Dep: prune_out_channels on layer1.0.bn2 (BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.0.conv2 (Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False))
> Trigger: <bound method BatchnormPruner.prune_out_channels of <torch_pruning.pruner.function.BatchnormPruner object at 0x7fe4a23cce50>>
> Handler: <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7fe4a23ccdc0>>
> Source Layer: BatchNorm2d(61, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
> Target Layer: Conv2d(64, 61, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
for g in DG.get_all_groups():
print(g)--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on fc (Linear(in_features=512, out_features=1000, bias=True)) => prune_out_channels on fc (Linear(in_features=512, out_features=1000, bias=True)), #idxs=1000
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer4.0.downsample.0 (Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)) => prune_out_channels on layer4.0.downsample.0 (Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)), #idxs=512
[1] prune_out_channels on layer4.0.downsample.0 (Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)) => prune_out_channels on layer4.0.downsample.1 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=512
[2] prune_out_channels on layer4.0.downsample.1 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_6(AddBackward0), #idxs=512
[3] prune_out_channels on _ElementWiseOp_6(AddBackward0) => prune_out_channels on layer4.0.bn2 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=512
[4] prune_out_channels on _ElementWiseOp_6(AddBackward0) => prune_out_channels on _ElementWiseOp_5(ReluBackward1), #idxs=512
[5] prune_out_channels on _ElementWiseOp_5(ReluBackward1) => prune_out_channels on _ElementWiseOp_4(AddBackward0), #idxs=512
[6] prune_out_channels on _ElementWiseOp_5(ReluBackward1) => prune_in_channels on layer4.1.conv1 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=512
[7] prune_out_channels on _ElementWiseOp_4(AddBackward0) => prune_out_channels on layer4.1.bn2 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=512
[8] prune_out_channels on _ElementWiseOp_4(AddBackward0) => prune_out_channels on _ElementWiseOp_3(ReluBackward1), #idxs=512
[9] prune_out_channels on _ElementWiseOp_3(ReluBackward1) => prune_out_channels on _ElementWiseOp_2(MeanBackward1), #idxs=512
[10] prune_out_channels on _ElementWiseOp_2(MeanBackward1) => prune_out_channels on _Reshape_0(), #idxs=512
[11] prune_out_channels on _Reshape_0() => prune_in_channels on fc (Linear(in_features=512, out_features=1000, bias=True)), #idxs=512
[12] prune_in_channels on fc (Linear(in_features=512, out_features=1000, bias=True)) => prune_out_channels on _ElementWiseOp_1(TBackward), #idxs=512
[13] prune_out_channels on layer4.1.bn2 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer4.1.conv2 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=512
[14] prune_out_channels on layer4.0.bn2 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer4.0.conv2 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=512
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer3.0.downsample.0 (Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)) => prune_out_channels on layer3.0.downsample.0 (Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)), #idxs=256
[1] prune_out_channels on layer3.0.downsample.0 (Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)) => prune_out_channels on layer3.0.downsample.1 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=256
[2] prune_out_channels on layer3.0.downsample.1 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_10(AddBackward0), #idxs=256
[3] prune_out_channels on _ElementWiseOp_10(AddBackward0) => prune_out_channels on layer3.0.bn2 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=256
[4] prune_out_channels on _ElementWiseOp_10(AddBackward0) => prune_out_channels on _ElementWiseOp_9(ReluBackward1), #idxs=256
[5] prune_out_channels on _ElementWiseOp_9(ReluBackward1) => prune_out_channels on _ElementWiseOp_8(AddBackward0), #idxs=256
[6] prune_out_channels on _ElementWiseOp_9(ReluBackward1) => prune_in_channels on layer3.1.conv1 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=256
[7] prune_out_channels on _ElementWiseOp_8(AddBackward0) => prune_out_channels on layer3.1.bn2 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=256
[8] prune_out_channels on _ElementWiseOp_8(AddBackward0) => prune_out_channels on _ElementWiseOp_7(ReluBackward1), #idxs=256
[9] prune_out_channels on _ElementWiseOp_7(ReluBackward1) => prune_in_channels on layer4.0.downsample.0 (Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)), #idxs=256
[10] prune_out_channels on _ElementWiseOp_7(ReluBackward1) => prune_in_channels on layer4.0.conv1 (Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)), #idxs=256
[11] prune_out_channels on layer3.1.bn2 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer3.1.conv2 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=256
[12] prune_out_channels on layer3.0.bn2 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer3.0.conv2 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=256
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer2.0.downsample.0 (Conv2d(58, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)) => prune_out_channels on layer2.0.downsample.0 (Conv2d(58, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)), #idxs=128
[1] prune_out_channels on layer2.0.downsample.0 (Conv2d(58, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)) => prune_out_channels on layer2.0.downsample.1 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=128
[2] prune_out_channels on layer2.0.downsample.1 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_14(AddBackward0), #idxs=128
[3] prune_out_channels on _ElementWiseOp_14(AddBackward0) => prune_out_channels on layer2.0.bn2 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=128
[4] prune_out_channels on _ElementWiseOp_14(AddBackward0) => prune_out_channels on _ElementWiseOp_13(ReluBackward1), #idxs=128
[5] prune_out_channels on _ElementWiseOp_13(ReluBackward1) => prune_out_channels on _ElementWiseOp_12(AddBackward0), #idxs=128
[6] prune_out_channels on _ElementWiseOp_13(ReluBackward1) => prune_in_channels on layer2.1.conv1 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=128
[7] prune_out_channels on _ElementWiseOp_12(AddBackward0) => prune_out_channels on layer2.1.bn2 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=128
[8] prune_out_channels on _ElementWiseOp_12(AddBackward0) => prune_out_channels on _ElementWiseOp_11(ReluBackward1), #idxs=128
[9] prune_out_channels on _ElementWiseOp_11(ReluBackward1) => prune_in_channels on layer3.0.downsample.0 (Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)), #idxs=128
[10] prune_out_channels on _ElementWiseOp_11(ReluBackward1) => prune_in_channels on layer3.0.conv1 (Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)), #idxs=128
[11] prune_out_channels on layer2.1.bn2 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer2.1.conv2 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=128
[12] prune_out_channels on layer2.0.bn2 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer2.0.conv2 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=128
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on conv1 (Conv2d(3, 58, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on conv1 (Conv2d(3, 58, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)), #idxs=58
[1] prune_out_channels on conv1 (Conv2d(3, 58, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)) => prune_out_channels on bn1 (BatchNorm2d(58, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=58
[2] prune_out_channels on bn1 (BatchNorm2d(58, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_20(ReluBackward1), #idxs=58
[3] prune_out_channels on _ElementWiseOp_20(ReluBackward1) => prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward), #idxs=58
[4] prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_out_channels on _ElementWiseOp_18(AddBackward0), #idxs=58
[5] prune_out_channels on _ElementWiseOp_19(MaxPool2DWithIndicesBackward) => prune_in_channels on layer1.0.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=58
[6] prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on layer1.0.bn2 (BatchNorm2d(58, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=58
[7] prune_out_channels on _ElementWiseOp_18(AddBackward0) => prune_out_channels on _ElementWiseOp_17(ReluBackward1), #idxs=58
[8] prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_out_channels on _ElementWiseOp_16(AddBackward0), #idxs=58
[9] prune_out_channels on _ElementWiseOp_17(ReluBackward1) => prune_in_channels on layer1.1.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=58
[10] prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on layer1.1.bn2 (BatchNorm2d(58, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=58
[11] prune_out_channels on _ElementWiseOp_16(AddBackward0) => prune_out_channels on _ElementWiseOp_15(ReluBackward1), #idxs=58
[12] prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.downsample.0 (Conv2d(58, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)), #idxs=58
[13] prune_out_channels on _ElementWiseOp_15(ReluBackward1) => prune_in_channels on layer2.0.conv1 (Conv2d(58, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)), #idxs=58
[14] prune_out_channels on layer1.1.bn2 (BatchNorm2d(58, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.1.conv2 (Conv2d(64, 58, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=58
[15] prune_out_channels on layer1.0.bn2 (BatchNorm2d(58, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on layer1.0.conv2 (Conv2d(64, 58, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=58
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer1.0.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer1.0.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=64
[1] prune_out_channels on layer1.0.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer1.0.bn1 (BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=64
[2] prune_out_channels on layer1.0.bn1 (BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_21(ReluBackward1), #idxs=64
[3] prune_out_channels on _ElementWiseOp_21(ReluBackward1) => prune_in_channels on layer1.0.conv2 (Conv2d(64, 58, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=64
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer1.1.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer1.1.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=64
[1] prune_out_channels on layer1.1.conv1 (Conv2d(58, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer1.1.bn1 (BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=64
[2] prune_out_channels on layer1.1.bn1 (BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_22(ReluBackward1), #idxs=64
[3] prune_out_channels on _ElementWiseOp_22(ReluBackward1) => prune_in_channels on layer1.1.conv2 (Conv2d(64, 58, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=64
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer2.0.conv1 (Conv2d(58, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)) => prune_out_channels on layer2.0.conv1 (Conv2d(58, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)), #idxs=128
[1] prune_out_channels on layer2.0.conv1 (Conv2d(58, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)) => prune_out_channels on layer2.0.bn1 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=128
[2] prune_out_channels on layer2.0.bn1 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_23(ReluBackward1), #idxs=128
[3] prune_out_channels on _ElementWiseOp_23(ReluBackward1) => prune_in_channels on layer2.0.conv2 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=128
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer2.1.conv1 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer2.1.conv1 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=128
[1] prune_out_channels on layer2.1.conv1 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer2.1.bn1 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=128
[2] prune_out_channels on layer2.1.bn1 (BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_24(ReluBackward1), #idxs=128
[3] prune_out_channels on _ElementWiseOp_24(ReluBackward1) => prune_in_channels on layer2.1.conv2 (Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=128
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer3.0.conv1 (Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)) => prune_out_channels on layer3.0.conv1 (Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)), #idxs=256
[1] prune_out_channels on layer3.0.conv1 (Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)) => prune_out_channels on layer3.0.bn1 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=256
[2] prune_out_channels on layer3.0.bn1 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_25(ReluBackward1), #idxs=256
[3] prune_out_channels on _ElementWiseOp_25(ReluBackward1) => prune_in_channels on layer3.0.conv2 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=256
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer3.1.conv1 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer3.1.conv1 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=256
[1] prune_out_channels on layer3.1.conv1 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer3.1.bn1 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=256
[2] prune_out_channels on layer3.1.bn1 (BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_26(ReluBackward1), #idxs=256
[3] prune_out_channels on _ElementWiseOp_26(ReluBackward1) => prune_in_channels on layer3.1.conv2 (Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=256
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer4.0.conv1 (Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)) => prune_out_channels on layer4.0.conv1 (Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)), #idxs=512
[1] prune_out_channels on layer4.0.conv1 (Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)) => prune_out_channels on layer4.0.bn1 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=512
[2] prune_out_channels on layer4.0.bn1 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_27(ReluBackward1), #idxs=512
[3] prune_out_channels on _ElementWiseOp_27(ReluBackward1) => prune_in_channels on layer4.0.conv2 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=512
--------------------------------
--------------------------------
Pruning Group
--------------------------------
[0] prune_out_channels on layer4.1.conv1 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer4.1.conv1 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=512
[1] prune_out_channels on layer4.1.conv1 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)) => prune_out_channels on layer4.1.bn1 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)), #idxs=512
[2] prune_out_channels on layer4.1.bn1 (BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)) => prune_out_channels on _ElementWiseOp_28(ReluBackward1), #idxs=512
[3] prune_out_channels on _ElementWiseOp_28(ReluBackward1) => prune_in_channels on layer4.1.conv2 (Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), #idxs=512
--------------------------------
import torch
from torchvision.models import resnet18
import torch_pruning as tp
model = resnet18()
example_inputs = torch.randn(1, 3, 224, 224)
imp = tp.importance.MagnitudeImportance(p=2)
ignored_layers = []
# DO NOT prune the final classifier!
for m in model.modules():
if isinstance(m, torch.nn.Linear) and m.out_features == 1000:
ignored_layers.append(m)
pruner = tp.pruner.MagnitudePruner(
model,
example_inputs,
importance=imp,
iterative_steps=1,
ch_sparsity=0.2, # remove 50% channels, ResNet18 = {64, 128, 256, 512} => ResNet18_Half = {32, 64, 128, 256}
ignored_layers=ignored_layers,
)
records = []
for g in pruner.step(interactive=True):
dep, idxs = g[0]
layer = dep.layer
pruning_fn = dep.pruning_fn
records.append((layer, idxs, pruning_fn))
g.prune()
for rec in records:
print(rec)
print("")(Conv2d(204, 409, kernel_size=(1, 1), stride=(2, 2), bias=False), [311, 139, 434, 102, 226, 402, 347, 24, 368, 451, 485, 408, 65, 34, 421, 152, 47, 116, 260, 2, 506, 367, 280, 66, 81, 171, 291, 53, 101, 439, 36, 244, 56, 365, 381, 189, 69, 204, 186, 112, 170, 23, 384, 471, 214, 255, 195, 13, 394, 206, 108, 190, 224, 220, 122, 431, 443, 144, 349, 177, 207, 390, 289, 374, 176, 92, 351, 76, 217, 382, 464, 249, 283, 133, 320, 131, 111, 209, 124, 160, 355, 30, 126, 10, 369, 110, 405, 242, 305, 62, 366, 343, 502, 119, 293, 297, 151, 254, 136, 129, 1, 328, 96], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(102, 204, kernel_size=(1, 1), stride=(2, 2), bias=False), [203, 5, 8, 16, 80, 54, 92, 95, 2, 253, 193, 99, 60, 224, 142, 55, 90, 11, 200, 45, 215, 43, 39, 59, 197, 115, 194, 107, 20, 13, 119, 72, 139, 84, 191, 210, 204, 83, 65, 62, 185, 214, 175, 235, 228, 216, 88, 138, 182, 22, 86, 159], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(51, 102, kernel_size=(1, 1), stride=(2, 2), bias=False), [7, 87, 105, 99, 1, 43, 97, 9, 26, 58, 76, 3, 77, 28, 5, 64, 95, 45, 74, 13, 47, 21, 98, 22, 90, 48], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(3, 51, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False), [26, 31, 63, 28, 52, 59, 3, 18, 4, 51, 2, 30, 11], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(51, 51, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False), [36, 45, 4, 58, 44, 47, 8, 30, 46, 16, 28, 27, 9], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(51, 51, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False), [47, 37, 56, 48, 8, 9, 52, 31, 36, 12, 50, 38, 51], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(51, 102, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False), [44, 124, 88, 83, 60, 63, 14, 113, 54, 90, 30, 114, 110, 127, 111, 40, 87, 11, 81, 98, 121, 100, 61, 36, 45, 58], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(102, 102, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False), [118, 61, 82, 1, 51, 86, 37, 70, 113, 58, 66, 125, 79, 55, 122, 72, 22, 48, 7, 50, 41, 99, 107, 21, 6, 23], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(102, 204, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False), [24, 13, 100, 89, 56, 216, 51, 29, 117, 102, 2, 239, 181, 34, 120, 104, 165, 182, 46, 196, 161, 93, 175, 195, 77, 63, 42, 22, 251, 194, 53, 249, 6, 105, 92, 81, 132, 201, 20, 75, 154, 128, 235, 207, 152, 112, 136, 59, 241, 149, 103, 236], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(204, 204, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False), [28, 226, 103, 108, 9, 30, 89, 149, 49, 105, 199, 10, 195, 4, 247, 20, 84, 133, 245, 181, 88, 232, 218, 229, 233, 209, 210, 65, 109, 46, 47, 94, 179, 136, 137, 48, 193, 164, 157, 189, 212, 62, 188, 113, 59, 139, 161, 194, 31, 184, 141, 168], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(204, 409, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False), [219, 326, 509, 359, 463, 505, 159, 496, 311, 106, 488, 145, 60, 323, 81, 482, 284, 168, 243, 207, 401, 445, 441, 85, 510, 174, 328, 487, 200, 411, 449, 161, 375, 169, 259, 27, 389, 215, 454, 15, 125, 124, 452, 410, 148, 327, 291, 262, 472, 293, 356, 166, 308, 172, 355, 374, 462, 273, 113, 32, 38, 392, 191, 483, 442, 435, 477, 340, 307, 51, 209, 139, 37, 456, 163, 319, 347, 430, 105, 313, 155, 208, 30, 149, 310, 47, 8, 183, 271, 24, 63, 244, 203, 225, 469, 100, 267, 398, 65, 147, 151, 280, 199], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
(Conv2d(409, 409, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False), [32, 360, 223, 423, 242, 17, 146, 177, 503, 433, 430, 141, 40, 281, 214, 163, 197, 263, 65, 15, 471, 257, 392, 48, 493, 347, 268, 329, 339, 51, 72, 20, 499, 106, 510, 27, 86, 221, 293, 427, 482, 422, 224, 96, 57, 442, 179, 2, 298, 272, 374, 125, 139, 41, 408, 209, 54, 248, 303, 366, 113, 93, 175, 312, 128, 473, 73, 99, 142, 264, 42, 75, 33, 235, 330, 437, 396, 195, 464, 414, 461, 498, 205, 8, 36, 487, 384, 276, 496, 237, 147, 181, 418, 368, 393, 297, 67, 306, 456, 165, 296, 231, 495], <bound method ConvPruner.prune_out_channels of <torch_pruning.pruner.function.ConvPruner object at 0x7f33583a9d30>>)
Zzz (¦3[▓▓] |