Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'NoneType' object has no attribute 'name' in FusedBatchNorm #925

Open
machanic opened this issue Jun 28, 2021 · 3 comments

Comments

@machanic
Copy link

Platform (like ubuntu 16.04/win10): ubuntu 16.04

Python version: 3.7.3

Source framework with version (like Tensorflow 1.4.1 with GPU): Tensorflow 1.4.0

Destination framework with version (like CNTK 2.3 with GPU): PyTorch 1.7.1

Pre-trained model path (webpath or webdisk path): https://drive.google.com/file/d/1awFM8y4A9jWcfUFaVs6S3CBGZramLBkY/view?usp=sharing
Running scripts: mmconvert -sf tensorflow -in model-50.meta -iw model-50 --dstNode resnet10/logits/BiasAdd -df pytorch -om model-50.pth

@machanic
Copy link
Author

It seems that this is a bug that caused by Fused Batch Normalization layer, but I did not use that layer.

I only use tf.layers.batch_normalization as the BN layer in ResNet-10, but I didn't pass fused argument, which is set to None by default. None means it uses a faster, fused implementation if possible

@JiahaoYao
Copy link
Contributor

JiahaoYao commented Jun 28, 2021

Hi @machanic , I looked into the network. Here is a temporal solution:

You might want to replace these two liners with

        mean = self.get_parent(source_node.name, [3], True)
        var = self.get_parent(source_node.name, [4], True)

also, replace these two liners with

            self.set_weight(source_node.name, 'mean', self.ckpt_data[mean.name.replace('Const', 'moving_mean')])

            self.set_weight(source_node.name, 'var', self.ckpt_data[var.name.replace('Const_1', 'moving_variance')])

You can find your src path via import mmdnn; print(mmdnn.__path__)

Most likely, after the temporal hotfixes, you can get

    def forward(self, x):
        resnet10_Pad    = F.pad(x, (3, 3, 3, 3), mode = 'constant', value = 0)
        resnet10_conv1_Conv2D = self.resnet10_conv1_Conv2D(resnet10_Pad)
        resnet10_pool1_MaxPool, resnet10_pool1_MaxPool_idx = F.max_pool2d(resnet10_conv1_Conv2D, kernel_size=(3, 3), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_pool1_MaxPool)
        resnet10_block1_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block1_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block1_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Pad)
        resnet10_block1_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block1_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_add = resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block1_unit_1_bottleneck_v2_add)
        resnet10_block2_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block2_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block2_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Pad)
        resnet10_block2_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block2_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_add = resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block2_unit_1_bottleneck_v2_add)
        resnet10_block3_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block3_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block3_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Pad)
        resnet10_block3_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block3_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_add = resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block3_unit_1_bottleneck_v2_add)
        resnet10_block4_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block4_unit_1_bottleneck_v2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block4_unit_1_bottleneck_v2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D_pad = F.pad(resnet10_block4_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1))
        resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D_pad)
        resnet10_block4_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block4_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_add = resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_batch_normalization_FusedBatchNorm = self.resnet10_batch_normalization_FusedBatchNorm(resnet10_block4_unit_1_bottleneck_v2_add)
        resnet10_Relu   = F.relu(resnet10_batch_normalization_FusedBatchNorm)
        resnet10_pool5  = torch.mean(resnet10_Relu, 3, True)
        resnet10_pool5  = torch.mean(resnet10_pool5, 2, True)
        resnet10_logits_Conv2D = self.resnet10_logits_Conv2D(resnet10_pool5)
        return resnet10_logits_Conv2D

Good Luck!

@JiahaoYao
Copy link
Contributor

Hi @machanic, did this solve your issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants