Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onnx format conversion to kmodel format bug #1193

Closed
meimei-123 opened this issue Apr 26, 2024 · 2 comments
Closed

Onnx format conversion to kmodel format bug #1193

meimei-123 opened this issue Apr 26, 2024 · 2 comments

Comments

@meimei-123
Copy link

Describe the bug
In my execution:
compiler.import_onnx(model_content, import_options)
Encountered an error:
Process terminated. Assertion Failed
3d4e51ffe6608ca492ec5955e0b7f8b1

So I checked the model structure and annotated it line by line from back to front
Once x=self.fc6 (x) is commented out, it can be run. self.fc6=nn Linear (3072, 192), in ONNX format, is a Gemm operator.
I checked the list of onnx operators supported by nncase and found that there is Gemm.

To Reproduce
Command line or scripts to reproduce the behavior:

Expected behavior
I want to convert the pytorch model into a kmodel model, with the following model structure:

Origin model and code
model:

class Mymodel(nn.Module):
    def __init__(self, C=1024, m=0.2, s=30, n_class=2):
        super(Mymodel, self).__init__()

        self.conv1 = nn.Conv1d(80, C, kernel_size=5, stride=1, padding=2)
        self.relu = nn.ReLU()
        self.bn1 = nn.BatchNorm1d(C)
        self.layer1 = Bottle2neck(C, C, kernel_size=3, dilation=2, scale=8)
        self.layer2 = Bottle2neck(C, C, kernel_size=3, dilation=3, scale=8)
        self.layer3 = Bottle2neck(C, C, kernel_size=3, dilation=4, scale=8)
        # I fixed the shape of the output from MFA layer, that is close to the setting from ECAPA paper.
        self.layer4 = nn.Conv1d(3 * C, 1536, kernel_size=1)
        self.attention = nn.Sequential(
            nn.Conv1d(4608, 256, kernel_size=1),
            nn.ReLU(),
            nn.BatchNorm1d(256),
            nn.Tanh(),  # I add this layer
            nn.Conv1d(256, 1536, kernel_size=1),
            nn.Softmax(dim=2),
        )
        self.bn5 = nn.BatchNorm1d(3072)
        self.fc6 = nn.Linear(3072, 192)
        self.bn6 = nn.BatchNorm1d(192)

    def forward(self, x):
        x = self.conv1(x)
        x = self.relu(x)
        x = self.bn1(x)

        x1 = self.layer1(x)
        x2 = self.layer2(x + x1)
        x3 = self.layer3(x + x1 + x2)

        x = self.layer4(torch.cat((x1, x2, x3), dim=1))
        x = self.relu(x)

        t = x.size()[-1]

        global_x = torch.cat((x, torch.mean(x, dim=2, keepdim=True).repeat(1, 1, t),
                              torch.sqrt(torch.var(x, dim=2, keepdim=True).clamp(min=1e-4)).repeat(1, 1, t)), dim=1)

        w = self.attention(global_x)

        mu = torch.sum(x * w, dim=2)
        sg = torch.sqrt((torch.sum((x ** 2) * w, dim=2) - mu ** 2).clamp(min=1e-4))

        x = torch.cat((mu, sg), 1)
        x = self.bn5(x)
        x = self.fc6(x) # This is where the error occurred,after I annotate the code here and after, it will work fine
        x = self.bn6(x)
        return x

code:

    input_shape = [5, 80, 626] # The input data is a mel spectrogram converted from wav    
    dump_dir = 'tmp/export_onnx'
    if not os.path.exists(dump_dir):
        os.makedirs(dump_dir)

    # onnx simplify
    model_file = onnx_simplify(args.model, dump_dir)

    # compile_options
    compile_options = nncase.CompileOptions()
    compile_options.target = args.target
    compile_options.preprocess = False # 
    compile_options.swapRB = False
    compile_options.dump_ir = True # 是否输出ir
    compile_options.dump_asm = True # 是否输出汇编
    compile_options.dump_dir = dump_dir # 输出目录

    # compiler
    compiler = nncase.Compiler(compile_options)

    # import
    model_content = read_model_file(model_file)
    import_options = nncase.ImportOptions()
    compiler.import_onnx(model_content, import_options)

    # ptq_options
    ptq_options = nncase.PTQTensorOptions()
    ptq_options.samples_count = 6
    ptq_options.set_tensor_data(generate_data(input_shape, ptq_options.samples_count, args.dataset))
    compiler.use_ptq(ptq_options)

Environment (please complete the following information):

  • OS: [Win11 WSL2 Ubuntu20.04]
  • nncase version :nncase 2.8.1, nncase kpu 2.8.1
  • DL Framework :pytorch 2.0.1
  • other: Python 3.8.0, nncase 2.8.1, nncase kpu 2.8.1, numpy=1.19.5, onnx1.9.0, onnx simplier 0.3.6
@curioyang
Copy link
Member

@meimei-123 upload onnx model file.

@meimei-123
Copy link
Author

mymodel_256.txt

Github does not support ONNX format files, so I changed its suffix to. txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants