inference 60 characters, cost 0.86 sencond on cpu, how to acceleration time

[liuhuang31]

hi,
Thanks the provided code and model.
When i use the g2pw to do g2p, it cost too long time.
conv = G2PWConverter(style='pinyin', enable_non_tradional_chinese=True)

I inference 60 characters, cost 0.86s on cpu, have any way to accelerate it? Thanks again.

[yt605155624]

maybe you can convert model to onnx and use onnxruntime, check pr in #5

[liuhuang31]

谢谢，对g2pw代码进行了修改：dataloader也进行了相应修改，最后改成直接预测一整句话，速度大概在0.07～0.13s左右。也是在paddle下增加了g2pw选项。

[yt605155624]

nice~ I have printed the time of G2PWOnnxConverter, the first time of using onnxruntime will be slow (it's onnxruntime's feature)

I will be appreciated if you can include your perfect improvement into paddlespeech after our onnxruntime version of g2pw merged, and I also looking forward to your pr if you are using paddlespeech TTS :)

input text:

我有长头发，我长高了，头发变得长长的，不想长大，你的头发很长

[GitYCC]

谢谢，对g2pw代码进行了修改：dataloader也进行了相应修改，最后改成直接预测一整句话，速度大概在0.07～0.13s左右。也是在paddle下增加了g2pw选项。

@liuhuang31
Thanks for your response. The feature of "predicting whole sentence in one shot" sounds interesting.
Could I invite you to give a PR and become a contributor?
Or, if your time is not available, could you show a piece of codes to help us adding this feature by ourself?

[liuhuang31]

Thanks for your response, i want to give a PR, it is much convenient.

[GitYCC]

@liuhuang31 Thank you! I am looking forward your PR.

[beyondguo]

Hi, could you please tell me how to use G2PWOnnxConverter ? I didn't find it in the code.

[liuhuang31]

Hi, the newest code default use OnnxConverter model to predict, so just install the newest code and use it.

[liuhuang31]

Hi, could you please tell me how to use G2PWOnnxConverter ? I didn't find it in the code.

解决了嘛，没解决话是又遇到啥问题了呢～ @beyondguo

[beyondguo]

@liuhuang31
Hi！
我是这几天直接pip安装的，应该就是最新版了。但感觉预测速度还是比较慢：

from g2pw import G2PWConverter
conv = G2PWConverter(style='pinyin', enable_non_tradional_chinese=True)

%timeit conv('然而，他红了20年以后，他竟退出了大家的视线。')

平均时长：

701 ms ± 30.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

不是初次加载模型，是每次跑都差不多这个时间。我现在的需求是对一个大语料库做注音，所以希望推理速度快一点。

[liuhuang31]

@beyondguo 你可以看看我的pull request, 我在旧的版本上，60个字能达到0.08-0.13秒。

[beyondguo]

@liuhuang31
我刚刚下载了你的版本（https://github.com/liuhuang31/g2pW），测试了一下，一样的代码：

>>> %timeit conv('然而，他红了20年以后，他竟退出了大家的视线。')
<<< 1.44 s ± 39.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

然而更慢了，不知道我是哪里用错了吗[Lol]

[liuhuang31]

@beyondguo 应该没用对，等有时间再跟你讨论，我先忙工作事情，比较紧急

[JohnHerry]

确实慢，有办法做模型精简吗？onnx也慢。

[liuhuang31]

g2pw每次预测会先做句子分词，然后一句话可能会分成10次，那么就要调用10次去预测。
没有做模型精简，我是直接修改代码，直接一句话输入进去，只预测一次，所以速度很快。

可以参考之前的一个代码，大概逻辑是：循环生成数据变成只一次就生成预测数据；模型循环预测变成只一次调用模型。
去年写的代码了，有些细节有点忘了。
https://github.com/liuhuang31/g2pW