Add C++ inference example and code #191

ZDisket · 2020-08-07T02:52:29Z

This is complete C++ code (from text processing to saving audio) for inference with TensorflowTTS/FastSpeech2 (phonetic MFA-aligned from my fork) and Multi-Band MelGAN using the Tensorflow C API. Can compile and run for Windows 64-bit out of the box(solution and project), but the code is cross-platform assuming one provides the required libraries. The project builds a simple command line program where one inputs sentences and they are generated and saved as WAVs.

There's a link for compiled binaries, libraries, and a sample model required to compile for Win64 in the README.

It will allow deploying TensorflowTTS models in a portable way into desktop environments.

dathudeptrai · 2020-08-07T03:00:37Z

@candlewill @machineko @azraelkuan can you guys take a look :))).

dathudeptrai · 2020-08-07T03:22:01Z

@ZDisket can we run it on ubuntu ? :))

ZDisket · 2020-08-07T03:30:19Z

@dathudeptrai Out of the box, you can try WINE, it works for simple console apps in my experience. Since the program has no platform dependent code, to compile, you just need libs and includes for Linux versions of Phonetisaurus, OpenFST and Tensorflow C API (that one can be downloaded, the rest you'll have to compile from source).

machineko · 2020-08-07T10:07:59Z

@dathudeptrai Out of the box, you can try WINE, it works for simple console apps in my experience. Since the program has no platform dependent code, to compile, you just need libs and includes for Linux versions of Phonetisaurus, OpenFST and Tensorflow C API (that one can be downloaded, the rest you'll have to compile from source).

Can you add it in readme or make some bash file to download dependencies based on platform?

ZDisket · 2020-08-07T15:12:08Z

@machineko I'll add to the README what has to be downloaded. The users will have to compile those from source.

…atforms

ZDisket · 2020-08-07T17:30:20Z

@machineko @dathudeptrai The backend's mostly complete (going to start work on TensorVox now); look good? Any success running the demo in Ubuntu with WINE? Also, do any of these people that were requested know C++?

machineko · 2020-08-07T17:33:22Z

@dathudeptrai Didnt have time yet to compile everything (im on arch not ubuntu but still shouldnt be a problem) after weekend I will try it :P

dathudeptrai · 2020-08-07T17:35:07Z

Also, do any of these people that were requested know C++?

@candlewill is a master in C++ :)), also in TTS. @azraelkuan not sure. @machineko not sure :)). About me: almost zero =)))))))))))))))))))). So i can give you 1 approve rightnow, haha :)).

machineko · 2020-08-07T10:22:02Z

examples/cppwin/TensorflowTTSCppInference/Voice.cpp

@@ -0,0 +1,99 @@
+#include "Voice.h"
+#include "ext/ZCharScanner.h"
+const std::vector<std::string> Phonemes = { "AA","AA0","AA1","AA2","AE","AE0","AE1","AE2","AH","AH0","AH1","AH2","AO","AO0","AO1",


Shouldn't we consider making it more versatile and just load phonemes and phonemes id from files?

Same about every other constants that can be transfered to file based config used in the project

I also considered that before deciding against it; this is just an example and if someone wants to adapt it to their dataset then they should take it then change it. It's not designed to be a one size fits all solution, but something that people can use as a base for their project.

yeah :)) it's just example :D, we do not need make it general :D. Flexible is more important :D

Maybe but every other example right now can be used in very easy way just changing few things in scripting part
I think we should move in this direction. (In future prepro there will be saving all needed mappers so you can load phonemes and other mappers from json files just like configs right now :P)

Still im not following any C++ standards so this is a question for @candlewill

:))) for this case, i just to make sure it can be run then i will approve merge =)))))))))))))))). want to learn c++ :'(

In future prepro there will be saving all needed mappers so you can load phonemes and other mappers from json files just like configs right now

Why not leave it like this right now then later when those preprocessing steps are implemented I make it so that the program can load exported dictionaries? Because another issue is to agree on a format for storing the characters and IDs.

We can just wait for @candlewill to check it and whole pr :P

examples/cppwin/TensorflowTTSCppInference/TensorflowTTSCppInference.vcxproj.filters

machineko · 2020-08-07T17:37:53Z

I have almost zero exp in c++ but a lot in pure C mostly for optimization :P

machineko · 2020-08-07T20:08:35Z

Also did u do some benchmarks it is faster than python based version :D?

ZDisket · 2020-08-07T21:29:34Z

@machineko For a rough estimate of performance, see #179 (comment) . I never ran the Python version in my local PC, so I can't compare.

machineko · 2020-08-07T21:44:18Z

I'll compare later in next week on i9-9900k :P

candlewill

Thanks @ZDisket for providing these useful c++ code for inference on Windows OS.

To compile the cpp code, Microsoft visual studio is required and I don't have a working visual studio environment. Could you also provide a CMakeList file to make it possible to compile on linux based system?

ZDisket · 2020-08-10T02:46:54Z

@candlewill The only experience developing on Linux that I have is with Qt Creator, I can make a Qt project for compiling on Ubuntu 18.04.

candlewill · 2020-08-10T02:54:22Z

@ZDisket Well, I would base on your code to support CMake compilation.

machineko · 2020-08-12T22:06:00Z

I also can't compile on gcc 7.5 on arch will upgrade gcc later

ZDisket · 2020-08-12T22:29:32Z

There's now Linux support, a precompiled demo can be downloaded (tested in Kubuntu 18.04) here, and build instructions (w/qmake) are included in the README.

machineko · 2020-08-12T22:47:47Z

It's working for me on 7.5 gcc, on 9.3 I got (probably tensorflow stuff)

*** stack smashing detected ***: <unknown> terminated
Aborted.

I'll benchmark it later

dathudeptrai · 2020-08-13T02:23:09Z

@ZDisket @machineko it run so fast in my laptop CPU :))

ZDisket · 2020-08-13T04:48:43Z

@dathudeptrai So, you like it?

dathudeptrai · 2020-08-13T05:13:30Z

@dathudeptrai So, you like it?

:))) the question is so serious :))). Just wait @candlewill review again :v

ZDisket · 2020-08-13T06:07:14Z

@dathudeptrai Don't worry, I was just surprised it was running fast, not in a hurry right now especially since I'm busy with the tool.

machineko · 2020-08-13T16:44:27Z

It works fine on ubuntu but I still can't compile it on arch (i've tried gcc 7.5, 8, 10.3 and diff versions of dependencies dunno if its cause of tensorflow but i won't be trying any more right now :P).
Also can u compile second version as CLI or some rest API to test it?

orikama · 2020-08-15T12:15:05Z

Since I was getting this error: "was created with an older compiler than other objects; rebuild old objects and libraries" for Phonetisaurus and OpenFST libs while building this project in Visual Studio 2019, I wrote normal, not auto generated CMake, that builds Phonetisaurus, OpenFST and this project from source. Also tested it on Manjaro(arch) with gcc 10.
And changed Phonetisaurus sources a little to fix linker errors, because I couldn't find gcc analog for /FORCE :)

orikama · 2020-08-15T15:19:59Z

CMakeLists.txt
Phonetisaurus.zip

ZDisket · 2020-08-15T15:32:28Z

@orikama My fork of Phonetisaurus that I made for Linux compiling already has the required change to not require /FORCE; soon I'll make that one Windows compatible too.
And while I appreciate those CMake lists you made, I won't add them here because I can't maintain them. This C++ implementation will be updated in the future to keep up with the repo's developments.

orikama · 2020-08-15T15:39:57Z

@ZDisket why you just can't get me to help? Plus, you depend on QT in Phonetisaurus which is unnecessary.
And what will be you solution to people who might want to compile this in VS 2019 ?
Although, I don't mind. I'll just fork it then, want to play with it for a little.

ZDisket · 2020-08-15T15:56:39Z

@orikama

And what will be you solution to people who might want to compile this in VS 2019 ?

Tell them to compile Phonetisaurus and OpenFST from source in VS2019. I provide prebuilt for VS2017 for convenience, but there's nothing stopping them from compiling it themselves. I'll update my Phonetisaurus fork to support Windows in a bit.

Plus, you depend on QT in Phonetisaurus which is unnecessary.

Good point, I'll modify the Phonetisaurus README to use qmake instead of the full IDE.

why you just can't get me to help?

I assumed your contribution was just a one-time thing only. Let me think about it.

orikama · 2020-08-15T16:15:29Z

@ZDisket btw AudioFile.hpp also gave me error, when I was trying to build it on manjaro with gcc 10.
I tried make CXXFLAGS="-fpermissive", but it didn't worked for me. I'm not familiar with gcc and make, so may be I did it wrong.
So I changed AudioFile also to make it work.

Tell them to compile Phonetisaurus and OpenFST from source in VS2019

I was lazy to figure out how to build Phonetisaurus with 'make' on windows, so I almost gave up :) Especially since you don't need to build it all and only need to include a couple files from it.

ZDisket · 2020-08-15T16:22:18Z

@orikama Weird that -fpermissive didn't work, but it did for me.

I was lazy to figure out how to build Phonetisaurus with 'make' on windows, so I almost gave up :) Especially since you don't need to build it all and only need to include a couple files from it.

That's why I'm providing my fork to allow people to compile it, because this doesn't use Phonetisaurus but rather libPhonetisaurus, my modification with all the executables gone and only libraries.

orikama · 2020-08-15T16:27:33Z

@ZDisket If I'm not mistaken you still use more files from Phonetisaurus than you need to. You should check what files I left in my Phonetisaurus.zip

ZDisket · 2020-08-15T16:30:18Z

@orikama Interesting, I'll check it out.

dathudeptrai · 2020-08-20T05:14:27Z

@ZDisket Finish ?

ZDisket · 2020-08-20T05:17:04Z

@dathudeptrai Should be, I don't have anything else to do except clean up the libPhonetisaurus repo, but of course that's somewhere else.

Add C++ inference code and README

a581b41

ZDisket mentioned this pull request Aug 7, 2020

Cannot run mb_melgan inference with C API: You must feed a value for placeholder tensor 'saver_filename' with dtype string #179

Closed

dathudeptrai requested review from azraelkuan, candlewill and machineko August 7, 2020 03:00

Remove unused precompiled header and SampleRate in Voice.h

d785f84

ZDisket added 3 commits August 7, 2020 12:20

Update CppInference README to include build instructions for other pl…

f17f0ff

…atforms

Finish up CppInference README, fix list formatting issue.

40d69b8

Disable warnings from included headers and some code

ab270b1

dathudeptrai self-requested a review August 7, 2020 17:10

dathudeptrai previously approved these changes Aug 7, 2020

View reviewed changes

machineko suggested changes Aug 7, 2020

View reviewed changes

Ensure that TextTokenizer will add single word numbers

dc517be

ZDisket dismissed dathudeptrai’s stale review via dc517be August 7, 2020 23:04

Merge remote-tracking branch 'upstream/master' into cppinference

54aa9b3

candlewill reviewed Aug 10, 2020

View reviewed changes

Make code more GCC-tolerant and add Qt project for Linux compiling

18c8e3e

ZDisket dismissed machineko’s stale review via 18c8e3e August 12, 2020 22:08

Update .pro and README (Linux support)

814eddb

Add link to Colab notebook for model conversion in README

95e96a3

Merge branch 'master' into cppinference

d71e6d4

ZDisket added 2 commits August 19, 2020 01:04

Remove fstfar and ngram from .pro

594bbf9

Clarify model requirement for using precompiled demo

a0e0a66

Merge branch 'master' into cppinference

3921fc7

dathudeptrai removed the request for review from azraelkuan August 20, 2020 05:17

dathudeptrai approved these changes Aug 20, 2020

View reviewed changes

dathudeptrai merged commit b38187c into TensorSpeech:master Aug 20, 2020

Add C++ inference example and code #191

Add C++ inference example and code #191

Conversation

ZDisket commented Aug 7, 2020 • edited

dathudeptrai commented Aug 7, 2020

dathudeptrai commented Aug 7, 2020

ZDisket commented Aug 7, 2020

machineko commented Aug 7, 2020 • edited

ZDisket commented Aug 7, 2020

ZDisket commented Aug 7, 2020 • edited

machineko commented Aug 7, 2020

dathudeptrai commented Aug 7, 2020 • edited

machineko Aug 7, 2020

Choose a reason for hiding this comment

machineko Aug 7, 2020

Choose a reason for hiding this comment

ZDisket Aug 7, 2020

Choose a reason for hiding this comment

dathudeptrai Aug 7, 2020 • edited

Choose a reason for hiding this comment

machineko Aug 7, 2020 • edited

Choose a reason for hiding this comment

dathudeptrai Aug 7, 2020 • edited

Choose a reason for hiding this comment

ZDisket Aug 7, 2020

Choose a reason for hiding this comment

machineko Aug 7, 2020

Choose a reason for hiding this comment

machineko commented Aug 7, 2020

machineko commented Aug 7, 2020

ZDisket commented Aug 7, 2020

machineko commented Aug 7, 2020

candlewill left a comment

Choose a reason for hiding this comment

ZDisket commented Aug 10, 2020

candlewill commented Aug 10, 2020

machineko commented Aug 12, 2020

ZDisket commented Aug 12, 2020 • edited

machineko commented Aug 12, 2020 • edited

dathudeptrai commented Aug 13, 2020 • edited

ZDisket commented Aug 13, 2020

dathudeptrai commented Aug 13, 2020

ZDisket commented Aug 13, 2020 • edited

machineko commented Aug 13, 2020 • edited

orikama commented Aug 15, 2020

orikama commented Aug 15, 2020

ZDisket commented Aug 15, 2020 • edited

orikama commented Aug 15, 2020 • edited

ZDisket commented Aug 15, 2020 • edited

orikama commented Aug 15, 2020 • edited

ZDisket commented Aug 15, 2020

orikama commented Aug 15, 2020 • edited

ZDisket commented Aug 15, 2020

dathudeptrai commented Aug 20, 2020

ZDisket commented Aug 20, 2020

ZDisket commented Aug 7, 2020 •

edited

machineko commented Aug 7, 2020 •

edited

ZDisket commented Aug 7, 2020 •

edited

dathudeptrai commented Aug 7, 2020 •

edited

dathudeptrai Aug 7, 2020 •

edited

machineko Aug 7, 2020 •

edited

dathudeptrai Aug 7, 2020 •

edited

ZDisket commented Aug 12, 2020 •

edited

machineko commented Aug 12, 2020 •

edited

dathudeptrai commented Aug 13, 2020 •

edited

ZDisket commented Aug 13, 2020 •

edited

machineko commented Aug 13, 2020 •

edited

ZDisket commented Aug 15, 2020 •

edited

orikama commented Aug 15, 2020 •

edited

ZDisket commented Aug 15, 2020 •

edited

orikama commented Aug 15, 2020 •

edited

orikama commented Aug 15, 2020 •

edited